Can't get Apprise working with Home Assistant or MQTT

[kingpepe85]

Hello there!
I just installed birdnet pi and it works really great so far. Now I want to integrate it into my smart home with Apprise but it seems like nothing happens. I put in a hassios and a MQTT line but none of them seem to receive something. Here is my two lines:

hassios://192.168.188.42/******AiOiJKV1QiLCJhbGciOiVmNTg0MTk0MDZhYWE*****************************?port=8123 mqtt://192.168.188.51:1883/birdnet

(I took out a bit of my Access token from HA)

Does anyone have experiences with integrating HA or MQTT? Is there a way to debug the Apprise integration like logging or something? The log in the GUI doesn't say anything about the output of Apprise.

Thanks a lot in advance!

[mcguirepr89]

Howdy, @kingpepe85 -- thanks for installing BirdNET-Pi and for you interest in one of its newest features.

The issue you've encountered likely has to do with missing dependencies. According to the MQTT link on the Apprise repo, you'll need to add paho-mqtt to the Python virtual environment. You can do that with this command:

~/BirdNET-Pi/birdnet/bin/pip3 install -U pip wheel setuptools
~/BirdNET-Pi/birdnet/bin/pip3 install paho-mqtt

Let me know if that gets things working for you -- if so, I'll go ahead and add paho-mqtt to the requirements.txt file

My best regards,
Patrick

[kingpepe85]

~/BirdNET-Pi/birdnet/bin/pip3 install -U pip wheel setuptools
Ouput:

Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Requirement already satisfied: pip in ./birdnet/lib/python3.9/site-packages (22.1.1)
Requirement already satisfied: wheel in ./birdnet/lib/python3.9/site-packages (0.37.1)
Requirement already satisfied: setuptools in ./birdnet/lib/python3.9/site-packages (62.3.2)

~/BirdNET-Pi/birdnet/bin/pip3 install paho-mqtt
Output:

Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Requirement already satisfied: pip in ./birdnet/lib/python3.9/site-packages (22.1.1)
Requirement already satisfied: wheel in ./birdnet/lib/python3.9/site-packages (0.37.1)
Requirement already satisfied: setuptools in ./birdnet/lib/python3.9/site-packages (62.3.2)
birdnet-pi:~/BirdNET-Pi$ ~/BirdNET-Pi/birdnet/bin/pip3 install paho-mqtt
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Collecting paho-mqtt
  Downloading https://www.piwheels.org/simple/paho-mqtt/paho_mqtt-1.6.1-py3-none-any.whl (75 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 75.6/75.6 kB 1.0 MB/s eta 0:00:00
Installing collected packages: paho-mqtt
Successfully installed paho-mqtt-1.6.1

So to me it looks like wheel and setuptools are already there.

I then rebooted and now the MQTT works. I got a message with payload New BirdNET-Pi Detection A Cyanistes caeruleus Blaumeise was just detected with a confidence of 0.83986276.

Any idea about the Home Assistant part?

Thanks a lot so far!

[mcguirepr89]

So to me it looks like wheel and setuptools are already there.

Yep, it's just best practice to update those before installing a new module, as some modules will need them to build properly.

I then rebooted and now the MQTT works. I got a message with payload New BirdNET-Pi Detection A Cyanistes caeruleus Blaumeise was just detected with a confidence of 0.83986276.

That's great!

Any idea about the Home Assistant part?

Just looking through the documentation for that, I can't say that I see any pieces missing that would prevent this from working on the BirdNET-Pi side of things.

Just be sure that you are trying all of the valid syntax options -- I'm sorry but that is the most I'll be able to help with Home Assistant stuff :/

Syntax
Valid syntax is as follows:

hassio://{host}/{long-lived-access-token}
hassio://{user}:{pass}:{host}/{access_token}
hassio://{user}:{pass}:{host}:{port}/{access_token}
hassio://{host}/optional/path/{access_token}
hassio://{user}:{pass}:{host}/optional/path/{access_token}
hassio://{user}:{pass}:{host}:{port}/optional/path/{access_token}
By default hassio:// will use port 8123 (unless you otherwise specify). If you use hassios:// (adding an s) to the end, then you use the https protocol on port 443 (unless otherwise specified).

So the same URL's above could be written using a secure connection/port as:

hassios://{host}/{access_token}
hassios://{user}:{pass}:{host}/{access_token}
hassios://{user}:{pass}:{host}:{port}/{access_token}
hassios://{host}/optional/path/{access_token}
hassios://{user}:{pass}:{host}/optional/path/{access_token}
hassios://{user}:{pass}:{host}:{port}/optional/path/{access_token}
The other thing to note is that Home Assistant requires a notification_id associated with each message sent. If the ID is the same as the previous, then the previous message is over-written with the new. This may or may not be what your goal is.

So by default Apprise will generate a unique ID (thus a separate message) on every call. If this isn't the effect you're going for, then define your own Notification ID like so:

hassio://{host}/{long-lived-access-token}?nid=myid

[kingpepe85]

News from the Homeassistant: I'm getting this ouput when using the comand line with Apprise

birdnet-pi:~/.local/bin$ ./apprise -vvvv -t "testtitel" -b "test test" hassios://homeassistant-server/*******?port=8123
2022-05-27 15:46:47,931 - TRACE - HomeAssistant URL hassios://homeassistant-server/*******?port=8123 unpacked as:
user="None"
password="None"
port="None"
host="homeassistant-server"
fullpath=""
path="/"
query="*******"
schema="hassios"
url="hassios://homeassistant-server/*******"
qsd="{'port': '8123'}"
qsd+="{}"
qsd-="{}"
qsd:="{}"
secure="True"
verify="True"
accesstoken="*******"
2022-05-27 15:46:47,933 - DEBUG - Loaded HomeAssistant URL: hassios://homeassistant-server/e...Q/?format=text&overflow=upstream&rto=4.0&cto=4.0&verify=yes
2022-05-27 15:46:47,933 - DEBUG - Using selector: EpollSelector
2022-05-27 15:46:47,934 - INFO - Notifying 1 service(s) asynchronously.
2022-05-27 15:46:47,937 - DEBUG - Home Assistant POST URL: https://homeassistant-server/api/services/persistent_notification/create (cert_verify=True)
2022-05-27 15:46:47,937 - DEBUG - Home Assistant Payload: {'title': 'testtitel', 'message': 'test test', 'notification_id': '83d0d754-84f4-4087-8e56-11f222e7e403'}
2022-05-27 15:46:47,972 - WARNING - A Connection error occurred sending Home Assistant notification to homeassistant-server.
2022-05-27 15:46:47,973 - DEBUG - Socket Exception: HTTPSConnectionPool(host='homeassistant-server', port=443): Max retries exceeded with url: /api/services/persistent_notification/create (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f9a9f1d90>: Failed to establish a new connection: [Errno 111] Connection refused'))

Any idea about the connection error?

[mcguirepr89]

2022-05-27 15:46:47,973 - DEBUG - Socket Exception: HTTPSConnectionPool(host='homeassistant-server', port=443): Max retries exceeded with url: /api/services/persistent_notification/create (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f9a9f1d90>: Failed to establish a new connection: [Errno 111] Connection refused'))

This is a response from your HA server saying it refused the connection -- I would assume some sort of token or authentication will need be sent along in notification URL?

[mcguirepr89]

I noticed you used the secure hassios protocol -- does your HA installation serve SSL certificates? If so, were they issues by a recognized CA, or are they self-signed? If they're self-signed, you'll need to trust the HA CA on the BirdNET-Pi so that it accepts the self-signed certificates

[kingpepe85]

2022-05-27 15:46:47,973 - DEBUG - Socket Exception: HTTPSConnectionPool(host='homeassistant-server', port=443): Max retries exceeded with url: /api/services/persistent_notification/create (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f9a9f1d90>: Failed to establish a new connection: [Errno 111] Connection refused'))

This is a response from your HA server saying it refused the connection -- I would assume some sort of token or authentication will need be sent along in notification URL?

That's right. That's the access token i send along. I replaced it with the ******.

I noticed you used the secure hassios protocol -- does your HA installation serve SSL certificates? If so, were they issues by a recognized CA, or are they self-signed? If they're self-signed, you'll need to trust the HA CA on the BirdNET-Pi so that it accepts the self-signed certificates

The Homeassistant server has a Let's-Encrypt certificate. But the certificate goes for the external URL via Duckdns. While this could be an issue I get the same error when using the external URL in birdnet-pi. One thing I noticed is that the port 443 is being used (seen at the error message). But I specified the port 8123 in the command. Thought that query string would be the right way but could not find any detailed information about how to specify a port...

[caronc]

Are you serving home assistant on https?

You could update your url to be hassio://host:port (drop the s. Based on comments above, maybe it should read hassio://myha.server:8123 ?

(I'm simplifying the url, you still need to include your tokens, etc)

[jds11111]

Sorry to resurrect this old discussion, but having problems with MQTT notification. It looks super simple, and I have checked everything multiple times. Probably, I am just seeing what I want to see, and not what is really there. Here is my notification:

mqtt://username:password@192.168.1.91:1883/birdnet/sightings

Using these exact settings, I can connect to my mosquitto broker using MQTT Explorer.
I ran the python commands above about installing paho-mqtt, but it just says that Requirement is already satisfied for everything.
Here is the error when trying the test notification:

2024-02-10 14:56:30,272 - ERROR - Could not load MQTT Notification URL: mqtt://username:p...l@1...1:1883/birdnet/sightings

Of course, I have changed the username and password above. I did turn it off and back on again. Maybe somebody spots my stupid error.

[caronc]

What happens if you add more debugging to your command.

# add -vvv to increase verbosity
 apprise -vvv -b 'test' "mqtt://username:password@192.168.1.91:1883/birdnet/sightings"

[jds11111]

went to ~/BirdNET-Pi/birdnet/bin and ran ./apprise with your suggested switches. Got:

Exception ignored in: <function Client.__del__ at 0x7f9a6ebb80>
Traceback (most recent call last):
  File "/home/pi/BirdNET-Pi/birdnet/lib/python3.9/site-packages/paho/mqtt/client.py", line 874, in __del__
    self._reset_sockets()
  File "/home/pi/BirdNET-Pi/birdnet/lib/python3.9/site-packages/paho/mqtt/client.py", line 1133, in _reset_sockets
    self._sock_close()
  File "/home/pi/BirdNET-Pi/birdnet/lib/python3.9/site-packages/paho/mqtt/client.py", line 1119, in _sock_close
    if not self._sock:
AttributeError: 'Client' object has no attribute '_sock'
2024-02-10 17:00:04,529 - ERROR - Could not load MQTT Notification URL: mqtt://username:p...d@1...1:1883/birdnet/sightings
2024-02-10 17:00:04,529 - ERROR - There are no service(s) to notify
2024-02-10 17:00:04,530 - DEBUG - Using selector: EpollSelector

[caronc]

Interesting, This seems related to this ticket from 4 years ago. It seems to be related to a very old version of paho-mqtt installed into your environment as it was fixed upstream. How are you installing your instance of BirdeNet-PI. Is it a custom install? Docker container? Pi Build?

# maybe this will help
pip3 install --upgrade paho-mqtt

[jds11111]

That did not help. Also, it says that v2.0.0 is installed, which is much newer than what is in that ticket. The install is just the recommended way, on PiOS 64bit Legacy (not lite, though).

Linux birdnetpi 6.1.21-v8+ #1642 SMP PREEMPT Mon Apr  3 17:24:16 BST 202
3 aarch64 GNU/Linux

[Der-Sven]

Same issue here.
but got it partially to work:

BidNet-Pi comes with apprise 1.2.1 installed in .BirdNet-Pi/birdnet/bin/apprise
I moved this to apprise.old and installed the latest apprise version.
This executes without errors.

Still struggling on how to exchange the one that comes with BirdNet-Pi

[caronc]

The problem is v2.0 of the mqtt library recently released.

pip3 --install paho-mqtt==1.6.1

You may need to remove the current version installed

[caronc]

Downgrading Apprise is not necessary. The issue is with mqtt and it's (new) v2 release that isn't backwards compatible with its previous major release (the breaking change you're experiencing)

• source
• Apprise workaround for now which is just fixing the version to 1.6.1 so you don't have to do it manually (like you're having to do here)

[jds11111]

Aye, that did the trick. Thanks @caronc and @Der-Sven !

[ih8gates]

Sorry, but I'm having trouble understanding: what's the fix here? Do I run the commands that @Der-Sven mentioned previously in the thread? Will that roll back paho-mqtt? Or do I need to do some sort of uninstall/reinstall? (I've installed BirdNET and got it running, but not Python savvy). Thanks!

[Der-Sven]

Hi,
I have not tried @caronc 's version, as I got it to work and now I won't touch it anymore :-) But I do confirm, that you do not need to downgrade apprise, as birdnet installs v 1.2.1. which is rather old. (In fact, I updated mine to v 1.6.0 which is a downgrade against the latest release)

I have also modified the "requirements.txt" to match the versions above, in case any update script uses it.

For "my" solution, you don't need to uninstall anything, the last line will "update" it to the version specified, in this case 1.6.1.
It is crucial, that you activate the virtual environment, as explained, otherwise, apprise/birdnet will still use the wrong version.

Cheers,
Sven

[ih8gates]

Thanks, @Der-Sven. Your commands above did indeed fix things for me. MQTT is working as expected now.