Airthings-ble still unable to read after 2024.5.1 update #116770

JesusSanchezLopez · 2024-05-04T10:53:22Z

The problem

2024.5.1 update implemented retries for pulling data from airthings-ble sensor. This has provided major improvement for the connection, however I am still seeing an occasional disconnect.

From log:
2024-05-03 22:56:22.462 ERROR (MainThread) [homeassistant.components.airthings_ble] Error fetching airthings_ble data: Unable to fetch data: Disconnected from FC:A8:9B:F2:CA:CD
2024-05-03 23:43:08.505 ERROR (MainThread) [homeassistant.components.airthings_ble] Error fetching airthings_ble data: Unable to fetch data: Disconnected from FC:A8:9B:F2:CA:CD

I just now, I have enabled debug for bluetooth and airthings-ble hopefully I can catch additional details.

What version of Home Assistant Core has the issue?

2024.5.1

What was the last working version of Home Assistant Core?

No response

What type of installation are you running?

Home Assistant OS

Integration causing the issue

airthings-ble

Link to integration documentation on our website

https://www.home-assistant.io/integrations/airthings_ble/

Diagnostics information

No response

Example YAML snippet

No response

Anything in the logs that might be useful for us?

No response

Additional information

No response

home-assistant · 2024-05-04T10:53:26Z

Hey there @vincegio, @LaStrada, mind taking a look at this issue as it has been labeled with an integration (airthings_ble) you are listed as a code owner for? Thanks!

Code owner commands

Code owners of airthings_ble can trigger bot actions by commenting:

@home-assistant close Closes the issue.
@home-assistant rename Awesome new title Renames the issue.
@home-assistant reopen Reopen the issue.
@home-assistant unassign airthings_ble Removes the current integration label and assignees on the issue, add the integration domain after the command.
@home-assistant add-label needs-more-information Add a label (needs-more-information, problem in dependency, problem in custom component) to the issue.
@home-assistant remove-label needs-more-information Remove a label (needs-more-information, problem in dependency, problem in custom component) on the issue.

_{^{(message by CodeOwnersMention)}}

airthings_ble documentation
airthings_ble source
_{^{(message by IssueLinks)}}

JesusSanchezLopez · 2024-05-04T10:57:00Z

@jaydeethree: Are you seeing any issues after updating to 2024.5.1?

jaydeethree · 2024-05-04T15:48:11Z

I haven't updated to 2024.5.1 yet. I should have time to update and test this in the next few days.

bdraco · 2024-05-04T15:56:29Z

Which Bluetooth adapter are you using?
Have you followed the steps for reducing interference https://www.home-assistant.io/integrations/bluetooth/#simple-actions-that-should-improve-most-bluetooth-setups-and-common-root-causes-of-interference?
Do you have your Bluetooth adapter on an extension cable or it it plugged directly into your device?
How far away is the airthings device away from the Bluetooth adapter?
Please provide the full logs requested in Airthings-ble becomes flaky with HA Core > 2024.1.5 #113610 (comment) Airthings-ble becomes flaky with HA Core > 2024.1.5 #113610 (comment)

def1149 · 2024-05-04T16:27:46Z

After the update, I haven't had any instances of values becoming "unknown" or breaks in the data on the data graph in the last ~22 hours. Monitoring with DEBUG enabled continues

HA Yellow with CM4 w/ on-board BT
Wave+ 5 feet away through one US type sheet-rock wall

Note: a typical connection occurs every five minutes and takes 3.5 to 6.5 seconds, 95% about 4.5 seconds

I've seen one instance where retries recovered initial connection failure
2024-05-03 14:30:27.730 WARNING (MainThread) [homeassistant.components.airthings_ble] Timeout getting command data.
2024-05-03 14:30:27.731 WARNING (MainThread) [homeassistant.components.airthings_ble] Wrong length data received (0) versus expected (28)
2024-05-03 14:30:30.482 DEBUG (MainThread) [homeassistant.components.airthings_ble] Disconnected from F4:60:77:XX:XX:XX
2024-05-03 14:30:30.483 DEBUG (MainThread) [homeassistant.components.airthings_ble] Finished fetching airthings_ble data in 9.563 seconds (success: True)

I've seen five instances where retries recovered from unexpected disconnect
2024-05-03 15:46:46.272 DEBUG (MainThread) [homeassistant.components.airthings_ble] Disconnected from F4:60:77::XX:XX:XX
2024-05-03 15:46:54.486 DEBUG (MainThread) [homeassistant.components.airthings_ble] Disconnected from F4:60:77::XX:XX:XX
2024-05-03 15:46:54.488 DEBUG (MainThread) [homeassistant.components.airthings_ble] Unexpectedly disconnected from F4:60:77:79:82:82
2024-05-03 15:47:04.481 DEBUG (MainThread) [homeassistant.components.airthings_ble] Disconnected from F4:60:77::XX:XX:XX
2024-05-03 15:47:04.482 DEBUG (MainThread) [homeassistant.components.airthings_ble] Finished fetching airthings_ble data in 19.563 seconds (success: True)

I've seen three instances of the following with no apparent impact on connectivity
2024-05-04 01:08:35.928 DEBUG (MainThread) [homeassistant.components.airthings_ble.config_flow] Discovered BT device: <habluetooth.models.BluetoothServiceInfoBleak object at 0x7f57f085c0>
2024-05-04 01:12:00.189 DEBUG (MainThread) [homeassistant.components.airthings_ble.config_flow] Discovered BT device: <habluetooth.models.BluetoothServiceInfoBleak object at 0x7f4f995c40>

def1149 · 2024-05-04T16:48:29Z

I recommend a user-settable number of retries as users are effectively doing the same thing with automation hacks. This will allow users with a lot of interference to try to compensate. A user-settable timeout may also be useful

The current default settings worked for me but other situations may require more adjustability

JesusSanchezLopez · 2024-05-04T18:57:50Z

@bdraco
Which Bluetooth adapter are you using? - Raspberry Pi 4 Bluetooth adapter

Have you followed the steps for reducing interference https://www.home-assistant.io/integrations/bluetooth/#simple-actions-that-should-improve-most-bluetooth-setups-and-common-root-causes-of-interference? - Yes

Do you have your Bluetooth adapter on an extension cable or it it plugged directly into your device? - Integrated on Raspberry Pi 4

How far away is the airthings device away from the Bluetooth adapter? - Approx 20 feet

I caught 2 gaps at 7:56 AM and 1:30 PM (full log attached):

2024-05-04 07:56:03.493 DEBUG (MainThread) [homeassistant.components.airthings_ble] Finished fetching airthings_ble data in 66.608 seconds (success: False)
2024-05-04 13:30:06.429 DEBUG (MainThread) [homeassistant.components.airthings_ble] Finished fetching airthings_ble data in 43.543 seconds (success: False)

airthing.txt

JesusSanchezLopez · 2024-05-04T19:01:07Z

I recommend a user-settable number of retries as users are effectively doing the same thing with automation hacks. This will allow users with a lot of interference to try to compensate. A user-settable timeout may also be useful

The current default settings worked for me but other situations may require more adjustability

I agree, user configurable values would be great. Current settings work most of the time, but my hunch is one extra retry would make it work 99.99% of the time.

bdraco · 2024-05-04T19:20:08Z

Generally we don't add configuration options to change timeouts or retries as that is something the integration would handle. That type of change would be rejected in code review with a request to fix it in the integration instead.

Increasing to 3 retries is probably a reasonable path forward.

If we do that we should only do one retry on startup and than do the extra retries only after successful setup so we don't end up delaying startup.

bdraco · 2024-05-04T19:22:06Z

Do you have your Bluetooth adapter on an extension cable or it it plugged directly into your device? - Integrated on Raspberry Pi 4

How far away is the airthings device away from the Bluetooth adapter? - Approx 20 feet

While the drivers have improved over time, the internal RPIi adapters in these devices are not so great so its expected that you will get connection drops and since they are close to the USB ports that generate interference, the board design doesn't help.

The problem will likely completely go away if you disable the internal adapter and replace with an ESPHome Bluetooth proxy or an external adapter on the high performance list.

bdraco · 2024-05-04T19:29:54Z

Airthings/airthings-ble#38

fixes #116770

bdraco · 2024-05-04T19:34:01Z

#116805

bdraco · 2024-05-04T19:41:29Z

Since the scan interval is 300 seconds, we can even go to 5 attempts before declaring failure since the maximum connection time is 60s

JesusSanchezLopez · 2024-05-04T19:50:49Z

Thanks for increasing number of retry attempts! Ideally this issue is fixed for everyone with an RPi, but as a last resort I do have an esp32 set up as Bluetooth proxy waiting to be plugged in.

Thank you again for supporting us with sub-optimal setups!

JesusSanchezLopez · 2024-05-04T19:54:58Z

Since the scan interval is 300 seconds, we can go to 5 attempts before declaring failure since the maximum connection time is 60s

Wouldn't 5 retries in 300 seconds be too much since next scan would immediately happen? Max 4 retries?

bdraco · 2024-05-04T19:59:47Z

It would mean it would have to timeout connecting every time which isn't the problem here. If we start hitting that timeout, the device is likely truly unreachable. The data exchange timeout is much lower so it shouldn't be a problem

Also it can't miss the refresh as it always schedules the next one once the current one finishes

core/homeassistant/helpers/update_coordinator.py

Line 213 in 65120e5

def _schedule_refresh(self) -> None:

The risk of setting it too high would mean it would take a long time to notice the device is unavailable

def1149 · 2024-05-05T19:46:42Z

After 48 hours I've not had any data dropouts. For me, it looks like the retries have fixed the problem.
Thanks for the fix.

BTW, I have other BT devices even farther away (25 ft vs 5 ft for Airthings) that never experience data dropouts which casts some doubt on the USB interference theory. If you can't measure it, it's just a WAG (Wild Ass Guess).

It may be that the Airthings BT chip or device interface code is the problem. An engineer with the appropriate BT expertise and equipment can monitor/measure and get to the root cause.

But if that never happens hopefully the retry fix holds up over the long term. Time will tell and I'll be back if the problem reoccurs.
In the meantime, I'll keep my failure detection automation that triggers when C02 goes unknown for >60 seconds, sets a persistent notification so I know there was a data drop, and restarts the integration.

bdraco · 2024-05-05T19:51:16Z

BTW, I have other BT devices even farther away (25 ft vs 5 ft for Airthings) that never experience data dropouts which casts some doubt on the USB interference theory. If you can't measure it, it's just a WAG (Wild Ass Guess).

I don't know which other Bluetooth devices you are referring to, but that may be because Airthings devices need a GATT connection to update. Many other Bluetooth sensors use advertisements which do not have the overhead of two-way connection establishment, which also means they usually have a much longer range.

Bluetooth advertisements are generally limited to 31 bytes of data. Hence, some vendors use GATT instead or some combination of advertisements and GATT connections to update data that changes infrequently, like battery %.

Most vendors that use GATT connections have a stated range of 20ft. I'm not sure what Airthings specifies, though.

…116805) * Fix airthings-ble data drop outs when Bluetooth adapter is flakey fixes #116770 * add missing file * update

jaydeethree · 2024-05-06T15:26:20Z

@jaydeethree: Are you seeing any issues after updating to 2024.5.1?

I haven't had any drop-outs since updating to 2024.5.1 and switching back to the built-in airthings_ble integration, so it seems like the recent changes fixed everything for me :) Thanks so much @bdraco !

dxmnkd316 · 2024-05-06T23:41:28Z

I'm not sure if this adds any info to the ongoing issues, but last night I got annoyed and just decided to start over. Backed up the database, deleted both Airthings devices, deleted both Airthings apps on my phone, readded the devices in rhe Airthings BLE integration, and renamed the entities so they pointed back to the original entity names.

I have no idea if it was deleting the apps that helped or if it was deleting the integrations, but since then I haven't had any dropouts that weren't fixed by reloading the AT BLE integration. I'm still on 2024.5.1 and was only getting maybe one or two reading a day from one of my devices. I had an automation that would reload airthings BLE after ten minutes of unavailable then after another ten minutes would reload the Bluetooth integration then the AT BLE.

im wondering if the apps were somehow taking priority over the HA connection or if it was something else entirely. But I'm going to wait to update to 5.2 to see if I start having dropouts after a day or two.

JesusSanchezLopez · 2024-05-07T10:28:22Z

@dxmnkd316 - I have both Airthings apps installed on my phone (iphone), so I don't think that has any impact.
2024.5.1 fixed majority of the drop outs for me (only had 2-3 missed data points over 24 hours).
I installed 2024.5.2 last night and have not seen any drop outs over last 12+ :)

Thank you again @bdraco for all your help!

dxmnkd316 · 2024-05-07T13:41:11Z

So I might just have a bad connection in the original location. Which is odd, because there are other Bluetooth devices on the far side of that room that connect fine.

Anyways, I started to see dropouts again later in the evening. So I moved the device two feet from the bedroom to the hallway, towards the raspberry pi. I haven't had a drop since.

Perhaps more testing on my part is due. But unsurprisingly it seems like bdraco and lastrada were onto something about the flakey connection and the timeouts. I might have found the edge of the range.

JesusSanchezLopez · 2024-05-07T14:44:30Z

@dxmnkd316 Upgrade to 2024.5.2 and see if your issue is resolved. The extra retries seem to have resolved my issue without moving either RPi or Airthings Wave+ (approx 20 feet apart). I had no drop outs in last ~19 hours since I've upgraded to 2024.5.2

def1149 · 2024-05-07T15:38:55Z

Since 2024.5.1 I've had no dropouts. I'm on 2024.5.2 since release

dxmnkd316 · 2024-05-09T23:47:22Z

It's been three days since i've had any dropouts. I decided to update to 2024.5.2 as well.

Great work @bdraco & @LaStrada

home-assistant bot added the integration: airthings_ble label May 4, 2024

bdraco added a commit that referenced this issue May 4, 2024

Fix airthings-ble data drop outs when Bluetooth adapter is flakey

df61a69

fixes #116770

bdraco mentioned this issue May 4, 2024

Fix airthings-ble data drop outs when Bluetooth connection is flakey #116805

Merged

20 tasks

balloob closed this as completed in #116805 May 5, 2024

balloob pushed a commit that referenced this issue May 5, 2024

Fix airthings-ble data drop outs when Bluetooth connection is flakey (#…

d970c19

…116805) * Fix airthings-ble data drop outs when Bluetooth adapter is flakey fixes #116770 * add missing file * update

frenck pushed a commit that referenced this issue May 6, 2024

Fix airthings-ble data drop outs when Bluetooth connection is flakey (#…

ae28c60

…116805) * Fix airthings-ble data drop outs when Bluetooth adapter is flakey fixes #116770 * add missing file * update

github-actions bot locked and limited conversation to collaborators Jun 9, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Airthings-ble still unable to read after 2024.5.1 update #116770

Airthings-ble still unable to read after 2024.5.1 update #116770

JesusSanchezLopez commented May 4, 2024

home-assistant bot commented May 4, 2024

JesusSanchezLopez commented May 4, 2024

jaydeethree commented May 4, 2024

bdraco commented May 4, 2024

def1149 commented May 4, 2024 •

edited

Loading

def1149 commented May 4, 2024 •

edited

Loading

JesusSanchezLopez commented May 4, 2024

JesusSanchezLopez commented May 4, 2024

bdraco commented May 4, 2024 •

edited

Loading

bdraco commented May 4, 2024

bdraco commented May 4, 2024

bdraco commented May 4, 2024

bdraco commented May 4, 2024 •

edited

Loading

JesusSanchezLopez commented May 4, 2024

JesusSanchezLopez commented May 4, 2024

bdraco commented May 4, 2024 •

edited

Loading

def1149 commented May 5, 2024

bdraco commented May 5, 2024 •

edited

Loading

jaydeethree commented May 6, 2024

dxmnkd316 commented May 6, 2024 •

edited

Loading

JesusSanchezLopez commented May 7, 2024

dxmnkd316 commented May 7, 2024

JesusSanchezLopez commented May 7, 2024

def1149 commented May 7, 2024

dxmnkd316 commented May 9, 2024

Airthings-ble still unable to read after 2024.5.1 update #116770

Airthings-ble still unable to read after 2024.5.1 update #116770

Comments

JesusSanchezLopez commented May 4, 2024

The problem

What version of Home Assistant Core has the issue?

What was the last working version of Home Assistant Core?

What type of installation are you running?

Integration causing the issue

Link to integration documentation on our website

Diagnostics information

Example YAML snippet

Anything in the logs that might be useful for us?

Additional information

home-assistant bot commented May 4, 2024

JesusSanchezLopez commented May 4, 2024

jaydeethree commented May 4, 2024

bdraco commented May 4, 2024

def1149 commented May 4, 2024 • edited Loading

def1149 commented May 4, 2024 • edited Loading

JesusSanchezLopez commented May 4, 2024

JesusSanchezLopez commented May 4, 2024

bdraco commented May 4, 2024 • edited Loading

bdraco commented May 4, 2024

bdraco commented May 4, 2024

bdraco commented May 4, 2024

bdraco commented May 4, 2024 • edited Loading

JesusSanchezLopez commented May 4, 2024

JesusSanchezLopez commented May 4, 2024

bdraco commented May 4, 2024 • edited Loading

def1149 commented May 5, 2024

bdraco commented May 5, 2024 • edited Loading

jaydeethree commented May 6, 2024

dxmnkd316 commented May 6, 2024 • edited Loading

JesusSanchezLopez commented May 7, 2024

dxmnkd316 commented May 7, 2024

JesusSanchezLopez commented May 7, 2024

def1149 commented May 7, 2024

dxmnkd316 commented May 9, 2024

def1149 commented May 4, 2024 •

edited

Loading

def1149 commented May 4, 2024 •

edited

Loading

bdraco commented May 4, 2024 •

edited

Loading

bdraco commented May 4, 2024 •

edited

Loading

bdraco commented May 4, 2024 •

edited

Loading

bdraco commented May 5, 2024 •

edited

Loading

dxmnkd316 commented May 6, 2024 •

edited

Loading