V4.4.0 - Vitals and FleetAPI #479
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
@jasonacox Are the alerts supposed to look like they're active from now until infinity or is that just a consequence of adding them back in. Seven strings again! Finally!!! |
|
jasonacox/pypowerwall:0.10.0t58-beta9 Any way to provide some better error reporting so we can figure out what is causing these issues, even though they're intermittent? i.e., is the packet malformed or something else getting corrupted? |
These are active alerts from the Powerwall. They stay active until the Powerwall clears them. This is what we had before (with vitals). Below is a glimpse of my system for past 24 hours. The ones that are solid all the time make sense (e.g.connect to grid) while others are intimement (e.g. solar RGM comms issues) which could indicate a problem. The RGM comms is something that caused me troubles in the past and I lost the view when vitals went away. The alert time series graph is different from others. A good way to think about it is that you have a dashboard with a bunch of indicator lights that can flicker on and off. You may miss them change state. But when it is stretched over a time graph, you can see when those blips happen. 
Actually, what you provided is perfect. I know what happened (line 400 of the tedapi library) where one of the JSON walks ('esCan', 'bus', 'PVAC') has a step that is None. I can fix that. Thanks for sending that! Let me know if you see others. 🙏 |
|
Here's some more that looks slightly different in case that's helpful: |
|
Thanks @jgleigh - those are caused by the same error, unfortunately. It's really blowing up big when it goes. I pushed a patch and now a new version if you want to try it: |
|
For any who want to help beta test before 4.4.0 is released: Testing - Using pyPowerwall with VitalsAdd Network RouteFirst, you need to set up the host that will run your Dashboard to have a route to your Powerwall Gateway. This will vary based on your network and OS. If your host is able to ping your Powerwall on your LAN (e.g. 10.x.x.x) you can do something like this: # Linux & RPi
sudo ip route add 192.168.91.1 via <Powerwall_IP>
# MacOS
sudo route add -host 192.168.91.1 <Powerwall_IP>
# Windows
route add 192.168.91.1 mask 255.255.255.255 <Powerwall_IP>
# Test
ping -c 1 192.168.91.1
# Note: This is a temporary route. To make it survive a reboot, you need to set it in the system
# or add it to something like /etc/rc.local Update Dashboard InstallEdit the Edit Apply the changes using: ./compose-dash.sh up -d
# and to tail the logs... ^C to quit
docker logs pypowerwall -fWelcome Back VitalsEnjoy getting extended vitals metrics back: Strings, Powerwall Capacity, Alerts 
|
|
@jasonacox Still getting errors: Also noticed all the meters are reporting as solarRGM and null firmware, which isn't correct. Not sure if this is a parsing issue or something else. |
|
Just followed the beta test instructions on my side - the data is back!
This error seems to occur in the logs every few minutes: |
|
@jgleigh - The problem is that we actually are not getting Neurio firmware or location (e.g. SolarRGM) data. The SolarRGM was my bad as I just copied in my old vitals JSON structure and forgot to remove the placeholder. Having said that, can you pull the raw TEDAPI data for me for your Neurios? I added this API to the proxy which will show you the raw data: Both have Neurio data but not exactly what we had before. I only have 1 so it is a bad sample size. @richbaron - Thanks for testing! I will take you up on your offer for instructions! Feel free to post here and I'll include in a helps doc or Discussion area for reference.
Yes, for you and @jgleigh - I'm an idiot! 🤦 I changed the logic to handle missing keys in the TEDAPI response but not if the response was Null. I addressed the core |
|
Here you go. |
|
Thanks @jgleigh ! I got them but removed the links for privacy. Important parts: status.json config.json I can use the serial numbers to match the device to get "location". Unfortunately, it doesn't look like we can get firmware. |
|
The locations in config looked correct. I have four active meters. Two on solar, one in the gateway, and one site on a subpanel. So the gateway plus the site meter equals the real total load. |
|
@jasonacox Okay back to the usual corrupted packets. Anyway to better see why these are getting rejected? 2024-06-05 16:32:29 06/05/2024 04:32:29 PM [proxy] [ERROR] Socket broken sending API response to client [doGET]: [Errno 32] Broken pipe |
@jgleigh how is your gateway connected? WiFi or hardwired (Ethernet)? If WiFi, this behavior could be explained through the connection being unstable. I have not seen a similar error since going to either beta release, but my gateway is hardwired. |
|
@richbaron Both, but using hardwired for the proxy currently. This has been happening for quite a while, but trying to figure out where the error is getting injected into the packets. I'm curious, does your Ethernet cable enter from the top or the bottom of the Gateway? Mine is routed from the bottom and has to pass by all the AC lines before getting up to the RJ45 jack. @jasonacox New error popped up today. |
@jgleigh This is low level python code that handles https calls. Some quick research and it shows these as potential causes:
I can't replicate it and there doesn't appear to be any way for me to adjust the code to better handle this that wouldn't just mask the problem. The error is indicating something is truly wrong. I'm not sure if that helps you, but I would suggest trying:
I have been thinking about dropping these to DEBUG logging levels because they can be confusing. They are typically harmless and everyone should be getting them occasionally anyway (e.g. force closing your browser that is rendering the powerflow animation can cause this if it hits at the same time one of the 5s updates hit). It just means the client hung up before getting the payload. Another (more likely) possible cause is that the Powerwall takes longer than 5s to deliver the packet, at which time telegraf (the client in this case) would timeout and try again. Pypowerwall would log the timeout as a disconnect. Unlike the above error, I don't think these provide actionable help. |
|
@jasonacox Okay, still pointing to possible bad cabling on my side. Maybe I'll run a cable test this weekend. |
|
@jasonacox - this is brilliant! Nice work. Sorry I have been out of the loop for the past few weeks. I wish I had more time to be able contribute and help with review/testing, however too much going on IRL at the moment though. 😞 Testing TEDAPI support with jasonacox/pypowerwall:0.10.1t58-beta4 and it appears to be working great! I'm not seeing any problems on my setup.
|
|
Thanks @mcbirse (and no worries)!! That's looks great! thanks for testing. @jgleigh I updated the debug information to provide less ambiguous messages during these client disconnect events. It should still record any error level event that isn't the cause of the client hanging up early. That is the only change in the latest beta: I'm planning on pushing the pypowerwall beta patch (0.10.1) to latest and merging this PR today. I fully expect to see some edge case fixes we will need, but at this point I think we have found most of them. Thanks for the help, everyone! |
|
Hey @jgleigh - I forgot to add your discovery about the Neurio block. I added logic to pull the correct location. Also, I noticed the power scale factor was not being applied. I added that so the CT RealPower values will match the aggregate power data now (hopefully). Sadly, still no way to get the firmware version (at least I haven't found it). Please give this latest image a test and let me know how it looks (http://localhost:8675/vitals): Vitals "NEURIO--VAHxxxxxxxxxxx": {
"NEURIO_CT0_InstCurrent": 4.820000171661377,
"NEURIO_CT0_InstReactivePower": 1.7999999523162842,
"NEURIO_CT0_InstRealPower": 1132.5799560546875,
"NEURIO_CT0_InstVoltage": 121.12999725341797,
"NEURIO_CT0_Location": "solarRGM",
"componentParentDin": "1232100-00-E--TG121048001E4G",
"firmwareVersion": null,
"lastCommunicationTime": "2024-06-08T08:39:00-07:00",
"manufacturer": "NEURIO",
"meterAttributes": {
"meterLocation": []
},
"serialNumber": "VAHxxxxxxxxxxx"
} |
|
@jasonacox Looks good. I didn't see any firmware version reporting in all the tedapi data. Looks like they removed it unfortunately. |
|
Thanks, Jeff! Non-beta image: |
jasonacox
commented
Jun 8, 2024
| echo " 2 - Tesla Cloud (Solar-only systems or Powerwalls without LAN access)" | ||
| echo " 1 - Local Access (Powerwall 1, 2, or + using the Tesla Gateway on LAN) - Default" | ||
| echo " 2 - Tesla Cloud (Solar-only systems or Powerwalls without LAN access)" | ||
| echo " 3 - FleetAPI Cloud (Powerwall systems using Official Telsa API)" |
There was a problem hiding this comment.
SETUP SCRIPT: This release adds the ability to use FleetAPI. If selected, it will kick off the FleetAPI setup process. If FleetpAPI is set up, the pypowerwall will always prefer that over cloud (Tesla Owners unofficial API) as FleetAPI is a Tesla provided official API.
jasonacox
commented
Jun 8, 2024
| if [[ "$response" =~ ^([nN][oO]|[nN])$ ]]; then | ||
| IP="" | ||
| else | ||
| echo "Congratulations!" |
There was a problem hiding this comment.
SETUP: If local mode is selected, the setup script will attempt to find the Powerwall gateway at the default IP (192.168.91.1) and ask the user if they wish to set up the TEDAPI extended device vitals metrics. Otherwise, it will prompt the user for the IP address (or to scan).
This means that the user needs to set up the network routes to the PW Gateway prior to running setup.sh.
|
UPGRADE SCRIPT note: The upgrade script does nothing to switch to using FleetAPI or TEDAPI. This is by design. Upgrade should only update the version but keep the same mode. To switch modes, users will need to re-run |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Updates
TEDAPI Support for Device Vitals
/vitals) - This requires connecting to Powerwall WiFi directly or setting up a network route on the Dashboard host to allow it to reach the GW address (192.168.91.1). See discussion (more helps soon).upgrade.shand then runsetup.shto choose these new options.FleetAPI Support
upgrade.shand then runsetup.shto choose these new options.Other Updates
Upgrade to pyPowerwall v0.10.0 proxy t58 - Release addresses some issues, including fixing Solar Only grid_status issues as reported by @lsgc123 in Power Flow Always Shows Off-Grid - Solar Only #478
Fix setup.sh for docker group permission bug identified by @hulkster in Minor bug in setup.sh on docker group test if docker is the FIRST group returned by "id -Gn" #476
Addresses several open issues and discussions:
The Return of Vitals