The last several weeks, I've been playing around with improving device and mesh monitoring on my hub. I've found, as probably many in this community already know, that nothing is as accurate as using the built-in Zwave and Zigbee Details "last message" Reports found under "Settings". These have been 100% right-on, while other approaches, such as using the "Last Activity State" for devices, has generally been not as reliable since that appears to rely on well written drivers.
So my question is whether anyone knows of an automated way of monitoring the Zwave/Zigbee Last Message information and possibly generating an alert message when a device doesnt report in a given length of time. I've looked around and havent found any existing Apps that can do this.
I already gave a try at writing a PC app to scrape the /hub/zigbeeinfo page, but the web page has a bunch of imbeded javascripts that appear to go and grab the data from the hub. I agree it would great if that information could be made available in some way. Also, it would be good to have the Z-Wave route information available (assuming that parameter serves as the best indicator of whether a Z-wave device is still talking to the mesh)
See the difference between the lastMessage and the lastActivity :
"lastMessage": "2024-06-07T20:29:05+0000", <- last check-in message from the device - now!
"lastActivity": "2024-06-06T16:52:13+0000", <- last event generated by the driver - yesterday..
I am hitting the zwave details json endpoint every time you view the main screen on my new SmartStart app and have not noticed any issues. Every hour even seems excessive anyway unless it is a critical device. I think somewhere in the 12-24 hours range would be good for most people.
I could probably do a patch on the "Device Activity Check" app and add the endpoints to it. That would actually be a nice upgrade. I already have a fork of that app going on GitHub.
Or were you thinking of adding it to your Health Check app @kkossev ?
Yeah I have found this also, even with some of the system generic drivers that filter events out instead of just posting them and letting the hub deal with duplicates. Its unfortunate.
They just added the last message time to the zwave details page so that would be the best. I tested it and it does update even when the last activity on the device page may not.
I'm getting a somewhat different result with "last message" from the zwave details page. Several of my zwave devices are not updating their "last message" entry unless they have had their driver activated, such as by turning them on or off. This behavior is despite their still being in the z-wave mesh, since when I do turn them on or off manually, at that time the "last message" entry entry will update. Based on this I'm assuming that the "last message" entry in the zwave details page is driver dependent...and not maintained at the zwave radio level. If that is true, then could the Z-wave route information entry from the zwave details page provide a better indicator of whether the device is still talking to the mesh? Or maybe it has the same problem?
For the Zigbee mesh, it appears to me that the "last message" entry in the zigbee details page is being kept updated with or without being updated by a device driver. So I assume those entries in the zigbee details page are being provided by a separate process running closer to the zigbee radio?
For sure it's all confusing and it would be good to have one definative source for each devices ZW/ZG mesh connectivity status, preferably collected as close to the radio communications handlers as possible and not device driver dependent. I don't know if that is possible in the current HA design.
For zwave, devices do not regularly send any messages so for something like a light switch it will be idle and never send anything unless you activate it or send a refresh.
Some drivers however maybe block the events from a refresh causing the last activity to not update, but the last message on details will.
Not a limit it’s just how the devices work. Zigbee tends to be more chatty. If those details were used on an app like Device Activity Check it has a feature to refresh devices before it sends an alert so it would work perfectly for Zwave and Zigbee.
Perhaps then the last Z-wave "last message" information can be saved between hub reboots, since right now it appears that this information is cleared at hub reboot and one can't tell how long it has been since the last activity. Perhaps Device Activity Check saves this information...I'll give it a look
I did some successful quick tests yesterday, it should be an easy task to add an additional column to the Device Health Status app for both the Zigbee and Z-wave devices. @bcopeland it would be great if a lastMessage column is added to the HE Matter Details page as well..
I have only a few Z-wave devices, and the lastMessage seems to be accurate for all my devices as well! But I am not a Z-wave expert and I haven't monitored the Z-wave devices long time enough.
It is not about filtering, but receiving periodical check-in or heartbeat messages that usually do not produce any hub events. These messages are an excellent indication that the device is healthy and online! I used to use a workaround in some of my drivers - resending 'powerSource mains' as an example when a heartbeat message was received and the device.lastActivity was more than 4 hours ago. Sending an event which is not state change leaves no trace in the events history, but updates the device.lastActivity and makes the Device Activity Check app happy.
lastActivity == last time when an event was generated by the driver.
it doesn't necessarily mean the device is alive and online.
I gave Device Activity Check a try but no joy. The "last message" information doesn't appear consistent between Z-wave details and the "last activity at" on the device details page. I also took a screenshot of the Device Activity Check report which is identical to the "last activity at" information.
The Z-wave details page does appear to have more current information which I verified was updated upon "refresh" by the Device Activity Check App. But I don't see that this updated information is being updated in the "last activity at" information on the device details page....and apparently this also affects the accuracy of the Device Activity Check App.
This all is pretty confusing I realize but I hope this helps. If only there was a way to grab the updated information on the Z-wave details page.
Correct, my suggestion was that the use of the last message info could be ADDED to that app, instead of making something new from the group up. Currently it only uses the device page Last Activity value. If the last message was also stale a refresh could be sent (already a feature of that app) which should surly cause a reply message if the device is online.
I will look into the proposals for Device Activity Check, but I'd encourage everyone to keep in mind that some "healthy" devices still do not regularly communicate with the hub simply because of how they work, so there will never be a perfect solution for all Zigbee and Z-Wave devices no matter what you try.
Good point. Still I think there are opportunities using what is available now and on network status info the hub could be made capable of providing. And hopefully as legacy devices are phased out the newer protocols can improve on the situation overall.
I know for me I've learned a lot the last few weeks exploring how to best monitor devices. I've found it to be a real challenge at times. sometimes with conflicting information sources. I've come to find the Zw/ZG details info in the hubs settings page to be the best source. Would be great if you can grab that info in your app.
@bertabcd1234 My thought of how to use it in your app would be to just have an option to use Last Message / Activity (new options instead of just the last activity). I would pull in both endpoints and populate all the info from there first, then any device with nothing in there (not populated or not z-wave / zigbee) then fall back to the last activity from the device page. Once you have that the rest of the app could function the same, refresh inactive devices and then check again, etc...
That's a good idea, as understanding what are the differences between lastActivity and lastMesage and deciding which one to use may confuse some HE users.
