I have been using a C7 for years now with a pretty massive Z-wave network. For the last few weeks, it seems that overnight (maybe related to when the hubitat reboots as I have had it set to do nightly for years) I lose the entire Z-wave network. Not every night, but randomly.
If I look into the logs, it shows absolutely nothing - no errors, no complaints. If I open Z-wave details ALL of my devices are completely gone. If I open the Z-wave logs they are completely blank. Rebooting the hubitat does not fix this. Removing power and then restarting DOES fix this.
How can I troubleshoot this further to figure out if it is a hardware problem? Appreciate the help.
For logs, check the Hub Events and Location Events tabs. One of those tabs will show an error about zwave crashing if that is what happened.
I would stop the daily reboots, the hub should be able to easily make it 1-2 weeks or longer without rebooting. So if you feel like you need to keep it then please change it to weekly at least for now. If they hub cannot make it a week without rebooting then that is a sign of other bad things going on.
How is Hub being powered? Supplied power block or other (explain)
The hub is powered via power over ethernet, which has not changed in years.
Screenshots showing entire Z-Wave details Page
No need for a screenshot - the entire page is blank!
Screenshot of Z-Wave Topology Map (red/blue squares)
No map - it's blank!
So I was able to find this in the logs, under LOCATION EVENTS (WHY NOT IN THE LOGS?!?!?!)| This started after 8AM but the nightly reboot was uneventful at 4am.
There have been a couple of cases with sensor devices for some reason crashing the zwave on the hub. If you have any sensors turn on logging if not on already and then around the time the zwave crashes check the logs to see if there is a pattern of a certain device reporting in right before the crash.
Also, please look through my pinned post I linked to above, there are a bunch questions in there as well as info requested when asking for help with Zwave such as the zwave details page, etc... please provide all that info.
There are steps in there to do a backup/restore which cleans up the database, I would recommend that as well.
I believe they have it being posted in location events because that can be used as a trigger for actions (I think). The normal logs are just for info only and cannot be acted on (not easily anyway). It would be nice if it threw an error log in the normal log at the same time though.
What is the method used to "reboot" the hub? Scheduling a reboot can lead to unintended consequences like corrupting the hub's database or even the radio's database. We do not recommend using schedules to power cycle or reboot hubs. The need of such action is merely a band-aid covering other issues that should be resolved, not ignored.
A crashing radio is indicative of a hub in serious trouble. There could be any number of reasons that can cause the radio to fail, including hardware malfunction, but we often see this happening when one or more devices within the mesh network are misbehaving.
I'm a little uncomfortable posting the z-wave details here because it shows a lot of internal stuff...
Now that I have rebooted the hub, everything is fully operational as usual.
I can confirm that there are zero ghost devices, everything has a route, everything reports status ok, route changes are 0 for every device.
Here is the topology map. It's a pretty massive network. There are a lot of battery powered devices (my entire house has water, door, and window sensors running on z-wave).
Looks like your Z-Wave troubles are not new. Were you able to stabilize your Z-Wave mesh since May? Power outages can lead to critical radio failure over time:
Have had z-wave trouble since day 1 yes, would not recommend it to my worst enemy. However, it has been a long long time without issues other than this one. My hubitat is on battery backup, so power outages do not seem to be an issue for it. I have no reason to believe the mesh is unstable, other than these random radio crashes. Everything works perfectly and quickly otherwise.
A struggling mesh will eventually give up without addressing issues head on. Most users don't have Z-Wave problems, not since day one, and not after day 2190 as in my own case. If you'd like, you can send me your hub ID and I'd be happy to check your hub's engineering log to see if anything stands out to explain the problems you are having. As I am sure you've already done, it wouldn't hurt to review this post to make sure your devices aren't using too much of your Z-Wave radio bandwidth, which can lead to radio collapse:
I agree. I just think it's hidden in a very very counterintuitive place. I would expect an alert from the hubitat that shows up upon login (just like for an available software update) when the entire z-wave network has crashed. I would set things up so my wall tablets show a big fat sad face with the text Z-wave radio crashed, except it's pretty obvious when all the lighting and thermostats etc etc in my house abruptly stop working.
I really appreciate your time and expertise and will gladly message you the hub ID - thanks bobbyD! I do have a few polling events that run as well as a couple of power reporting devices, but have not found that they spam the network enough to cause issue - but I could be wrong.
