This morning at about 4am I noticed that my hall lights didn't come on, though the green indicator light on the motion detector flashed. I checked my zwave details page on my main hub and probably 80% of my zwave devices no longer had routes! I shut the hub down, powered it off for a minute or so, and powered it back on. It looks like that has resolved the immediate issue but I am concerned I don't know why it happened.
I checked the logs and can't see anything obvious. No zwave busy message, no unusual or excessive device activity from zwave devices, etc. However, I do reboot my hub every Sunday morning and it appears the reboot was in the suspect time period, somewhere between about 1:30am and 4am. The reboot is at 3:15.
I made a whole bunch of changes to my HE environment yesterday so it's a bit difficult to say if I introduced something that caused the issue. Here's what I recall doing:
I moved a bunch of CPU-intensive activity OFF my main hub onto a secondary. Most of this was rules that repeatedly ping my iphone to see if it's still there. (Yeah, I know... life360. But apparently tracking household members isn't acceptable.) I don't see how this could possibly have done anything but help the primary hub.
I excluded and removed a couple of Fibaro flood sensors. They were not paired s0 but they were also not reporting temperature. I did not re-add them since they're a pain to include without security. I excluded and re-included a strips water sensor because it wasn't reporting temperature. And I added a couple of zigbee leak sensors to replace the Fibaros. After removing and adding zwave devices I always check the zwave details page to make sure I haven't introduced any obvious issue, so I know it was clean then. The strips sensor is reporting temperature and seems to be working fine. I do generally practice good zwave hygiene though I have a couple of outlier devices that are a bit weak.
I updated the hubs from 2.3.0.118 to 2.3.0.119. I did this mid-day and everything looked good afterward. There's nothing in the release notes about any zwave changes, only fixes for database corruption. I haven't seen any evidence of that, nor can I explain why it would have affected zwave routes and nothing else, so I have to think the .119 upgrade was a coincidence, but who knows.
Any ideas or additional troubleshooting steps?