This is not critical as I seem to have recovered but wanted to share this -
My configuration is three hubs - C4 and two C7's. The C4 basically just services an old Schlage lock. It runs Z-Wave only, Zigbee radio is turned off. One C7 runs Zigbee devices and connects to two Lutron Caséta hubs (Z-Wave radio is turned off), the other C7 runs Z-Wave devices (Zigbee radio is turned off). Hub mesh implemented (udp over ethernet) for the devices that need to talk to other hubs. All hubs on 2.3.3.140. Changes are rare and stability has been excellent.
This morning after I woke, I went to open the bedroom shades (Graber/Springs/Somfy - Z-Wave). The setup is Pico button at bedside, talks (clearconnect) to Caséta Pro Hub, Caséta Pro hub talks (telnet over ethernet) to C-7 hub #1, that Pico device is meshed to C-7 hub #3, hub #3 has a rule machine rule to process the button press and activate the shade accordingly.
I pressed the button - nothing. Tried the button on my wife's side of the bed - nothing.
Started checking things. I could see the correct telnet traffic coming in from the Caséta Pro hub on C-7 #1. I looked at hub #3 and I could see the RM rule was being fired and the hub thought the shades were opening. So I went directly to the device page for one of the shades and tried to activate it from there - nothing. At this point I concluded it must be a Z-Wave issue and I remembered that resetting the Z-Wave radio requires a shutdown and unplug. So I did. As soon as the hub came up I was able to control the shades both from their device pages and from the Picos.
Now I am wondering if I have a failing Z-Wave device somewhere that may be jabbering . . . or is it something else? I would appreciate any suggestions of things I should check.
I'm gonna guess you've already checked everything I'm about to suggest but here goes anyway...
Have you added/removed/moved any zwave devices recently? I'd also do the routine check for ghosts. A device with a really high response time or a large number of route changes might indicate an issue, though not necessarily a spamming one. And how about s0 devices? Any of those? Or Zooz 4-in-1 devices?
No Z-Wave device changes (additions or deletions). I still have to check all the devices to make sure they are all alive and working - most of my Z-Wave devices (except for the shades) are sensors so I don't notice a failure as quickly as I do with a device that won't activate.
No ghosts.
No S0 devices.
No Zooz (or Aeotec) 4-in-1 devices.
I am seeing a pretty high RTT on one of the window shades - 237ms. I will keep an eye on that.
I'll watch for route changes (since I just reset this morning, I believe those values were cleared).
I've got a bunch of those and never seen one cause an issue... but ya never know! Does anything else route through it or just that one shade? Unfortunately, it's not like you can unplug the ring extender and plug it back in since it has a battery backup!
I have 2 C7 hubs and plan to install a Lutron pro hub with some Picos scattered throughout the house. If I install the Lutron integration on both hubs will the Telnet integration increase the load on both hubs? Right now both hubs are lightly loaded.
I was considering only using the Lutron integration on one hub and using Hub Mesh like @Eric.C.Miller, but I am open to suggestions.
There wouldn't be anything in the engineering logs other than what you already know, and that is the "zwaveCrashed". The reason why it crashed is likely in the past logs. Excessive number of events from one or more misbehaving devices can lead to Z-Wave crashing. If there is nothing significant in logs already, you may need to enable debug logging for your Z-Wave devices. Also check for ghosts and stranded devices.
Last but not least, the fact that the problem resurfaced following an update is likely because when the hub reboots, every device checks in with the hub, thus the radio has to suddenly manage a lot more events at once, which may be too much for a radio that is struggling already.
