Hi Guys looking for some help/advice. I "upgraded" my hub a couple of weeks ago from a C8 hub and have been suffering from very poor zwave performance ever since. My system regularly misses or has very slow response to some trigger events. This is particularly noticeable on a couple of contact sensors one of which is only 1 metre away from the hubs location.
I have confirmed that its a zwave issue as I have fed one of my gates input simultaneously into a Figaro zwave smart implant and frient zigbee I/O module. When the system plays up the smart implant totally misses the gate opening but the zigbee module immediately changes state.
For completeness everything seemed to go ok with the migration except when first booted none of the zigbee devices responded to commands but that seemed to rectify themselves
after a couple of hours and Zigbee performance now seems fine.
After reading numerous similar posts I have tried a soft reset, z-wave repair etc but still having the same issue.
I have just shutdown the hub and powered down for 5 mins but I am still having issues.
I am now at the point of pulling my hair out as I cant seem to find a way forward.
Does anyone have any suggestions? I am thinking that the hub is somehow defective but having purchased this from a retailer in the UK so getting a replacement will be difficult as they are notorious for not accepting refunds and was hoping that this post with any subsequent troubleshooting steps may help grease the wheels.
As a side note I was thinking of signing up to a hubitat subscription to get some help from support but given the fact that the warranty offer wont apply as we have to purchase through a retailer in the UK. I will only do this if hubitat support will help remedy this issue. Will support get involved with this issue if I have a subscription?
Fingers crossed that one of you smart folks can help me figure this out.
Thanks
Please provide your hub model (C7, C8, etc.) and its platform version from Settings>Hub Details.
Check out the following post for help troubleshooting problems and gathering details that will help others to identify and solve the problem you are experiencing: ‼ READ FIRST - Before Posting in Get Help
Those two, 0x10 & 0x13 have only one neighbor and it's possibly each other, which is why they can't provide info to the hub. It's possible, being battery driven that they just don't wake up often and the other efforts you're making is just delaying their recovery.
Your eyes and the hub are not going to follow the same path BUT you need to be able to see a path of blue squares that get back to Node1 (the hub) Take 0x38 as an example, it has no direct connection to the hub, but you can follow a blue path. From 0x38 on the left.. step right to the first blue square and then go UP to the top line and see that 0x06 has a blue square for Note 1. That says there's at least one path. It turns out 0x38 has 9 blue squares horizontally and all of them have a blue square at the top #1 row.
All the rest seem to have more than one neighbors and zero Route Changes. Both are good values to see.
Maybe first try a full z-wave repair. After its done might be good to shut down and unplug for 30 seconds to restart the radio.
Once its back up again manually active some of the devices near those two (x10 and x13), and make sure they are getting events to the hub. Once that is working manually wake those two devices up a few times (wait a full minute between each one). There should be a wake up procedure in the documentation for the devices. Forcing the wake up should make them try to re-route if they cannot get a good route.
Some battery devices I have found also just always report 1 neighbor, but in my case it is the hub so they have a blue mark for 01. Sometimes it will use other routes but still only shows 1 neighbor.
Worst case if you have some problem devices you can exclude and pair them again (there are tricks to swap your apps so you dont lose any automations).
OK quick update based on your comments node 0x10 I knew the battery was dead and kept meaning to remove it but got tied up trying to figure out what was going on but I have now removed..
Node 0x13 is a battery operated switch which has a direct connection to the hub (would have been helpful if I hadn't snipped the last columns off but just in case I excluded and reinstalled it and again it has a direct connection to the hub.
The two nodes that I am really noticing an issue with are 0x10 and 0x16. Given that both of these device are reporting 10 neighbours and 0 route changes is this counter intuitive??
Anyway I have just done a full z-wave repair, shutdown and unplug. Will update on progress..
OK guys (@jtp10181 & @csteele) so the plot thickens. What do you make of this...............
So I tested one of the troublesome sensors tonight (forgot to mention that the issue seem to get worse at night. Not sure that's relevant both thought it was worth mentioning)
Guess what the hub totally ignored the sensor being triggered in both the open and closed direction and this was repeated multiple time however this time I was able to use a Sniffer to capture the traffic.
Disclaimer I am not a Zwave expert but surely the third line of the traffic snippet confirms that the hub saw the trigger @ 19:32:58 (first line) as it has generated a response back to the sensor. However the sensor events in the hub show that the time this sensor was in an open condition was 12:41:36 almost 7 hours previously
Therefore I know think this actually suggests that although I was thinking this was a ZWave performance issue it actually appears to be an internal hub problem as the hub clearly responds to the radio message but the does nothing to process it.
Forgot to mention that I use @thebearmay hub information driver (great tool) and this is the hub load shown at the time the above event was triggered. Although not 100% conclusive this would suggest that it was not a hub overload issue wouldn't it
Turn on debug logging on the device in Hubitat and repeat the experiment. See if the driver logs anything in the Hubitat logs on these events you see on the zniffer.
That could help tell if it is a driver issue or a hub zwave subsystem issue. Or both or neither.
OK so after an hour or so of repeatable missed events it suddenly starts working again without actually changing anything (For what its worth this has been my previous experience) however thought I would show the Zniffer/hubitat logs when everything works ok:
I think it's relevant to mention that the low level communication is being handled by the ZWave Radio SOC, not the hub CPU.
There are plenty of conversations where we call the Radio "stuck" and suggest a power cycle to unstick it. I can't say I have any knowledge of how the Radio SOC <--> Platform CPU interact in 2024, years back it would have been at a level well above the detail shown in a sniffer trace. Effectively, a sniffer trace is monitoring the SOC. Certainly the 'orchestration' of the communications is done by the platform, but the ACKs and retries would very likely be completely the SOC's task.
@csteele thanks for the insight. Reading between the lines I think your confirming that you don't think its the radio hardware causing the issues as the Zniffer logs confirm that the radio is reading and acknowledging the transmission but it would appear that no further processing is happening on the missed events.
Update for today - I have been testing the back door contact (the sensor related to all the logs etc above) worked all day and it hasn't missed a beat. However I checked a different sensor which is less than 2 meters from the hub and again the hub totally misses it but the Zniffer log both the transmission and acknowledgment from the hub
However I think I am now at the point where I have lost all confidence in this hub and I cant see a way forward and therefore I think I am going to consign it to the bin and try and get my system to some kind of reliable state.
To that end I want to put my original C8 hub back in service. Therefore does anyone know if it possible to use the migration tool/process to migrate from a C8 pro hub back to a standard C8 hub? Unfortunately I wiped my C8 hub with the intention of replacing my old C7 that is being used in my hub mesh but at least I know its in good working order
If you migrate back the problems will probably follow it. The C8 Pro and C8 are basically identical except for the main SOC processor and ram. So whatever is messed up is probably in the radio database or hub database (which both get copied for a migration).
I am feeling like the issue was created in the zwave database somewhere during the migration process. If you were to reset the radio and pair all devices again the problems will probably be gone.
You could migrate back, but only using the paid hub protect cloud backups. The free tool only lets you go forward. If you do a warranty claim maybe they would grant you a one time backwards migration as a solution?
@jtp10181 Yeah I think what you're saying make sense. If I have to go through the pain of pairing/unpairing all the devices I might give the C8 pro another shot.
If I reset the radio and repair do you think I can replace the existing devices so apps/automations can be "saved" or do you think this risks allowing the gremlin to survive?
Yes, mostly. It is a tedious procedure and if not followed correctly you can make a mess of things. I would have to dig up another post explaining it. Swap Apps can be used on singular devices. Anything with child/parent devices you have to swap manually.
I feel like the radio reset would solve the issue, its possible its in the hub database but track record is telling me its z-wave.
One other thought, if the cloud migration backup is still on the server (it should be). You MIGHT be able to load that back onto the C8 and it might go back to working as it was before. I think you would need hub protect or the help of staff to allow it to be loaded.
Yup I was going to suggest this also but forgot. If you have hub protect you could reset the hub, mess around, then restore back to your current sort of functional state.
IF you have a Cloud Backup, of the "today" configuration, then you should be able to restore it. In fact, you can literally try it but that needs Hub Protect.
With that cloud backup in hand, you can follow the factory reset recipe and have a blank hub. Add a couple troublesome devices.. see if the results are what you want.
but that needs Hub Protect.
