[2.3.5.113] C8 Z-Wave dropout at random times of the day

We are officially off of the POE adapter, and running the OEM power cable. So far everything is good. Devices are nice and responsive, and everything has been updating well. Going to holdout on updating to the 2.3.5.121 update, just to only change one variable at a time. Time will tell whether this fixes things or not. Looking forward to hearing others experience swapping out as well!

3 Likes

A combination of the hub info driver from @thebearmay and Hubigraphs, which I think is still available via HPM

2 Likes

OMG OMG OMG I was actually able to use my ring security keypad this evening to disarm my alarm system! Big plus for the police not showing up!

Not ready to declare victory yet but positive signs ditching the PoE splitter. 4am bathroom run (hey i’m of that age) will seal the deal.

2 Likes

I have had very good results since removing the hub from the POE splitter. Everything is snappy and responsive. Ghost nodes that couldn't be removed were easily dispatched. It's only been about 4 hours and I intend to test it for several days but so far it is working far better than it ever has. For the last 4 hours I have been happy I migrated to the C8, even my 3 Schlage locks are working the best they ever have. Finger crossed!!!

2 Likes

I am using a Unifi 8 port POE, which is powering 5 Hubitats and 2 Harmony Hubs. No issues with it prior to the C8s being introduced.

Its just been a few hours for me, but things so far seem better too. Devices seem more responsive, and the delays that happened when toggling a device seem to be reduced. Dashboard also updates more quickly, showing a switch has been turned on or off.

So far, its been very good!

1 Like

This morning things still seem pretty good. I did have a slow response from my ring keypad this morning but it did eventually respond and it functioned correctly all night. No 5am blackout either. I did also move backups to 9am so we'll see but so far things are looking a lot better.

Just had the same z-wave dropout I've been having... so I have to say at least in my case it is not all the PoE sensor.

Multiple z-wave devices are not reporting in though I can see by their indicator lights they are sensing motion.

I changed the backups to kick off at 9:15, the spike occurs shortly thereafter, and my z-wave network ceases to function. I also noticed that although the local backup ran, the cloud backup failed.

At this point I think I can say pretty conclusively that something in the scheduled backup process is causing my z-wave network to fail. That may not be the ONLY problem but it is definitely A problem

You may want to try and isolate it between local and cloud backups. At least in the past only the cloud backups did the radio NVM backup, so it could be that only the cloud backups are triggering it.

1 Like

A simple reboot seems to be restoring functionality.

And yes that's an excellent idea. I have disabled cloud backups and I'll try it manually.

Well... neither a cloud nor a local backup executed manually seems to cause the issue. But I noticed you cannot set the two to automatically run at a different time. Perhaps it is the confluence of the two causing the issue? Or perhaps when you fire off a manual backup the same cleanup does not occur?

I'll try disabling one for a while and then the other.

@bobbyd is there anything you guys could gain by looking at the engineering logs and seeing what's going on during those automated backups?

Yeah ring keypads offline again. PoE splitter was not the issue. I think my earlier good news was just the placebo effect. Definitely a hub process or something related breaking z-wave,

Well I am happy to report my system is still going strong since I swapped away from the POE splitter. Everything this morning was where it should have been (on/off), and all my motion lights have been responding well. My door sensors (battery) have been reporting well too, and I am no longer seeing issues where it thinks doors are open when actually closed, etc.

I take it with the cloud backups you are subscribed to hub protect? I am not a subscriber, so my backups are only local. I have mine set to run at 3:15am daily, and it seems it didn't break anything.

I would recommend what you mentioned of disabling cloud backups and running without the POE splitter for another day or two to see how things respond. Hopefully it helps, but I'm guessing it might not since your keypads are being problematic as well. Also, what firmware are you running? I am still on .118, as I wanted to see the effect from only changing the splitter. It could be a couple things causing issues with Z-Wave performance; the splitter potentially being one, and something with cloud backups being another. Hopefully the Hubitat team can review your logs and find something in regards to a fix.

I'm going to run mine for another day in its current state, and if it continues working well I am going to order the POE splitter @JasonJoel shared. Might be another day or two before it arrives, so I should know for sure whether it maintains stability or not by the time I swap sources again.

1 Like

Yeah I concur. That seems like a good plan.

121

1 Like

I don't have a C8 Hub however I do run my hub with a PoE splitter. One thing I have noticed with the few that people have posted is they seem to all be 2.4A splitters. You might consider a splitter that can do 3A like the one that I have. I know some Raspberry Pi's can run on 2.4A but are better with the 3A headroom. Additionally mine claims to do isolation circuit protection which is a plus. Here is the one I have been using. Hope it helps!

https://www.amazon.com/gp/product/B09GM7BB77

I have experienced that w/a pi myself...2.4A supply got me undervotage warnings, 3A fixed it.

https://www.amazon.com/gp/product/B09GM7BB77

Just ordered one. I have not put one of my C7's back in service yet, when I do I'll move the C8 to the 3A splitter and put the C7 on the existing 2.4A splitter.

Just remember that the C7 is micro USB and the C8 is USB-C. So unless you find something to change the port (dunno if that is even possible) you’ll need to order the 3A USB-C version for the C8.

I've actually purchased the 12V 2A barrel jack version of that splitter for another project. That one is working well, might be worth trying out the USB C version to see how it goes.

Though 5V 3A is good, I wouldn't think needing that much extra headroom is necessary. The power brick that comes with the C8 (and I believe C7 too) are 5V 1A. So a 5V 2A power capable device should have more than enough headroom to handle any extra hub load. Though, given the issues with the 5V2A one, maybe this is the case.

If I had to guess, the voltage output is sagging under higher hub loads, which probably explains issues when backups run. Either that or the POE splitter is generating a lot of noise and it is causing issues with the hub. Given the tolerances with electronic components, really either could be necessary, and it explains why some adapters work well and some aren't.

I'll probably pickup a 3A version and see how it does. So far with the hub provided power brick, my system has been flawless! Would like to keep POE power though, given I have a ~2 hour battery backup if the power goes out.

I've used adapters in the past and they've been fine going from C to micro USB.

But frankly I just missed that the one I just ordered is micro USB. Don't know how I missed that. Thanks for pointing it out.

Getting rid of the PoE splitter has not resolved my issues. I had multiple complete failures of the z-wave network last night where devices simply became unresponsive. A simple reboot (not a power down) brings it back. The mesh itself looks fine - no indication of any issues other than two ring keypads that reroute frequently. I can see motion sensors detecting motion but the event tab shows no motion events and no rules triggered by the sensor run. There’s nothing in the logs to give any indication of what’s going on. There are no devices or apps generating excessive load. There is no spike in CPU in the timeframe in question.

I noticed this happened during backups so I tried moving backups but it happened last night twice when no backups were schedule.