Intermittent z-wave mesh failure

For the last couple of months the entire z-wave mesh has been failing in a seemingly random fashion. Virtual devices work, but all z-wave devices are unreachable/failed. There are 80+ devices and when this happens none of them work and a z-wave repair doesn’t fix it. Soft reset doesn’t fix it. Disabling then enabling the z-wave radio doesn’t fix it. The only thing that fixes it is shutting down the Hubitat and unplugging it and then rebooting.

There are no ghost nodes. I’ve had them in the past, but I’ve removed them with a z-stick.

It may be coincidental but this started happening after we lost power. The hub is connected to an APC backup and I’m not sure if it ever actually lost power… so I have a hard time attributing the z-wave failure to the power failure.

Considering that this happens in a seemingly random fashion without any changes to the devices or apps… I think this may just be a problem with the hub itself. I’m considering getting a new hub.

Two questions:

  1. Does this sound like it could be an issue with the hub itself? Or am I off base?

  2. If I do replace the hub - is there an easy way to migrate the devices without having to unpair and repair every single device? I’ve seen write-ups on this in the past, but didn’t Hubitat just come out with some new z-wave migration tool?

I know this is a long post and there are a lot of variables but I’d appreciate any insight y’all might have. My son needs his bedtime routine automation to work reliably!

Well here’s what I’m going to try… I have two hubs. One runs all the z-wave devices and the other controls a lot of the programming. I’m going to put all the potentially destabilizing S0 devices (locks) on the other hub. If this doesn’t work then I’ll also put all the S2 devices (garage doors) on the other hub. If this doesn’t work then I’ll put all the battery powered devices (various sensors) on the other hub. We’ll see what happens…

I missed this the other day.

I would say more likely the database is corrupted. That can happen with power outages more so than Zwave radio issues due to power loss.

Also, what hub firmware are you on?

Can you post screenshots of the Zwave details page? Maybe that would give someone a clue as to what is going on.

You could try that, if you have a lot of these S0 devices. But a couple locks probably isn't the issue. The S0 thing is really more likely to happen with something like a multi-sensor that has temp, humidity, light, and battery reports. Or a power reporting device that has all reports turned on.

Speaking of which, do you have lots of devices that report power/watts? If misconfigured, those can wreak havoc with the hub.

1 Like

That fits the bill of an overwhelmed radio. Please check the Logs to make sure that you don't have one or more rogue devices that are spamming the radio. Also, check this article:

If there is nothing suspicious in your Logs (past Logs) send us an email to support@hubitat.com and we would be glad to further investigate.

2 Likes

I am on the latest firmware.

There’s nothing abnormal on the z-wave details page but will post screenshots if I continue to have issues.

None of my devices report power.

I DO have a multisensor and looking at the logs it has been reporting quite a bit! Like 2-3 times a minute. Could this be the problem? I changed the settings so it should report much less frequently now.

I don’t see anything else suspicious in the logs, but I can’t say I know exactly what I’m doing as far as that goes.

1 Like

Do you have any zooz devices on S2? They didn't play well in my mesh. They would randomly lock up on a weekly to daily basis taking the integrity of the zwave network with it. I even have some paired without security that lock up every once in a while. I ended up repairing everything I could without security and that fixed most of the issues.

LOL, in the past few days that could be anything!

I think it would be a good idea.

A couple times a minute, probably not. If it was doing (just to pick a number) 4-5 reports in maybe a second or two, then I would point fingers. But it doesn't hurt to turn the settings down and see if it helps.

Mainly things like [warn] [error] and so on would be a good starting point. Or things that fire in rapid succession. You may also want to tab over while in logs and see if any of the device stats, or app stats items have a high usage. I turn on all columns (click the double arrow to the right of hub uptime) for these stats just to get an idea how much and how heavily these apps/devices impact my hub.

I have a few Zooz multirelays controlling fireplaces and actuators but they have never caused any problems and are paired without security.

Here are the z-wave details:









Thus far I’ve moved all S0 devices to a different hub and changed the settings on the multisensor to report less frequently.

I don’t see any errors or rapidly repeating things in the logs.

Considering that the z-wave network typically crashes every few days… I won’t know for certain whether the problem is fixed until some time has passed… but, so far so good.

I really appreciate the help.

1 Like

I think it’s time for an update. Since placing all S0 and S2 devices on a secondary hub (with a few range extenders)…. I haven’t had any z-wave issues.

I think I was just asking the hub to do too much. To lockdown the house I’ve got the main hub setup to interrogate most of the light switches and if they are not off it will send a signal to turn them off. It does this every ten seconds until they’re all off (times out after two minutes). That’s probably a pretty taxing process - but works just fine now.

In the end, separating the devices that require some level of security is probably better anyway… it’s the door locks and garage doors. They’ve worked flawlessly since putting them on a separate hub.

I know some people use node red and/or a raspberry pi to do some of the more complex automations. That’s just not an endeavor I have time to pursue. I might get a third Hubitat for that purpose at some point.

I experienced a similar failure of the entire z-wave mesh. The "inbound" direction stayed up, in that multisensors and power measurement devices continued reporting, however all "outbound" activity vanished. I could not control any devices via z-wave. Zigbee devices remained operational.

After reading this thread, I traced the problem to a new Zooz ZEN32 scene controller I had just joined to my mesh with no security. Removing the device from the mesh (via "Force Remove" and factory reset of the device) restored everything.

I will try adding the ZEN32 again using S2. I've had good results with other Zooz devices in the past. I hope there are workarounds or future fixes because their products appear to be excellent.

I have some 700 series Zooz (Zen73/74) switches that are paired with no security and are working great. Maybe there is an issue with the Zen32 device itself as I was under the impression that no security is less chatty than S2 security and having one device shouldn't make a difference either way.

There does seem to be some debate as to whether to use S2 or not for non-security devices.. I prefer to go with no security for reduced mesh impact while others just accept the security defaults which also makes sense as that configuration is probably the best tested.

1 Like

From what I’ve read on here I think S0 is the most “talkative” and not very secure. S2 creates some traffic but is important for locks and garage doors.

I’ve got several zooz multirelays connected without security and I do not have any issues with them, although I’ve read some zooz devices can be problematic.

My issue was clearly having too many z-wave devices. I know technically you can have like 200 something, but I doubt the z-wave radio can handle that. I now have 3 hubs, each with devices connected to them, all connected with hub mesh, and several repeaters. Some of the tasks I require can be pretty taxing and spreading the load between three hubs seems to have done the trick.

Most of the "problematic" Zooz stuff comes from non-S2 500 devices pairing as S0 on a C-7 thanks to a squirrely Z-Wave spec.

I have not heard of any Zooz (or other) S2 non-security devices either 500 or 700 series not working with "no security".. but that does not mean there aren't any..

2 Likes