Elevation C7: Possible faulty z-wave radio?

I've had trouble with my usb powered MS6's in general. They make terrible repeaters for some reason. Replaced most of them with Inovelli 4-in-1s. Only have 3 left - all are on the fringe of my network (basement/garage/exterior back of house) and seem to be okay so far.. have not migrated those to the C-7 yet.

edit: am using an older @csteele driver right now. v1.6.13 and have also upgraded the MS6 firmware to v1.13.

I have reset my hub and all of the devices to factory default yesterday and re included some of the hardwired devices (total of 14 of devices) last night. This morning Ive included couple of battary powerd devices and here is what my log looks like at the moment. I did reboot and did the Z-Wave network repair and nothing works.

Try shutting the hub down via the UI menus.
When the hub is shut down, remove the power for 1 minute and then power back up again.
This will reboot the z-wave stick and may cure your problem.

Mine are the backbone to my house, all perfect running at 100k.

I'm using built in driver.

1 Like

That's cool I wish I could say the same... :neutral_face:

My MS6's originally were surrounding my C-4 main hub and so everything was repeating through them. Was getting a ton of 9.6 to various devices. Tried to do the trick of pairing with a battery then switching over to usb power afterwards in order to stop the repeating but that did not work with the updated firmware sadly. The MS6's on the edge of my network seem to be working great though - even the one I use for lux detection which has been outdoors in a clear weatherproof box for like a year or so..

1 Like

Unfortunately, you're not really going to know if you are overloading the network unless you go down the sniffer route. I went down that route full tilt.

I've built/re-built my C7 network 8 times from the ground up. 120 devices. As a general rule, each time I got around 80-90 devices, the network would start having stability issues. Didn't seem to matter what the 80-90 devices were. I have very little automation that creates Z-Wave messages. Definitely not overloading the network with automation.

I've learned a bit along the way. In my last go-round, I laid down a bunch of repeaters as the first devices, and then very slowly added devices to the network over a week. With this approach, I was able to build the whole network out, however it ended up loosing stability about a day or two after completion.

What happens (in my network) is that a random device pops a routing error, usually in response to a hail. This in turn causes the hub and the device to try and establish a new route. Given the number of devices I have there are many routes to choose from, so things can bounce around a bit. This process generally hangs transmit on the hub for a short period of time--up to 120 seconds but usually less. This is bad, but in and of itself is not so much of a problem. It just means that further operations are delayed. Annoyance rather than End Of The World.

However, if this happens right before a supervised device sends a message that requires application level confirmation (garage door openers and thermostats in my network) it can grow into a serious issue. If these devices don't get serviced within a relatively short period of time, they fall into discovery mode. 50-100 messages in a brief moment. And if a second supervised device happens to need servicing while this is underway? It results in a swarm of discovery messages involving multiple devices. Bad news for the network. Usually the hub recovers after two minutes, but sometimes requires a reboot to right itself.

For me, the solution has been to move a dozen of the further devices to a second hub. Even though these devices were within 3 hops of the main hub, and worked fine when there were only 50 devices, when the network was fully built out they seemed to initiate a lot of route changes that destabilized the network. Having moved those devices, I've been stable on both hubs for over a week. I hope it stays that way because my wife's tolerance dropped to absolute zero.

As I've said before, I don't think these issues can simply be laid at the feet of Hubitat. The issues appear to be down below, either in the SDK or the firmware of the 700 series chip. Either way, the issues are out of reach of the Hubitat folk. I wish it were different, but it is what it is. I'm sure it's even more frustrating to the Hubitat folk than it is to their users.

Overall, I think SiLabs still have some significant issues to work out with the SDK and the 700 series chips. I hope they do it soon.

14 Likes

Very good analysis, I can see that a lot of work went in to discovering the issues.

1 Like

I hope you shared that with Hubitat staff. I am not sure if they can do much about it (see below), but maybe that information will lead to a fix.

That is my understanding the situation. Hubitat's hands are basically tied on some of this. They are relying on the underlying SiLabs firmware to work correctly, which it mostly does. But there are apparently some half-baked and flawed parts of the firmware that has not been fixed.

:+1:

I don't have a 700 series hub (yet), but thanks for taking the time to diagnose this. I hope this hard work benefits others.

1 Like

@dennypage Very detailed diagnostic data you have there.. It may be able to create a work-around.. Thanks for your post

2 Likes

There is actually quite a few work-arounds in the code to issues we discovered in the sdk

6 Likes

The only problem being all the work you have to do to get there, then they fix the firmware and you have to adjust it again. :expressionless:

2 Likes

I am in the process of consolidating my 2 C-4 hubs into a C-7 so the apparent 80-90 zw+ device limit is worrisome. Already thinking about putting my zigbee devices on a separate C-5 for giggles. I did like having 2 hubs by location though.

@dennypage Kudos for sharing this detailed analysis with us.
So I can stop my fiddling, wait for a fix and turn over to something different. :wink:

For me it didn't take that much devices to reproduce this problem, 35 seem to be enough.

1 Like

Could this issue affect c5's

I'm at 32 devices currently on my C-7 and the only issues were some weirdness pairing 2 Inovelli 4-in-1s and an automatic (can't change) S0 authentication pairing for my recessed door sensor.

Also my MS6 in the basement is back to reporting 9.6 even though an Aeotec 7 repeater is 7 feet away - so maybe that's an indication, dunno.

edit: sigh recessed door sensor not working properly.

Which brings me back my favourite theory, that things start to become really bad after including some FLiRS.
Do you you have any?

Good to know, thought about buying one, too. Maybe it works using the generic contact sensor drivers)

So my recessed sensor is a Gen5, not Gen7. Although I have now ordered some gen 7's as a test.

Not sure if I have any FLiRS devices - my Yale lock is zigbee. Mmm... maybe my Aeotec Range Extender 7? My smart switches and such are ZW+ but likely older chipsets.

For the recessed sensor I am using the generic driver...

Have one too, it is mains-powered, so no FLiRS. Did you include using S2? I first did, but then decided to re-included it without security.
Some battery-powered devices like TRVs and smoke sensors sometimes are FLiRS.
I have learned that this technology can create lots of network traffic.

if you get a gen 7 . I recommend this driver as you can actually run tests from the extender to individual devices at normal or reduced power to see how your mesh is working (at least from this extender to other devices)/

1 Like

I pair all my devices as unauthenticated no S2 if I can help it. The only thing I did pair S2 was the Ring Extender V2. Thought it was necessary for power reporting but maybe not.