[2.3.5.113] C8 Z-Wave dropout at random times of the day

Would be curious to see how many route changes on a full mesh, and how long the hub was running at the time of the screenshot.

Also, is that device always paired with access control? That means it has access to add/remove nodes on the mesh, and not sure what else exactly but its basically a secondary controller. Those keypads might act as coordinators on the normal Ring system? With the access control security it might be asking for node details on the entire mesh from the hub, causing even more traffic.

I am not sure with all the interface changes, if you can deselect some of the access when pairing with security, but if you can I would select just S2 Auth, and deselect S0 and Access control.
If that's not an option I would be curious about testing it with no security just to see if that changes anything.

I'm not sure I understand the comment about S2 access control and a secondary controller. S2 access control is a standard z-wave security protocol generally used with door locks, garage door openers, security keypads, etc. Usually when I pair a secondary controller I do with without security at all.

I have tried a manual inclusion and skipping security altogether (for testing - not something I would do in the real world) and the results are more or less the same. It has been a while though so I'll give it a shot just to see. Though a security keypad without security would be... well... a keypad!

So while trying a manual inclusion I skipped security altogether and the hub never completed the inclusion process. Did a smartstart inclusion and I got a partial inclusion with a discover button. There's just something about the C8 that the Rings dont like. Or vice-versa. But this is on a network with just the hub and the ring so it doesn't account for the other problems I'm seeing.

I guess I don't have any locks or or keypads so I was not aware they would normally be classified as Access Control, although it makes sense now. I always just assumed since my USB stick got access control it was only used for secondary controllers. According to that article it is normal, so probably not hurting anything. And I agree that device should be paired with security.

The part about it not finishing the including, is the same problem that is being reported with USB sticks. So I think you are onto something there, the C8 seems to have some sort of issue pairing S2 access control devices. I believe a lot of people are struggling with locks as well (but that seems to be normal, always people having issues with locks).

The issue where it wont pair without security at all may be on purpose, some secure devices refuse to pair without security (to protect us from our own stupid selves).

Yeah I thought about that too and you may very well be right. It's trying to be idiot-proof. Probably not a bad thing.

Yeah. It's kind of like aqara sensors. Probably not the best litmus test. Or maybe it's a FLIRs device issue... locks are usually FLIRs too.

I almost wish I could reproduce this myself so I could sniff the traffic. Mine is working great :person_shrugging:

I would like to take a peek at some details to see if there are any patterns, might help devs narrow it down.

For people having these issues, can you please let the hub run for at least a few hours (when there is normal household activity). Use this app: [BETA] A Z-Wave Mesh Tool [C7 and 2.2.4+ Only] and get a screenshot of the entire page.

Ideally if you could use ShareX (or equivalent) to get the full thing in one screenshot it would be helpful as well. There is an option called "Scrolling Capture..." which will get the entire page. Recommendation for Screenshot App - ShareX

This chrome extension works also: GoFullPage - Full Page Screen Capture - Chrome Web Store

Definitely happy to help as much as I can here. I installed the app, and can post some information later once it has been running for a while.

Any settings I need to set in the app interface that would also be helpful?

I don't know what columns are selected by default but this is what I always use:
Manufacturer could be useful in this case also to help spot known problematic devices or patterns.

image

Set my settings similarly. I attached a baseline I just screenshotted a couple minutes ago. Might be a little light on activity compared to normal, evening would probably be better with lighting needed. I can screenshot again later today or tonight sometime, just figured it might be good to have something to compare to. Should I reboot in between screenshots or keep the hub up the whole time?

No rebooting would be best, reboot clears the stats, the longer it is up the more you can start to see if certain devices are misbehaving.

Overall yours does not look bad, lots of neighbors. Most have good response times. A few devices are struggling but do not have excessive route changes, they are just not responding quickly (responses from device might not be getting to the hub reliably so they have to send multiple times).

Figured I shouldn't reboot, but wanted to check. The route changes causing issues makes sense. I sorted by max route changes, posted below is everything 2 or greater. Of course the Schlage lock is the most troublesome, followed by a couple Ecolink battery sensors.

I see for the Zigbee radio we can now control the power level, I wonder if this could be implemented for Z-Wave too? Would be nice to try to force weaker devices to use repeaters to see if the network would become more stable.

If things start acting up worse, I will post another full screenshot to see if anymore useful data can be read. For now, things have been cooperating.

Here's another odd z-wave thing going on... not seen this before, looks like it started a little while ago.

This aeotec trisensor has been happily chugging along up there for about a year, then all of the sudden these started appearing in the log. No changes have been made other than to move the sensor from the C7 to the C8 - and that was 6 weeks ago or more.

In the Zwave page, I've found if the device name isn't listed (note where the word "Discover" is present you have a ghost device. ghost devices destroy a Zwave network. You need to delete them. The problem is Hubitat really doesn't do this procedure well. You need to first attempt to repair it several time and at some point that icon will turn to delete. Once you've deleted all ghost devices I suspect your zwave network will become significantly more stable.

Note the big red cross in the middle from devices 22, 23, 24, 26, 41 and 44. I suspect those 6 devices are actually ghosts. You can confirm this if they have no actual device name and are still in discovery mode. I suggest deleted them then running a z-wave repair to help clean things up.

@danabw quite literally wrote the book on removing ghosts.

1 Like

You're right about ghosts being an issue when they occur. 22, 23, 24, and 26 have both names and routes. Unlikely that they are ghosts. Not sure what's up with 41 and 44 since they don't appear on the z-wave mesh report at all. To my untrained eye this looks like some of the unexplained z-wave stuff I've seen post-migration.

In the zwave map built into Hubitat, do you see the word "discovery" in place of any device names. Those are usually the ghosts?

Note the red box I've placed around one of your previous images.

So I just found something interesting.

@brad5 when this happens for you (stuff stops responding). Go to Settings, Zwave Details, and then Zwave Logs. Once that’s open, try a Zwave device.

Mine was non responsive. Opened Zwave log (to see what it said), then hit a button (remote) again, light immediately responded.

Makes me think this actually similar to zigbee issue HE staff are chasing. The hub is losing contact with the Zwave controller. :confused:

Same experience as yours. Sure enough this morning it happened to me right as I sat down at my desk. Z-wave motion sensor blinking green yet I am sitting in the dark. Z-wave logs showed no activity even as I pulled the battery. Green blinking light indicating motion, zero in the hub z-wave log or events for the device.

I powered down the hub for 30 seconds and restarted it and it came back. Which usually tells me the z-wave radio has become overloaded. Yet in this case no evidence of any device spamming (that I could see anyway) and the z-wave mesh looks fine, as it always does.

Something's broke for sure. The symptoms are different than the zigbee issues though. Once a zigbee sensor goes bye-bye it does not come back without intervention. The z-wave devices all recover when the radio recovers.

This all worked fine prior to migrating to the C8 and though there have been many changes in the z-wave mesh I have intentionally added NO new devices other than a ring keypad, which I have since removed (leaving two).

ah, sorry, I mean, when the z-wave stack goes unresponsive (stops responding to anything), see if opening the Z-wave log wakes it back up and it starts working again.