Recurring "NOT_RESPONDING" or "FAILED" on C-7

@bcopeland I've spent the last several days working on a complete rebuild of my C-7 install. Beginning with a hub with a soft reset and a radio reset, I now have a little over 100 Z-Wave devices on the network. All are mains powered, Plus devices. Dimmers, switches, receptacles, and 3 repeaters.

I have a handful of devices that show up as "NOT_RESPONDING" or "FAILED". If I go and push all the buttons on the devices, Plus does its thing, and the devices again show as "OK". 30 minutes later the same devices will again show as not responding or failed. The interesting part is that in the interim, the hub has neither sent nor received any packets. Not with the affected devices, nor with any others.

What would cause the hub to mark devices as not responding or failed with no attempted communication?

From what I've seen those are normally battery powered devices. I assume that they are "sleeping" awaiting something to do and aren't responding rapidly enough for the hub. I see this a lot in my door contact sensors.

1 Like

Presumably you've confirmed that the devices showing "Failed" or "Not Responding" still work as expected, regardless of the messaging on the the Zwave Details screen, is that correct?

They are all mains powered.

I have 2 Neo Coolcam Motion Sensors and also Fibaro motion sensors that show this all the time.
They work all the time and I am not concerned about these messages. They work. Probably getting these messages because, as has been said, they are battery devices that go to sleep to save battery.
Before we had the facility of checking the state of our z-wave devices the only time we had any concern was if they did not function any more.
I cannot help but think this 'new' option that has been provided is causing us concern when there isn't any.
The only time I check it now is if I have paired a new device and I just check to make sure no 'ghost' device has crept into the list.

I personally feel that the only thing we need to know is "Is it working as it should".
If the answer to that is 'Yes' then all is hunky dory regardless of the state in the Z-Wave device list.
If the answer is 'No', well then we need to investigate.

Just my 2 pence worth. :grinning:

2 Likes

This is actually coming from the SDK. The hub is just displaying the information. If your device is working, I wouldn’t worry about it.

4 Likes

The devices don't work when in that state. When attempting to control one of the devices, the hub will send no packets to the device. The hub also hangs for about a minute after the request, and will not send control packets to any other device. After a minute it recovers, and then will send control packets to other devices. There are no Busy messages during this.

Have you tried the individual Repair buttons on the devices that keep showing disconnected?

@bcopeland said recently, may apply here:

Any nodes that you feel are taking ridiculous routes: I would recommend doing single node repairs on the nodes in route and the final node.

1 Like

Good suggestion. I'll check the routes when they redisplay in the details page. Thanks.

I still seem to be able to get the radio to hang completely.

I was running repair node by node. Log output below, followed by a sniffer snapshot. The packets at 13:51 are the result of physically pushing the switch.

Since the last Busy message, the hub has not transmitted any control packets to this device or any other device. The "off" and "refresh" buttons have been pressed many times for the two devices closest to the hub (6-8') with no resulting packets. In case anyone is going to ask, no, there are no ghosts in the network. The hub doesn't have any and PC Controller doesn't see any either.

sys:12020-09-01 14:06:20.764 warnZ-Wave Network responded with Busy message.
sys:12020-09-01 14:06:10.639 warnZ-Wave Network responded with Busy message.
sys:12020-09-01 14:06:00.737 warnZ-Wave Network responded with Busy message.
sys:12020-09-01 14:05:50.623 warnZ-Wave Network responded with Busy message.
sys:12020-09-01 14:05:40.721 warnZ-Wave Network responded with Busy message.
sys:12020-09-01 14:05:30.584 warnZ-Wave Network responded with Busy message.
sys:12020-09-01 14:05:20.694 warnZ-Wave Network responded with Busy message.
sys:12020-09-01 14:05:10.541 warnZ-Wave Network responded with Busy message.
sys:12020-09-01 14:05:00.617 warnZ-Wave Network responded with Busy message.
sys:12020-09-01 14:04:50.508 warnZ-Wave Network responded with Busy message.
sys:12020-09-01 14:04:40.585 warnZ-Wave Network responded with Busy message.
sys:12020-09-01 14:04:30.482 warnZ-Wave Network responded with Busy message.
sys:12020-09-01 14:04:20.568 warnZ-Wave Network responded with Busy message.
sys:12020-09-01 14:04:10.463 warnZ-Wave Network responded with Busy message.
sys:12020-09-01 14:04:00.562 warnZ-Wave Network responded with Busy message.
sys:12020-09-01 14:03:50.430 warnZ-Wave Network responded with Busy message.
sys:12020-09-01 14:03:40.522 warnZ-Wave Network responded with Busy message.
sys:12020-09-01 14:03:30.405 warnZ-Wave Network responded with Busy message.
sys:12020-09-01 14:03:20.582 warnZ-Wave Network responded with Busy message.
sys:12020-09-01 14:03:10.373 warnZ-Wave Network responded with Busy message.
sys:12020-09-01 14:03:00.477 warnZ-Wave Network responded with Busy message.
sys:12020-09-01 14:02:50.346 warnZ-Wave Network responded with Busy message.
sys:12020-09-01 14:02:40.499 warnZ-Wave Network responded with Busy message.
sys:12020-09-01 14:02:30.326 warnZ-Wave Network responded with Busy message.
sys:12020-09-01 14:02:20.437 warnZ-Wave Network responded with Busy message.
sys:12020-09-01 14:02:10.300 warnZ-Wave Network responded with Busy message.
sys:12020-09-01 14:02:00.411 warnZ-Wave Network responded with Busy message.
sys:12020-09-01 14:01:50.277 warnZ-Wave Network responded with Busy message.
sys:12020-09-01 14:01:40.388 warnZ-Wave Network responded with Busy message.
sys:12020-09-01 14:01:30.246 warnZ-Wave Network responded with Busy message.
sys:12020-09-01 14:01:20.401 warnZ-Wave Network responded with Busy message.
sys:12020-09-01 14:01:10.236 warnZ-Wave Network responded with Busy message.
sys:12020-09-01 14:01:00.303 warnZ-Wave Network responded with Busy message.
sys:12020-09-01 14:00:50.192 warnZ-Wave Network responded with Busy message.
sys:12020-09-01 14:00:40.338 warnZ-Wave Network responded with Busy message.
sys:12020-09-01 14:00:30.181 warnZ-Wave Network responded with Busy message.
sys:12020-09-01 14:00:20.284 warnZ-Wave Network responded with Busy message.
sys:12020-09-01 14:00:10.115 warnZ-Wave Network responded with Busy message.
sys:12020-09-01 14:00:00.149 warnZ-Wave Network responded with Busy message.
sys:12020-09-01 13:48:22.676 infoFinished Z-Wave Network Repair
sys:12020-09-01 13:48:20.690 warnZ-Wave Network responded with Busy message.
sys:12020-09-01 13:48:20.663 warnZ-Wave Node 5A: Repair failed node neighbor discovery (timeout)
sys:12020-09-01 13:46:20.427 traceZ-Wave Node 5A: Repair is updating neighbors
sys:12020-09-01 13:46:18.159 traceZ-Wave Node 5A: Repair is pinging the node
sys:12020-09-01 13:46:18.149 traceZ-Wave Node 5A: Repair starting
sys:12020-09-01 13:46:08.514 infoStarting Z-Wave Network Repair
sys:12020-09-01 13:40:54.764 infoFinished Z-Wave Network Repair
sys:12020-09-01 13:40:52.745 traceZ-Wave Node 5A: Repair failed node unreachable
sys:12020-09-01 13:40:40.500 traceZ-Wave Node 5A: Repair is pinging the node
sys:12020-09-01 13:40:40.484 traceZ-Wave Node 5A: Repair starting
sys:12020-09-01 13:40:20.523 warnZ-Wave Network responded with Busy message.
sys:12020-09-01 13:39:55.467 infoStarting Z-Wave Network Repair
sys:12020-09-01 13:35:02.432 infoFinished Z-Wave Network Repair
sys:12020-09-01 13:34:59.748 traceZ-Wave Node 59: Repair is done.
sys:12020-09-01 13:34:58.877 traceZ-Wave Node 59: Repair is adding return routes
sys:12020-09-01 13:34:57.544 traceZ-Wave Node 59: Repair is deleting routes
sys:12020-09-01 13:34:47.400 traceZ-Wave Node 59: Repair is updating neighbors
sys:12020-09-01 13:34:45.125 traceZ-Wave Node 59: Repair is pinging the node
sys:12020-09-01 13:34:45.116 traceZ-Wave Node 59: Repair starting
sys:12020-09-01 13:34:35.269 infoStarting Z-Wave Network Repair

The problem may very well be in the SDK rather than your code... :slight_smile:

Hit it again immediately. Below is the log since reboot, omitting the several hundred Busy messages that have happened since:

sys:12020-09-01 14:30:50.234 warnZ-Wave Network responded with Busy message.
sys:12020-09-01 14:30:40.365 warnZ-Wave Network responded with Busy message.
sys:12020-09-01 14:30:30.206 warnZ-Wave Network responded with Busy message.
sys:12020-09-01 14:30:20.321 warnZ-Wave Network responded with Busy message.
sys:12020-09-01 14:30:10.146 warnZ-Wave Network responded with Busy message.
sys:12020-09-01 14:30:00.128 warnZ-Wave Network responded with Busy message.
sys:12020-09-01 14:28:01.701 infoFinished Z-Wave Network Repair
sys:12020-09-01 14:27:59.707 warnZ-Wave Network responded with Busy message.
sys:12020-09-01 14:27:59.669 warnZ-Wave Node 5A: Repair failed node neighbor discovery (timeout)
sys:12020-09-01 14:25:59.551 traceZ-Wave Node 5A: Repair is updating neighbors
sys:12020-09-01 14:25:57.271 traceZ-Wave Node 5A: Repair is pinging the node
sys:12020-09-01 14:25:57.264 traceZ-Wave Node 5A: Repair starting
sys:12020-09-01 14:25:52.162 infoStarting Z-Wave Network Repair
dev:2382020-09-01 14:25:32.482 infoOutside Recroom is off [digital]

This is the entire set of packets set/received since reboot:

Of note that the route assignment went out and was successfully received, even though it doesn't show in the logs.

@bcopeland, is there any information I can provide to help with diagnosing the issue?

So the device acked the NOP and acked the Neighbor Discovery.. But just chose to ignore it.. What device is this?

I see a "Command Complete" following the "Find Nodes In Range" command, and a "Node Range Info" following the "Get Nodes In Range" command. The sniffer is too far away from the device to see the "NOP Power" packets. Is there something I am missing?

To answer your question, this device is a Leviton DZ6HD-1BZ.

I'm seeing the behavior on another device as well. This is from a GE 14288 Enbrighten outlet:

sys:12020-09-02 09:02:02.422 infoFinished Z-Wave Network Repair
sys:12020-09-02 09:02:00.424 warnZ-Wave Network responded with Busy message.
sys:12020-09-02 09:02:00.396 warnZ-Wave Node 65: Repair failed node neighbor discovery (timeout)
sys:12020-09-02 09:00:00.267 traceZ-Wave Node 65: Repair is updating neighbors
sys:12020-09-02 08:59:58.018 traceZ-Wave Node 65: Repair is pinging the node
sys:12020-09-02 08:59:58.011 traceZ-Wave Node 65: Repair starting
sys:12020-09-02 08:59:48.405 infoStarting Z-Wave Network Repair

Once this happens, the radio requires a reboot to recover.

@bcopeland, is there any more useful information from the current configuration I can provide? I'd kinda like to exclude some of the devices and experiment with re-adding them, but don't want to destroy the current configuration if there is something useful that can be taken from it. Thanks.

I guess it could be/likely is coincidence. When I woke up this morning Z-Wave devices of all types were unresponsive. Lots of not_responding and failed in the list. I had to power down/pull plug to get everything back up. Things seem to be working OK now even as there are a couple not_responding in the list. Most of my Z-Wave devices are DZ6HD-1BZ.

I see the problem here.. Fix will be coming

3 Likes

Sweet!

@bcopeland FANTASTIC!