Z wave mesh frozen completely

I did more tests, as I said before, I have a Home Assistant system, fortunately the USB stick that I got for it is the same as HE sent me, I cloned HE usb stick to my usb stick, I can turn on and off everything just fine with zensys tools, when I try to delete the ghost nodes I get the usb stick frozen, the same problem with HE! And isn't a problem with zensys because I deleted completely my usb stick and I paired one of my rubbish GE dimmers and it worked, then I did the exclusion just fine.

I found the only way to bypass the frozen problem was closing the program, disconnecting the usb stick from PC, reconnecting the usb and opening the program.

So, my HE was frozen, then I did a reboot, did not work, I did a database restore, did not work, then I turn off the hub, disconnected the power and usb, reconnected everything, now it works.

Can anyone tell me what's going on here?

Because it works now that doesn't means this is over, eventually it will get slow and frozen, I believe we need to delete those ghost nodes. If I try to get data in zensys from the phantom nodes the usb eventually gets frozen, maybe because we can't see the initialize or remove button/status on the z wave table it could be the hub is trying to initialize ghost nodes and eventually the usb stick get slow and frozen.

Another option is to restore a software to my hub before 1.1.2.112 and I will do it my self, so @bravenel and @bobbyD please help me to keep me quiet for a while

To be quite honest with you, and I know you aren't going to like what I say, but at this point your best bet would be to remove all of your z-wave devices, reset the z-wave radio, and start over fresh. The amount of time you have invested in investigating and attempting to fix this mesh far exceeds the amount of time it would take to do what I'm suggesting.

From your descriptions above, it is pretty clear that your z-wave stick is messed up. How it got that way is moot at this point. How to get your system up and running correctly with the least amount of effort and highest expectation of success is the consideration. We don't have a magic wand that can fix your system. We removed the features that you want to roll back to precisely because the use of them was causing z-wave mesh corruption.

Please consider this course of action, as it is the only one that has any degree of predictable outcome.

By remove, I mean exclude each and every device, delete them from Hubitat, and start over from scratch. Each device must be excluded and factory reset.

Ok Bruce, I understand, I was trying to not reset the stick because that means that I must do all you mentioned above plus to re configure each RM rule, each SHM rule, Sharptools dashboard, Smarttiles, re configure groups.

Maybe my z wave was corrupted because the feature that was removed, because I use it a couple of times, the only thing is I know my mesh was getting frozen because my HEM, I removed it and added it a couple of times, but now those ghost nodes were created a few days ago, when the Lowes sale came, I removed my last 4 GE dimmer and added 5 Iris zigbee and z wave switches. I removed one Iris and I can see a ghost node for it too.

Please, I just want to try the software downgrade for now, I promise if it doesn't work I will do whatever must be do, but I can't reset the stick right now, on Wednesday I will start in a new job and I just want a few things working before I leave my house alone, but I have no time to reset all right now.

Please, help me, thank you for your time, I know its Sunday, no one in that other hub would be answering my stupid requests in a Sunday.

With ST, they are able to go in a remove ghost devices. I am guessing HE doesn't have that ability yet? Strange that you are not able to remove them with the Zensys software.

I believe there is some corruption already in my stick as Bruce said, I tried the method for ST IDE of adding a device with the same ghost device id and then delete it but HE consider that device as a virtual so no effect, after deleting everything doing a stick reset using zensys, I can add and remove devices normally. I just wanted to try a downgraded version of HE software to try an delete those ghost devices, probably I will reset everything 2 or 3 weeks later, but maybe I'm lucky and I don't need to.

When I try to remove the ghost node zensys said is not dead and cancelled the request.

To delete a device using Zentools or OZWCP you need to get them into the stick’s Failed List. You do that by repeatedly testing the node for fail. Be willing to try that 10 times. The “alive/good” status is cached and you must get past that. Then you can delete the failed node.

Note - it’s not clear if you did it that way or not... if yes, then my words may help someone reading this next month. :slight_smile:

1 Like

Actually not, I just tried it but no long wait, so thanks for the tip!

I started OZWCP to review... so I could be a tiny bit more specific..

I started with Receive Configuration. When OZWCP started and I connected to the Nortec Radio Stick, I noted it was 5 devices behind. Receive Configuration would copy a new DB from the Primary (Node1).

I started Receive Configuration and the first two times it failed, but third time's the charm and after starting Receive Configuration, I clicked Discover Devices on Hubitat (Node1) and my click timing was good enough and the copy was made.

Lesson learned: Try, try again. I know it works, I just have to click til it goes.

In OZWCP the command to test for a failed node is: Has Node Failed.
You click on the node that you believe is dead, then run Has Node Failed (again) til it agrees it's dead.

Once you have it "marked" as dead, you run the Remove Failed Node command. That one seems to work on the first try all the times I've done it.

Does a pretty nice Heal Network too. :slight_smile:

So it's safe to delete dead node with ozwcp? I am assuming that should be ok since HE does not show them anyway right?

Thank you for this useful info, one question, OZWCP is a Hassio addon app?

It's a standalone Open Source project that HASS integrated into their product. I would not chase OZWCP if you already have Zensys running. They arrive at the same result.

Notification: DriverRemoved
Notification: type 27 home f------3 node 0 genre 0 class 0 instance 0 index 0 type 0


********************* Cumulative Network Statistics *********************
*** General
Driver run time: . . . 0 days, 1 hours, 34 minutes
Frames processed: . . . . . . . . . . . . . . . . . . . . 2783
Total messages successfully received: . . . . . . . . . . 2783
Total Messages successfully sent: . . . . . . . . . . . . 1097
ACKs received from controller: . . . . . . . . . . . . . 1096
*** Errors
Unsolicited messages received while waiting for ACK: . . 1
Reads aborted due to timeouts: . . . . . . . . . . . . . 0
Bad checksum errors: . . . . . . . . . . . . . . . . . . 0
CANs received from controller: . . . . . . . . . . . . . 1
NAKs received from controller: . . . . . . . . . . . . . 0
Out of frame data flow errors: . . . . . . . . . . . . . 0
Messages retransmitted: . . . . . . . . . . . . . . . . . 1
Messages dropped and not delivered: . . . . . . . . . . . 47


2018-09-09 19:59:16.433 Info, mgr, Driver for controller /dev/ttyUSB0 removed

It also does a topology report and a statics report.

Thank you, great info that I will use tomorrow, I will let you know how it goes.

Is there a simple way to test the z-wave mesh is frozen? I only have one Z-Wave device (Garage door controller) and at the moment it is not responding to any commands. So not sure if it not getting the commands or the unit is faulty.

How far away and how many walls between the Hubitat Hub and the Garage Door Controller?

This sounds like a Z-Wave range issue and not a frozen Z-Wave. Move the hub closer to the controller and test.

@csteele thanks for your info, I got the ghost nodes removed and immediately I saw a change in my Z-Wave network speed, even deleting other nodes were faster after I deleted the node 2 of my mesh.

@bravenel Definitive the hub has something weird with ghost nodes, I'm not sure if this was corrected already but I deleted 4 GE dimmers after latest update 1.1.3.116 and they were there in the stick as ghost nodes, after I removed node 2, it took about 5 minutes, then everything else was faster.

Using Zensys I tested with my cloned stick first, I found ghost nodes 2,8,9,33,59 and 63. I removed them, everything was fine so I proceed to do the same on the HE stick, I noticed that node 8 was removed already by HE, I removed 2,9,33,59 and 63.

Right now everything looks stable, I will keep posting if I see something else.

Thanks

3 Likes

Thank you for posting your troubleshooting steps! I am sure this information will be helpful to others.

I am glad to hear your Z-Wave network is behaving better now and that you'll be able to enjoy your Hubitat Elevation Hub! :slight_smile:

2 Likes

Yesss, thank you, after Wednesday I will be away the house every day and I was a little stressed. :+1: :muscle:

1 Like

What would be good is for someone to do a separate writeup of a step by step guide on how to clear out ghost devices. (In a new thread). :wink:

4 Likes

When deleting a Z-Wave device, be sure to do an exclude on it. DO NOT FORCE DELETE. Doing that will leave the device on the stick.