Z wave Ghost issues

[ajohnson]

I have been working on Z Wave problems and cleaning up the Z Wave network. I am having problems removing devices that don't exist anymore. At the moment I have to first try to repair the Z Wave on the device so a remove button shows up. Then when I click the remove button it shows Pending but the device is never deleted. Am I doing something wrong or is this the way things work?

[bertabcd1234]

You should not be using "Repair." If there is no "Remove" button, use "Refresh." Wait a few seconds for it to finish, and try it one or two more times if not.

If you see "Remove" and it doesn't help, check "Logs" while you do it to see what is happening.

[danabw]

See this topic for more details on removing ghosts...

How To: Remove Ghosts using hub tools or a UZB Stick

The information below can help you resolve problems w/your Z-Wave mesh caused by Z-Wave device "ghosts." Review below for detailed information on the following: What are Z-Wave Ghosts and how are they created? Avoid creating ghosts Remove ghosts using built-in tools on the Hubitat hub How to remove them using a USB Z-Wave stick (AKA UZB Stick). A link to a PDF guide to remove ghosts using a Z-Wave UZB stick is provided below. (Created w/extensive collaboration/support from the amazing @erktrek.…

Try the steps in this section of the ghost removal topic first...

3. How to use the Hubitat hub's built-in ghost removal tools Thanks to @jtp10181

Details

Simple first steps - try these first

1. If a Discover button appears for a partially included device, selecting that button can allow a device to successfully complete joining the Z-Wave mesh.
2. Shut down the hub and restart it to force restarting of the Z-Wave radio - this can help w/ghost issues:
   -- a. Shut down your hub from the Settings menu.
   -- b. Then pull the USB power plug (recommended to do at the wall connection, not at the hub as the hub connection can be a little fragile, at least on some C-7 hubs) to completely remove power from your hub after it's shut down.
   -- c. Wait 30s
   -- d. Plug the hub in again.

Using the hub's built-in ghost removal tools
Below is a detailed summary from @jtp10181 of best way to use the hubs built-in ghost removal tools. Try these tools first before trying Ghost removal with a UZB stick:

• Click Refresh on the device ONCE, wait for the page to refresh. The Remove button should then appear. Click Remove ONCE. Again, wait. May want to have logs open while doing this as well.
• Sometimes it takes multiple tries of the Refresh button to get the Remove button to appear. Try to be patient and NOT click Refresh furiously. If you start getting "Zwave busy" error messages in your logs, power down and reboot again.
• It is common for it to take multiple tries at the Refresh/Remove buttons to remove a ghost, give it at least five or six attempts before assuming it's not going to work. Shutting down/restarting between successful removals can also help.
• From some anecdotal evidence, removal of a ghost device may be more successful if the device the ghost was created from is disconnected from power while you're trying to remove the ghost. This is probably not going to help, but it's easy to try - If it's a plug, you can unplug it. If it's a switch that has an air-gap, you can pull the air gap. You may also be able to flip a circuit breaker to temporarily remove power from a mains powered device. May not help, but an option you can consider.

If the above steps don't remove the ghost...

The hub may not be able to remove Ghosts that have too many existing neighbor nodes. You can use the "Hubitat Z-Wave Mesh Details" tool (available in Hubitat Package Manger) to easily identify the neighbors of your ghost device.

You can try the following steps before moving on to using a Z-Wave USB stick:

• Install the Z-wave mesh tool [BETA] A Z-Wave Mesh Tool [C7 and 2.2.4+ Only] from Hubitat Package Manager and use the Mesh tool to identify the neighbor nodes that the ghost devices are hanging onto. Use the expand arrow in to expand the ghost device row in the Mesh Details app to see the device's neighbors. See the "Neighbors" column in the example screen cap below:

If you exclude the neighbor nodes, then the ghost node should also be removable.

Steps:

1. Create a virtual device for each neighbor.
   -- Add Device at top of Device page, choose "Virtual" device type, and choose virtual device types that match w/your actual devices.
2. Use the Settings>Swap Apps Devices tool to swap out each neighbor w/a virtual device (this maintaings your automations using these devices)
3. Exclude the neighbor devices
4. Use the HE hub steps above to remove your ghost(s)
5. Re-include the neighbor devices
6. Use Swap App Devices again to swap the restored devices back into their automations, dashboards, etc.

[jtp10181]

Also make sure you have logs open on another tab to watch for errors.

[ajohnson]

Thanks to everyone for the suggestions. I managed after many attempts "Using the hub's built-in ghost removal tools" instructions.
I am retired but come from a background in industrial control and monitoring (SCADA) and to me it is difficult to understand why it is so difficult to remove a dead device from the communication network...

[rlithgow1]

@ajohnson

it is difficult to understand why it is so difficult to remove a dead device from the communication network..

It's because of the SiLabs SDK

[danabw]

Longer answer from the Z-Wave Ghost removal topic:

@danabw

Why doesn't Hubitat just fix this!?

Details

TL;DR: They can't - they are limited to the tools and options provided in the SilLabs Z-Wave SDK, as well as limitations imposed on them since they are a Z-Wave Certified hub.

Long answer: Ghost removal issues are due primarily to the Z-Wave SDK provided by SilLabs to all Z-Wave 700 & 800 hub vendors. That SDK (and its associated bugs/limitations and operating agreement that HE must conform to for certification) control/limit what the HE hub can do. HE has worked steadily to improve the Z-Wave experience in areas they have some control, but in the area of Ghost removal they're limited by what SilLabs provides in the SDK and the requirements of their Z-Wave certification. Software used in the ghost removal process detailed in the PDF here is not constrained in the same was as it is a developer tool and relies on a different SDK/process to remove ghosts, one that consumer-certified hubs like Hubitat cannot use.

SilLabs has continued to release Z-Wave updates and Hubitat integrates & releases those updates. Things are much better than they used to be, and should hopefully continue to improve over time.

The most important thing you can do to simplify issues related to ghosts is avoid them in the first place by creating a robust Z-Wave mesh, and checking each Z-Wave device on your Z-Wave Details page as you join it to ensure a proper pairing w/the hub.

[ajohnson]

Thanks for the explanation. My problem was a Z-Wave lock that totally failed which did not allow me to properly remove the lock from the network.