The information below can help you resolve problems w/your Z-Wave mesh caused by Z-Wave device "ghosts." Review below for detailed information on the following:
- What are Z-Wave Ghosts and how are they created?
- Avoid creating ghosts
- Remove ghosts using built-in tools on the Hubitat hub
- How to remove them using a USB Z-Wave stick (AKA UZB Stick). A link to a PDF guide to remove ghosts using a Z-Wave UZB stick is provided below. (Created w/extensive collaboration/support from the amazing @erktrek.)
Recommended approach to remove ghosts:
- Try the built-in Hubitat hub ghost removal tools - most ghosts should be able to be removed using those tools.
- If you're unable to remove your ghosts w/the hub's built-in tools, follow the instructions in the Ghost Removal PDF (link at the end of this post) to remove ghosts using a free software download and a Z-Wave UZB stick (which you will need to purchase).
1. What are Z-Wave Ghost devices and how are they created?
Z-Wave "Ghosts" are Z-Wave devices that have not successfully joined your mesh during pairing, leaving an entry in your hub's Z-Wave device table that is incomplete. In some cases multiple failed attempts to add a single device can result in multiple ghosts from each failed/partial pairing.
Note that having even a single ghost in your Z-Wave mesh can cause severe problems w/the responsiveness of some or all of your Z-Wave devices, making them slow or non-responsive.
The easisest way to identify a ghost is by looking at the Z-Wave Details page on your hub, and look for devices that do not show any routing in the "Route" column of the Z-Wave Details page. Problems usually show up quite soon after a Ghost appears, so Ghosts are usually one of the most recently added devices and are at or near the bottom of your Z-Wave Details list (most recently-added devices are at the bottom).
Below is a normal device in the Z-Wave Details page, showing normal routing and connection speed info in the Route column. Route: 01 -> 0A. Speed: 40kbps. Routing may be direct to the hub in this case (01 is the Hub, 0A is the device), or a device may route through one or more other devices to connect to the hub.
Ghost devices typically appear in the Z-Wave Details table w/no routing information in the Route column - looking something like below. Note that the last field in the row below is completely empty, shows no routing for the device. Below are a couple of examples:
- Note on Hubitat C-8 Migration: If you have recently migrated from a C-5 or C-7 hub to a C-8, it's normal/OK if all or almost all of the Z-Wave routes are empty initially after that update. As a part of the update the Z-Wave route cache is cleared and may take up to 2 to 3 days to fill in. In that case, (C-8) migration, wait for the mssing route info to fill in before assuming you have ghosts. Migration to a C8 will not remove any existing ghosts that were on the C-7 or C-5 that you migrated from.
2. How can I avoid/minimize creating ghosts? Courtesy of @rlithgow1:
If you have a failed Z-Wave pairing, STOP! Check your Settings>Z-Wave details page. If a ghost was created (you will see nothing in the routing column), it needs to be removed first. If you don't remove it you will start having problems with your mesh including pairing new devices. After removing the ghost, factory reset the device and attempt to pair it again. Rinse and repeat. This is one of the biggest keys to having a healthy mesh.
3. How to use the Hubitat hub's built-in ghost removal tools Thanks to @jtp10181
Simple first steps - try these first
- If a Discover button appears for a partially included device, selecting that button can allow a device to successfully complete joining the Z-Wave mesh.
- Shut down the hub and restart it to force restarting of the Z-Wave radio - this can help w/ghost issues:
-- a. Shut down your hub from the Settings menu.
-- b. Then pull the USB power plug (recommended to do at the wall connection, not at the hub as the hub connection can be a little fragile, at least on some C-7 hubs) to completely remove power from your hub after it's shut down.
-- c. Wait 30s
-- d. Plug the hub in again.
Using the hub's built-in ghost removal tools
Below is a detailed summary from @jtp10181 of best way to use the hubs built-in ghost removal tools. Try these tools first before trying Ghost removal with a UZB stick:
Removal of a ghost device is usually not successful if the device the ghost was created from is plugged in/connected to power. If it's a plug, unplug it. If it's a switch and has an air-gap, pull the air gap. You can also flip a circuit breaker to remove power from a mains powered device temporarily in order to remove a ghost that was created from it during a failed pairing process.
Click Refresh on the device ONCE, wait for the page to refresh. The Remove button should then appear. Click Remove ONCE. Again, wait. May want to have logs open while doing this as well.
Sometimes it takes multiple tries of the Refresh button to get the Remove button to appear. Try to be patient and NOT click Refresh furiously. If you start getting "Zwave busy" error messages in your logs, power down and reboot again.
It is common for it to take multiple tries at the Refresh/Remove buttons to remove a ghost, give it at least five or six attempts before assuming it's not going to work. Shutting down/restarting between successful removals can also help.
The hub may not be able to remove Ghosts that have too many existing neighbor nodes. In these cases you can use the following steps before having to move to using a Z-Wave USB stick.
- Use the Z-wave mesh tool [BETA] A Z-Wave Mesh Tool [C7 and 2.2.4+ Only] to identify the neighbor nodes that the ghost devices are hanging onto. Use the
arrow in that app to expand the ghost device row to see the device's neighbors - see the "Neighbors" column in the cap below:
If you exclude the neighbor nodes, then the ghost node should also be removable. Steps:
- Create a virtual device for each neighbor.
-- Add Device at top of Device page, choose "Virtual" device type, and choose virtual device types that match w/your actual devices.
- Use the Settings>Swap Apps Devices tool to swap out each neighbor w/a virtual device (this maintaings your automations using these devices)
- Exclude the neighbor devices
- Use the HE hub steps above to remove your ghost(s)
- Re-include the neighbor devices
- Use Swap App Devices again to swap the restored devices back into their automations, dashboards, etc.
4. How to remove ghosts using a Z-Wave UZB stick?
If you still have Ghost(s) after trying the hub's built-in ghost removal tools, remove them by following the steps in the ghost removal guide PDF below:
- Important: If you end up needing to follow the steps in the Ghost Removal Guide PDF:
- You have to be logged into Simplicity Studio to complete the install of the Z-Wave tools add-on in Simplicity Studio.
- You can use any brand or Z-wave version (500, 700, 800) UZB Z-Wave stick
- If you have problems on Win 10 w/your UZB stick not being recognized, you may need to update your UZB stick driver from this post on SilLabs support.
Mac users: There is no Mac version of the PC Controller software, so you either have to have access to a Windows PC, or run PC Controller using Parallels or Bootcamp. One example below using Parallels:
Where can I find UZB sticks?
You can use any UZB stick you want to, below are a couple of options that have been popular w/Hubitat users:
Silicon Labs UZB stick - it's inexpensive and works well.
Zooz UZB stick - a good option and can be found on their web site and on Amazon.
Additional information about ghosts, UZB sticks, & Z-Wave
What about removing ghosts created by battery powered devices?
Why doesn't Hubitat just fix this!?
TL;DR: They can't - they are limited to the tools and options provided in the SilLabs Z-Wave SDK, as well as limitations imposed on them since they are a Z-Wave Certified hub.
Long answer: Ghost removal issues are due primarily to the Z-Wave SDK provided by SilLabs to all Z-Wave 700 & 800 hub vendors. That SDK (and its associated bugs/limitations and operating agreement that HE must conform to for certification) control/limit what the HE hub can do. HE has worked steadily to improve the Z-Wave experience in areas they have some control, but in the area of Ghost removal they're limited by what SilLabs provides in the SDK and the requirements of their Z-Wave certification. Software used in the ghost removal process detailed in the PDF here is not constrained in the same was as it is a developer tool and relies on a different SDK/process to remove ghosts, one that consumer-certified hubs like Hubitat cannot use.
SilLabs released some Z-Wave SDK updates in 2022 and Hubitat integrated & released those updates. Things are better than they used to be, and should hopefully continue to improve over time.
Make sure you have applied any available Z-Wave FW updates
If you see the highlighted button in the screen cap below at the top of your Z-Wave Details page that means there is a Z-Wave FW update available for your hub.
Click on the button to apply it. If you do not see the highlighted button your hub's Z-Wave firmware is already up to date. In some cases you may have to try the Z-Wave Firmware update a number of times for it to complete. If you are having problems applying a Z-Wave FW update: a) Shutdown the hub & pull power; b) Wait at least a minute; c) Plug in and boot up; d) go to the Z-Wave Details page and wait 10m; e) Try the update again. (Reason for the waiting: When you go to the Z-Wave Details page it kicks off a lot of internal messaging on the hub. Waiting 10m or more gives that time to settle down before you add the additional load of the Z-Wave FW update.)
More info on using the UZB stick and PC Controller to update Z-Wave device firmware
@jtp10181 has created a topic on how to use a UZB stick and PC Controller app to update firmware on a Z-Wave device. Good news is using a UZB stick and PC Controller is generally faster than doing a Z-Wave device FW update using the HE FW updater. Check out his topic below:
Feel free to share feedback and suggestions here on improviing the contents of this post and/or the ghost removal document.
8/11/2023 Added links to Silicon Labs and Zooz UZB sticks (From post by jtp10181)
Update 7/19/23 Added note to try Classic Learn Mode when trying to join the UZB stick to the HE hub if NWI learn mode fails. Thanks to @dave17 for the feedback.
Update 7/15/23 Added link to @jtp10181 topic on updating FW using a UZB stick w/the PC Controller app.
Update 3/25/23 Added jtp10181's excellent summary of how to best use the hub's built-in ghost removal process.
Update 2/16/23 Added link to Aeotec Simplicity Studio install instructions for Simplicity Studio. Simpler and likely more reliable than me trying to keep up w/changes over time. Thanks to @denwood for the suggestion.
Update 1/1/2023 Added note that you need to be signed into Simplicity Studio to load the Z-Wave Tools, thanks to @Jlv03 for reporting that.
Update 10/14/2022: Added section to FAQ on how to roll back to a different UZB stick driver if you are getting the failed to update locally error when trying to update the Z-Wave Firmware on your UZB stick. Thanks to @rfg81 and erktrek.
Update 10/5/2022: Misc. clean-up.
Update 2/2/2022: Fixed the download link from Dropbox, thanks to info from @672southmain that the link was downloading the wrong version of the document (from back in July 2019!!). Apologies to anyone who got the old version...please re-download to get the current version.
Update 1/25/2022: Added csteele's very clear graphic to clarify steps when excluding the UZB stick.
Update 11/8/2021: Added some additional comments regarding the UZB stick inclusion not displaying security options, or providing the normal option to name the device.
Update 10/5/2021: Updated section in the FAQ on regional settings (aka Transmit Settings) to include info that 500 series UZB sticks have region set on the device in FW, only 700 series UZB sticks can change region in the PC Controller Transmit Settings. From bcopeland
Update 9/24/2021: Modified steps to join UZB stick to Hub - changed from using "Classic Learn Mode" to "Select Learn Mode>NWI" per bcopeland's suggestion. Also added a section in the FAQ on how to update FW on the SilLabs UZB stick.
Update: 7/10/2021: Updated info in the FAQ section of the document on using a UZB stick and PC Controller app to join devices to HE w/out security. Based on info provided by bcopeland.
Update 7/13/2021: Updated PDF creation process to maintain clickable links in the PDF. Re-download if you want a version where the links work! Thanks to @eibyer for pointing this out.
Update 7/16/2021: Updated to include information on new UZB stick version that Mouser/Silicon Labs are shipping. Packaging may show the original UZB stick version (SLUSB001A) but the newer version (SLUSB7000A, rev 1 update of the original) is the currently shipping hardware. Thanks to @672southmain for pointing this out.
NOTE: If you run into an error "Can't open Xmodem session" while trying to use the PC Controller app to update the firmware on your UZB stick, try the step below:
Who you gonna call...yes, them: