Zigbee is down - help!

Since yesterday, all my Zigbee devices are no longer responding. I have tried the usual reboot, and shut down for 15 minutes. I noticed upon reboot that there ws a system message that Zigbee was down, but by the time I get to the settings page it is already up. One time after shutdown, the channel had changed from 20 to 11. I reset it back to 20, but still nothing is working. Nothing shows up in the logs for the individual devices, however in the Zigbee log I am receiving these messages.

I have already notified support but haven't heard back.

Any suggestions? @bobbyD

If you changed the channel it may take some time for all of the devices to discover the new channel and report in.

I changed it back to where it was. It wasn't working on the old channel, so I doubt they changed to it. I also tried re-adding an existing device, but it was never discovered.

TTBOMK the hub doesnโ€™t have a built in auto-change of the channel - so possibly a corrupt DB. You might consider doing a soft reset and restore.

2 Likes

Any recent changes in your Wifi world? new AP's or router or anything that could cause a problem? maybe a wifi analyzer that shows all the channels in use on your cell to see if anything is stomping on the zigbee to explain the initial outage?

1 Like

I had to look up TTBOMK! Anyway, soft reset and restore had no effect. Still getting weird messages in Zigbee log, and unable to re-add an existing device.

Only change was I moved my tv from wireless to wired. I doubt anything was using that as a next hop to the hub!

I had one hell of a time with a channel change, and it's still not right. If you don't have alot of rebuild, just reset the entire thing,
I am still fighting months later but have near 85 zigbee devices, so a total rebuild would take me days.
If you reset you will have to re-pair and rebuild EVERYTHING, as if the hub was brand new

The fact I can't add anything now, makes me very reluctant to do anything without support getting involved.

Yep, I would see if @support_team can look at your hub for you. Something seems drastically wrong.

2 Likes

I have 63 and wanting to add a 64th. Not looking forward to a burn and replace.

1 Like

Check your wifi devices. I bought two Eufy wifi cameras and on two occasions something about them running and streaming continously to a motions server that was recording the stream killed my zigbee network. As soon as i turned off those devices and reset them. My zigbee devices started responding again.

They are 2.4ghz only so somehow they were causing interference.

Wifi analyzer isn't a bad idea either if you have neighbors nearby to see if there are other potential sources of interference.

1 Like

I do believe it is not an interference issue, although I can't be sure. The thing that makes me think this is the messages I am getting in the Zigbee log. I'm pretty sure these are not normal.

By the looks of it, one of your repeating devices is going wonky...

@Rxich had something very similar occur a few years ago. As I recall, it took multiple soft resets and reboots to get the hub to stick to the zigbee channel it was supposed to. Perhaps he could comment on this?

1 Like

If I remember correctly, @Rxich problematic hub was a C-4 and based on our investigation, we deemed the problem a hardware issue.

2 Likes

Which of your hubs is acting up? Mind sending me the MAC address so I can take a look at your hub's engineering logs to make sure you are not dealing with a hardware malfunction?

1 Like

Hey Aaiyar, good memory my friend. It was my C4 and it stopped by itself. I did do a soft reset and backup restore. Back then I had many non-standard devices, and for me it appears the hub was getting overloaded, and in that case Zigbee is the first to go. Wifi(RGBW LEDs and wemo plugs) and Aqara devices drove me nuts. The LAN devices seem to stress my hub with activity, and Aqara devices caused zigbee issues. I'm still using my C4, but only with supported devices.

@jameslslate do you have any LAN/WIFI devices or any aqara devices?

I think there are issues around changing zigbee channels with an established network. In Home assistant it appears you have to do a re-pair of devices? As well as with Zigbee2MQTT, devices need to be re-paired. So I'm not sure how HE handles Zigbee channel changes, but it must be some type of "magic" that has yet undiscovered bugs?
Maybe @Tony has some ideas or might be able to clarify if I made any incorrect assumptions about Zigbee channel changes within an established network.

2 Likes

So here's all I think I know about channel changes..

AFAIK with HE's Zigbee stack channel changes aren't supposed to happen spontaneously; when the PAN first gets created the coordinator does an 'energy scan' to find the quietest channel (at that moment) and will stay on that channel until manually adjusted. Zigbee 'channel agility' is however an optional feature of the protocol that will allow a coordinator to initiate a network-wide channel change if/when it detects the current channel is too busy or noisy. However, this feature isn't implemented by HE.

Normally each joining device is programmed with a 'channel mask' (provisioned in the stack) that tells it which channels to scan for an available Zigbee network; when reset, it sends out beacon frames on each of these channels and waits for a coordinator (or router that can accept child devices) to reply and continue with the join. This channel mask is the reason why some devices (depending on the brand) will not join on certain channels, even though the radio is capable of receiving them.

When a channel change is manually performed, there are provisions in the protocol for a channel change notification to be broadcast to help the process along; otherwise, if a device doesn't receive a channel change notification and the PAN's channel gets changed, the device becomes orphaned and goes through the 'orphan scan/rejoin' process to scan all channels and rejoin the network. It's supposed to happen transparently, and quickly (a full scan of all channels takes only about 20 seconds or so).

I've changed channels on my hub a few times; the first time I noticed that while most of the devices found the new channel and rejoined within minutes, a couple did not, even hours later. I manually reset/rejoined them.

Another time I was trying to pair a couple of new GE/Jasco dimmer plugs. After many tries, they wouldn't pair on my hub's channel (18); before giving up I tried them with my SmartThings hub and they paired instantly. So I set my HE to the ST hub's channel (24) and sure enough both plugs paired instantly to HE. I quickly set HE's channel back to the original 18; both the plugs instantly followed the channel change back to 18 and have been working fine ever since. I assume that these GE/Jasco plugs channel mask is the reason why a join didn't work on 18 (yet they were able to migrate to that channel during an orphan scan/rejoin).

The 'official' advice is to wait 24 hours or so when doing a channel change; practically speaking this is may be good advice for a network with hundreds of devices but (my opinion) for small networks if the stragglers haven't found the new channel within a half hour or so, you probably will have to reset/rejoin them.

4 Likes

Above seems like the most important troubleshooting step at the moment...hopefully Bobby can rule in or out a HW issue.

Any way to narrow this down to a repeater, or a repeater type? @jameslslate - what Zigbee repeaters are you using?

Might also help to know what were the last few Zigbee devices you added, and how long ago?

3 Likes