[Solved] Zigbee Instability is back!

From my test they are the real deal.

My house is 3 story 2500 sq ft.

My hubitat hub is sitting in one corner of my basement where I keep all my hubs and equipment. I have 3 ikea outlets on the main floor and one upstairs and also have my xbee upstairs in my office.

I have about 7 leak sensors 5 vibration sensors, 7 motion sensors, 10 temp sensors and a wack of contact sensors on every door/window. I also have a contact sensor inside my metal mailbox outside, a sensor on the side door of the garage and two vibration sensors on the garage doors at the front of the house (I'm actually shocked these stay connected as they are the furthest away from anything but they get used every day and still check in). My only other zigbee devices are 5 sengled bulbs in the basement. They are not routers.

Looking at the mesh, most of the devices connect directly to the hub or to the xbee. A few connect to the routers. I thought more would go through the routers but not as many as I though. They also jump around. I have devices in the basement sitting near the hub going through a router on the main floor. Its strange. I could pair one right next to the router and its still going directly to the hub. You can't assume they are using the router. But they do bounce around.

My wifi is on Channel 1 (its a eero mesh setup) and my hubitat hub zigbee channel is 23. I saw one of the biggest changes in reliability when I got the channels right. Higher channels though caused problems with zigbee devices. I started at 26 and moved down and it wasn't until 23 that it stabilized. I am also in a detached house. When I look at wifi networks in the area I see some but nothing over powering. I only have 1 neighbor and their wifi is very basic. So my wifi interference is very minimal if anything. I could see this being more of an issue in an apartment when your surrounded by wifi routers.

I have a script running (similar to the other device monitor scripts you find on the forum) that check the last activity of every xiaomi device. With my previous sylvania plugs I had it set to notify me if they didn't check in after 3 hours. Even though they missed check ins for that long they were still working though. Their checkin is just not making it to the hub. Now that I have all ikea outlets I set the script to notify me if they don't check in after two hours. I noticed that 99% of the devices would check in every hour but there would be that one that would check in at an hour and 20 minutes. It was still working though and been using 2 hours all week and I've gotten no notifications with missed checkins for these devices.

I've had the same issues on both the ST hub and the Hubitat hub. I never got it working as reliable on the ST hub but I also didn't stick around on ST long enough to work on it.

I've spent many hours working on this to get them working. Looking at mesh's, testing routers digging through driver code and writing my own apps to monitor. Its was a big investment but it has paid off and is now stable and working great.

I may be an exception here. I don't know if anybody monitors it like I do or has as many but I keep a really close eye on it all because if they get stuck it affects other automation's such as my heating, garage doors, security system etc.

I'm starting to wonder if some of the problems too are hardware related. They seem really extreme in some cases. You just never know.

(Another long ■■■ post)

2 Likes

That is not right. You should not have to put the hub back into inclusion to reconnect these devices. I believe you've helped me confirm this as a bug.

This particular Iris motion sensor, I wound up deleting completely from Hubitat and attempted to pair it as a new device. It's been initializing for over an hour. It is less than 10" from the hub, and a foot away from one of my connected XBees. You can see the hub's stick behind the monitor. The mounted to the wall.

Hubitat absolutely will not allow this motion sensor to reconnect. Iris did in about 11 seconds. I was hoping that bouncing it to Iris and back would change the behavior but it did not.

@mike.maxwell @chuck.schwer there are now two confirmed cases of the Iris/Centralite sensors dropping off of the network when the batteries are removed.

You're in good company. I monitored my SmartThings system like a hawk. Once a week I would check every sensor to make sure it was reporting and the battery had not died.

1 Like

I just ran a quick test using a spare Iris 3326-L motion sensor. I was able to add it to my hub (2.0.4.117], remove it, and re-add it (required a battery pull to get it into pairing mode, but I did not use the reset button.) I did this twice and it worked as expected. No issues. I then left it paired to my hub, pulled the battery, and then reinstalled the battery. It recovered normal behavior without any issues whatsoever.

I am not saying there is not an issue, just that I cannot reproduce it.

BTW - My 3326-L Motion sensors are running the latest Iris firmware version provided by the Iris hub.

1 Like

I'm really hoping that you have a bum radio, not because you deserved one, but I have no other explanation for your zigbee issues currently.
We should probably hold off on further testing until you receive your replacement.
At least 10 of my 30 motion detectors in production are these guys, and I've not seen the issues you describe, so we have another anomaly here.

2 Likes

@srwhite Any good writeups on how to load the Zigbee 3firmware on the Xbee 3's?

I am thinking of trying them out.

Only thing I could think of for not being able to discover your device is because of the failed initialization and you now have a ghost device in the database. This actually happened to be before and the only way to add the device back is to wait for the database to get rid of this ghost device. Just guessing but I think the hub does this nightly.
The First pic was yesterday and Hanged during discovery. Was not able to add it back. Will try again later.
The second is a pic of my zigbee logging and you can see a devices with HEX number and no name. This was caused by a hanged initialized. Multiple reset and hub reboots did not get rid of it. I waited 2 days and was able to add it back.

Just to support this being an anaomoly; I have 6 battery powered devices on the second floor; 16 battery powered devices on the 1st floor. I had issues until I added 5 plugs. But...I have no issue with devices dropping offline.

*Note: All the devices are zigbee. I have 60+ zwave devices throughout my house.

Hoops everyone is enjoying the storm! Yesterday was a mix of shoveling and tearing down the temporary Iris installation and moving all of the plugs back to Hubitat. All 11 of the new SmartThings v3 plugs I connected on Friday remain online, stable, and instantly responsive. So that’s a plus.

To recap.. I set up an experiment by taking my old Iris hub out of storage and moving a final total of 31 Iris SmartPlugs from Hubitat, plus some battery powered devices that were dropping off my Hubitat mesh, including 2 motion sensors and 5 contact sensors. All of those devices had up-to-date firmware as reported by Iris, all remained responsive and connected to Iris for the 30+ hours of the experiment.

Before moving devices back to Hubitat, I double-checked to ensure that all of them were completely deleted so they would pair as a brand new device. There was a considerable difference pairing each plug. Some took 20-30 seconds, while others took 5 minuted or more. Several hub reboots were required, and at one point Zigbee died completely (went to “Initializing”) although that was fixed with a reboot.

This morning, 16 of the 31 plugs and ALL of the battery powered devices, all of which were reconnected as new were completely dead this morning. I even powered down the hub for a solid hour. I still cannot even wake them up the plugs by power cycling them, refresh, etc. I probably should have left them on Iris, but I personally utterly and completely detest their mobile app. The only common thread is that these are Centralite devices. The new SmartThings plugs (Samjin) are all working fine.

Thanks to @bravenel who is sending another hub and stick (actually 2), there’s still quite a bit more testing to go. He’s offered some test scenarios to work through. With any luck I’ll be able to start that on Tuesday. Fingers crossed that it’s just a bad stick!

4 Likes

Best of Luck mate

1 Like

Yes all the best. I think there are quite a few people keeping an eye on the issues you’re having and hoping a successful out come for you is found soon.

5 Likes

Fingers are crossed here too. Thanks!

2 Likes

@chuck.schwer I have been pondering something. I have noticed that the 11 SmartThings plugs have been rock solid, despite the rest of the mesh being flakey. In contrast, routing devices that have been deleted and re-added lose their routes within minutes of being reconnected. Could the routing tables in the other routers be holding stale routes, or could the hub be holding onto stale routes??

XCTU is showing 16 unreachable routers. All but 2 are the plugs I reconnected yesterday. Very strange? I think so.

I wonder if you had not moved any Iris plugs back but had added all the battery devices to your Hubitat / ST plugs if they would have remained connected and functional. There are a huge number of routers in your setup.

Zigbee failed completely this morning. It was working early this morning (about 10% of devices were anyhow) until about 20 minutes ago when I couldn't control anything. A reboot did not restore, so I decided to reset (nuke) the Zigbee stack again since the replacement hub will arrive tomorrow. Zigbee stayed offline after nuking the stick, remained offline after a complete power cycle. I had to reset the stick a second time to get Zigbee to initialize.

I'm going to repeat my Iris experiment from the weekend on a clean Zigbee stack (on a very questionable stick) to see how it performs for the next 24-36 hours until the new hardware arrives. Nothing to lose at this point...

Are you on your replacement radio yet?

No.. Looks like that will come tomorrow. Since it crashed I decided to reset the stick and play around today. It's a holiday and about 0 degrees (F) outside, so I've got nothing better to do.

1 Like

Yeah, Amazon supposed to deliver my lock today and it's a holiday..... boring, problably I will factory restore all the hubs....

I've got 30 plugs connected to the temp Zigbee mesh on the hub. Two left to go on this floor. So far so good, although I made it a couple days until problems surfaced.

1 Like

Did you get the sniffer?

No... Looks like that's coming Wednesday.

WIth 30 devices on the Hubitat Zigbee mesh, I have already seen two SmartPlugs fail to respond to commands. These are the same plugs I moved to Iris Friday - Sunday and had no issues with.

I really hope it is a Zigbee stick issue!