I've been having trouble with zwave devices and a rule that turns on six lights. When the rule blasts out the commands, it seems that either RM gets clogged up and becomes unstable or the zwave stack gets in a bad state. So this morning I decided to try and replicate the problem outside of RM.
I made a dashboard with all the lights on it. If I rapid-fire a light switch and look at the zwave logs, I can see it changing routes all over the place even though it's the only device in the log. just by sending it repeated "Off" commands just so it doesn't wear itself out and I can replicate the problem pretty easily.
I think this is simulating what is happening when a RM rule fires off a bunch of commands at the same time. So maybe this is something going on with the zwave stack somewhere. I also notice that a lot of my devices have a high number of route changes - some of them are upwards of 18-20. Guessing this is happening when rules are firing off like I just simulated and the routes are getting changed for some reason.
If someone goes into the device properties with another window showing zwave logs, does repeatedly clicking the OFF button cause the routes to start changing for that device?
Here is me clicking OFF once about every 5 seconds.. I think I did it three times to show a normal command and the time it takes for it to happen.
Then I spammed the OFF button about twenty times in a row which seemed to clog things up and triggered route changes. Look at the timestamps and how much time went by. The last two was me clicking refresh to make sure it was done but you can see that the routes started to change when I did the click spam. I think this is what is happening when we run rules from RM and it blasts out a bunch of commands at the same time.
Given the number of current threads referencing Z-Wave lockups, I'm hopeful this could be a very important discovery, @Allistah.
I followed your steps and recreated the exact same behaviour. Repeated commands (in my case, an "Off" command to a Red-series Inovelli dimmer) caused the Z-Wave stack to lock-up and the route to change.
Ok, cool.. Not cool, but you know.. Cool that you were able to duplicate it.
I'm hoping that someone at Hubitat can use that to spam the zwave stack and duplicate the issue. It should be more robust and be able to handle it all no matter how many/fast commands come through. I would guess that at some point commands need to get dropped if it gets too many too fast but it shouldn't be triggering route changes and cause things to get all bound up.
Hoping one of the staff will see this and let us know they're able to repro it or not.
Route changes are handled by the SDK and completely out of control of the hub software.. Route changes happen automatically on failed packet delivery..
Ah, ok. Awesome on the fix. Any way I can help test it out? I'd love to bang on it for a while. I have been in QA for 30 years and am a QA Manager. Would love to be able to help out here..
Just to throw my hat in; after the upgrade I was having real backup issues. Some commands were taking a full minute to execute (as verified in the logs). Everything was fine for 30 seconds and then backed up...
My wife’s favorite phrase is “Alexa shut down the house”, which sends off commands to 56 devices. Anyway, I went to bed after a reset, reboot and multiple tests. I was seeing new routes all over the place on the zwave log. I was rather frustrated; this morning, everything was fine...
The wife asked what I did to “fix it”. I told her I used tried and true engineering techniques. I didn’t tell her I just waited for it to go away. Things are better now; after I waited and let things settle.
Are these devices having problems?.. These numbers could be indication of an issue, or could be your mesh settling down.. If they continue to increase, you may want to consider a repeater in route.
While route changes on most of my devices are low, on some it is high (150-200). I have 70+ repeating devices, all zwave plus, and the devices with high route changes work 100% of the time. So
Why do a few of the devices have high route changes when they have 40-50 neighbors each? Who knows? AKA voodoo.
I'm in the camp of "if it works fine, then ignore it". In the end you can only do exactly two things - start replacing devices to see if it changes, or add more repeaters and see if it changes. But even then you can't control it, so doing either of those things may not change it at all (and cost you a bunch of money for nothing).
No, they are not having many issues. All these devices are older GE switches and not Plus devices and all of these are within 25 feet of the hub. Not sure what repeater would help with these non Plus z-wave devices. I was having a lot more problems with these devices when I had the Z-Wave Poller app enabled but when I removed the app and set up a rule in RM to poll every 30s, things got better.
Glad I'm not the only one seeing this. Since I seem to have stopped the hub from flooding after disabling Z-wave Poller I think I am going to leave it alone.
Then .. I wouldn't worry about it.. If you were having issues, these numbers would help me to identify the cause .. But sometimes they can change with no problems at all.
Thank you for your interest in joining the beta program. We are getting ready to add a few new users to the program. If you didn't do so already, please make sure to fill out the following form:
Would love to be able to help out here..
