Tortured ZWave route

C8-Pro
Platform: 2.4.3.175
Z-Wave-JS (legacy is turned off)
Rogue device is a Zooz Zen76 >> 0x0071

I have one device with a strange route back to the hub. It is quite possible the node has a ZWave "RF hole" but I don't understand why the many hops back to the hub when the first hop to node 25 has a direct shot to the hub? How and who decides how these multi hop routes are built? Can they be reset or over run? Clearly the chosen route isn't the best path.....

rebuild route then refresh. See if it is better.

1 Like

Technically, yes, per the Z-Wave Spec, but this Applicaton Routing, is not exposed in HE, which does the basic source route (with 5 routes), per the Z-Wave spec. As @TArman nodes, your only real chance of changing this in HE, is to rebuild the route, and as long as the route is working, then it's likely going to stick with that path. You can try physically moving the device, to potentially force a link failure, and then it will try one of the alternate paths, or search for a new path with explorer frames.

Depending on the device, you can also try adding the endpoint as a LR node, then it's just a direct path, with no routing involved at all..

For some good background context is called out by SiLabs here (see P5)
And the more advance APR topic is called out here

2 Likes

Thanks to both. I am going to try to repair is as an LR node since it is LR capable. This will be my first LR device so fingers crossed. It hadn't occurred to me to try LR but if it works I have a couple devices it might be a good solution for..

There is something amiss with how the mesh hops are created but I don't expect a fix for that is trivial or forthcoming. So LR probably my best option.

I find that most of the time, when I reroute, it picks a more direct path, even if the current path is working adequately.


2 Likes

something is botched now as it won't complete a rebuild route now. Stuck for 30 minutes and won't complete..... anyway going the LR route....

May be time for a shutdown with 30 sec power off, just to get the Zwave radio into a known stable state, before doing too much in the way of changes - YMMV :man_shrugging:

4 Likes

I have found that if you aren't experiencing issues, worrying about routes on the Z-Wave Details page is an unecessary use of your worry-budget. :slight_smile:

And FWIW, if you do a Rebuild Route you won't necessarily see a change right away.

Good advice from @GuyMan to do the shut down/pull power/30s/restore power dance before you do anything else. :slight_smile:

3 Likes

OK moved the problem node and a couple other fringe units to LR. They seem to be fine on LR link.

I must say the LR inclusion process is a tad cryptic and poorly documented. I had to experiment a few ways before getting it right. Regular Z-Wave pairing is super simple. Smart start seems anything but. I still don't know how they paired as LR as I never got to a prompt to select regular pair (mesh) vs LR... fortunately they self selected LR.

Final thought: Changing a couple of the poorer performing nodes to LR has improved my large All On and All Off rules. They tended to get hung up and stumble due to the flood of ZWave traffic and I assume hops and retransmissions. It seems to be better with three nodes moved to LR.

1 Like

More on this. There is definitely a problem with how ZWave devices decides on multi hop routes. I've been chasing this issue for weeks now and its obvious to me. I suspect this is deep in the ST Micro code and how the mesh is managed.

I created an app to sequentially turn on a large number of lights. It waits until the ACK of the prior device before sending the next command in the list. The idea was to avoid flooding the ZWave network with a bunch of near simultaneous commands. The app writes to the log the time duration from command sent to ACK for each device and then the end to end duration of all the devices at the very end. Large variation of ACK durations even those with direct hops.

Below is the ZWave entry for Zooz 77 800 series dimmer. It is installed about 6' from Hubitat. Note the 3 hop route. This multi hop is clogging the network with knock on messages that aren't required as the device should direct connect to the hub. I have seen this extensively in this network on many nodes. So much so I coined the term "Tortured Route".

I also can't reconcile the duration variation across devices. FR Nook Lamp (9013ms), Powder Rm Hallway (8232ms), and Foyer Main (7860ms) are all direct routes to the hub. RSSi and PER is nothing unusual. I cant fathom why the long delays..


image

ZWave Device table:





full system logs for same sequence:




image

1 Like

Another chapter to this weird ZWave mesh phenomenon.

Three 800 series Zooz switches collocated in a 5 gang bank 33' from hub with direct line of sight from face of switches to hub antenna.

The ZEN76 (node 117) has down shifted to 9.6kbps and after a rebuild/refresh doesn't restore back to faster speed or direct shot. As a result it has a massive RTT of 301ms compared to its neighbors. I tried power cycling the ZEN76 and then a rebuild/refresh but it remained on the 9.6 path.

So I think I'll switch the ZEN76 to LR mode but it shouldn't be necessary given the neighbors performance.

This is just further evidence the ZWave mesh route management has issues. At a minimum devices should gradually seek (behind the scenes) to find a direct no hop route if one is available, in absence of that they should migrate to the least hop route possible. I guess this is what you get when the ZWave technology is essentially a monopoly from a single vendor.



Update:

Added device as LR. Now RTT 10x faster, RSSi about 10dB better..

LR is da bomb!

LR definitely has better RF link performance. I estimate it can go 10-12dB lower signal than mesh. The claims of 1+ mile range seem preposterous to me however. I have one LR device one story down and maybe 25' feet away that shows RSSi of -88db. I've seen LR work down to -100 and even -102 but that seems about where it drops out. So not sure how you get 1-2 mile range without out direct line of sight, outdoors, optimized antennas with ideal ground planes (i.e. not ones you could practically use), zero noise environment, ... In short, a lot of puffery around Z-Wave LR range claims. It definitely has its pluses but.... overblown.

For me I have a few devices that are direct associated (ZEN77 dimmers used in virtual 4 way setup) so these devices have to be on the mesh. Also I have some legacy Z-Wave that legitimately need a hop to get to the hub. As a result I can't go all LR and the balancing act is to have enough mesh to be resilient enough for those nodes that don't have a choice. I started with 3 nodes as LR to try to keep the mesh healthy. I have moved up to 15 LR now to deal with problem mesh nodes but now I risk hollowing out the mesh for those that must stay there (72 Z-Wave devices in total).

I also dropped in a couple of Matter smart plugs to retire some old ZWave. Just to dip my toe in some Matter.

I moved Hubitat away from my WiFi router a LAN switch to avoid Hubitat receiver desensitization (near field RF from those devices) and added a 3rd party (Z-Wave) antenna in place of the stock Hubitat unit.

All these factors combined and my network is working better than it ever has with the addition of more 800 series devices (retired some Z-Wave Plus and earlier units), and the new mix of mesh/LR.

I still feel there are issues with weak mesh nodes drifting into these tortured routes and never self healing but I have dealt with it other ways now. I don't suspect there is much urgency at Silicon Labs to do much about this anyway.

1 Like

No other way to get around that? I'm not into dimmers, but in the past I did try a ZEN34 with a Zooz dimmer, forget the number at the moment, and association did work. You'd think there'd be a Z-wave alternative at this point.

One thing you lose when you go LR is you cannot direct associate (DA) devices anymore. Not sure how much people use DA anymore but that is one thing you give up.

The 4 way virtual dimming is a clever capability with Zooz devices. Gives you that high end multi location dimming capability. You can do it without DA via a hub and rules but when I tried that before there was too much lag to get a smooth dim up down experience. You would tend to overshoot the target so wasn't intuitive. With direct association it works well and mirrors hardwired dimmers with companion wired dimmer remotes.

2 Likes

Again, while you may want it to work this way, that's NOT what the Zwave spec calls out in terms of routing behavior. See this earlier note: Automated Cleaning of ZWave "Tortured Routes" - #14 by GuyMan - As the SiLabs radio handles the routing controller, and the spec is well established. This is consumer grade stuff, not OSPF or EIGRP.

Per the other thread, the best bet for "all on/all off" behavior would be multicast support in either ZW or Matter (that seems like a reasonable change to make on the HE side as useful additional capabilities, given that it works with ZB). But your not going to get the ZW routing algorithms changed, regardless of how non-sensible they may be.

As your noted, your getting down to the balancing act of removing many of the redundant mesh links, given your conversion of problematic ZW mesh nodes to LR. Personally, I prefer working with Matter (WiFi, but Thread is basically smarter/better ZB), as you have more specific controls and visibility (with TCP/IP and wireless APs, etc.), and Inovelli white dimmers are very capable given the last FW upgrades. That all said, the DA stuff you doing in Zooz, is likely the best option you have for "hubless/fast" dimmer controls.

2 Likes