[Solved] Z-Wave Failed - DEAD

Also I would add that a customer would not be complaining if they weren't interested / somewhat passionate about the platform to begin with. Maybe there are better ways to express things but usually it seems everyone wants this to succeed - there is so much potential here.

I do think when posting a complaint one should exercise some caution/humility - a lot of times (for me anyway) it's been user errors/lack of understanding rather than issues with HE. There is a lot going on under the hood. Would still like to see more advanced diagnostic reporting though.

2 Likes

We also should be aware of cultural background. Some cultures are more direct others are more easily offended if more finess is not use in language, others love what I call foreplay of language.

We are a diverse, multicultural, global community so some consideration needs to occur as well. We should be looking for what is being said (facts, descriptive elements, etc) rather how it is said.

3 Likes

I experienced the same and reported it. It seems to have settled down, but still misses once in a while.

I think there is a difference in how HE initializes Zigbee devices and manages their sleep behavior. Could be the long/short polling intervals, but who knows. Just about every motion and contact sensor occasionally fails to report an event. Not that it's the devices fault. The LED's always flash indicating activity, but the hub never gets the message.

I completely agree, a read-only view of processes and resource utilization would be extremely helpful. There are those of us with extremely deep technical backgrounds that are capable of doing advanced diagnostics before involving support. It's just information, why hide it? Two thumbs up for that request!

I trust that wasn't directed at me. Speaking only for myself, I've exhibited an extremely high level of patience. How many customers would wipe and reconnect hundreds of devices to try to solve a problem that support had not even acknowledged? The reality is that most would throw the hub back in the box and return it.

I think the feedback myself and others have provided to the HE team should be useful, and welcome by their dev team. The platform has bugs. There's no need for you to suppress that knowledge, nor should the developers to hide from it. There isn't a coder alive that has ever written a bug free application, and there probably never will be. That kind of feed back, even if it may be hard to swallow, is the only way an application or platform can ever truly mature.

I have hundreds of hours in doing just that at this point. Having a large network brings its own issues and a certain implied commitment to increased troubleshooting time and effort. But there comes a point where one needs to reevaluate the value proposition that a particular platform brings.

Completely agree. The community here has been great and very supportive. There's no need to suppress any feedback, good, bad, or indifferent.

That too is correct. When I make a decision or agree to do something, I do it with 100% effort and dedication. The move from SmartThings to Hubitat is no different. There however remains a breaking point in every endeavor, where the value proposition of a project diminishes to the point where a project is no longer feasible. For me, that point will be reached when the end of return period of the hub draws near. At that point it will have been about 7 weeks of trying to get it running stable, a more than reasonable attempt IMO. In my attempts to get this working stable, I've purchased 3 xBee's as well as several motion and contact sensors, to track down issues and replace suspect devices. None of those efforts have paid off at all.

Anyhow, I received a confirmation that support was looking into the issue. I'm encouraged that the devices are still showing in the radio device list. I trust (assume) that means they're still in the sticks Z-Wave routing table.

1 Like

Are you using custom drivers or built-in drivers. Which built-in drivers are are involved in the devices that do not work properly?

Also, are any of your Z-Wave devices joined securely? This is known to drag down your Z-Wave mesh if used for devices other than locks. Every command involves 3 times as much traffic. Spread that around, and you'd have issues.

Iris Z-Wave repeaters are known to malfunction in a Z-Wave mesh. If you have any of these, they should be removed as Z-Wave devices.

These are the sort of issues we've seen in other customers' systems with problems like the ones you are describing: Force-removed Z-Wave devices, secure Z-Wave joining for non-locks, Iris repeaters, and custom drivers.

1 Like

Can you see this on the device page? It occurred to me that one of my aeotec's has a different colored led. So I want to check this out in my system. The multi allows for secure pairing, but I wouldn't intentionally pair it that way.

Yes, under "Data" field. See this link for more details:

None of my Z-Wave devices are working. I've got around 40 GE Dimmers/Switches/Fan Controls all using stock drivers. I have the Z-Wave repeater side of the Iris plugs using a custom driver I wrote for SmartThings and ported to HE. All that does is a periodic health check to spot any that fall offline.

Those include 5 locks, a Zooz 4-in-1 sensor, an Aeon 6-in-1 sensor, and an Aeon Energy Meter G5 securely joined.

I disagree with that assertion. Those that have the upgraded firmware 0x20082010 are rock solid as repeaters. The issue that Hubitat and Smartthings has, is that during pairing, the hub fails to set Association Group #1 to the ID of the hub. This causes the Z-Wave module to believe it's fallen off the mesh. Once Group 1 is set, you can pull reports, MSR, etc. from the plug, and they remain connected and stable.

I won't rule out force removed devices as being an issue. Hubitat is unable to pair my Schlage BE469 deadbolts. This led to a few "phantom" devices that had to be removed during my numerous attempts. But that was a while ago. It looks like the Z-Wave stack failed around 2:37am last night. Looking at logs, that appears to be the last message received from any devices. That was well after everyone was in bed.

These will mess up your mesh no matter what driver you use. Remove them to start.

Remove and then rejoin the Zooz and Aeon sensors, and the Aeon Energy Meter. Be sure to rejoin as non-secure.

This is not what we are observing. If you have a lot of powered Z-Wave devices, you shouldn't need any repeaters. Humor me, remove them, see if it helps.

You could well have a very hosed up mesh due to a combination of all of these factors. Our next release will have a number of Z-Wave improvements, including with respect to ghost nodes. My suggestion would be to wait for that, or to rebuild your Z-Wave from scratch, with no Iris repeaters, only secure Z-Wave for locks, and no custom drivers. Follow the guide here: How to Build Your Z-Wave Mesh.

1 Like

Interesting. As this is one of the things I worry about, it would be good to hear more on that.

I am always paranoid (probably irrationally so) that something happened to create ghost devices on my zwave stick...

Hopefully the new change will at least provide a way of detecting/seeing that, if not a way to fix or remove them.

@bravenel are the majority of the issues described in the last couple of days z-wave related?

In this topic, yes. Beyond that, don't know which ones you are referring to, so a generalization won't be of use.

I must ask then, what is so different in the Hubitat Z-stack that makes these so bad? I ran them for literally years on SmartThings and Iris even before that. It's only on Hubitat that Ive ever been told to get rid of them. I'd like some technical details on how these "mess up" the mesh. My experience with these devices, starting with a private, closed hardware beta with the Iris team several years ago has proven to me that these are good Z-Wave repeaters.

I will try to rejoin the others. However, right now the radio is completely unresponsive. That needs to be solved before I can do anything else. Also, The Aeon Energy Meter G5 only pairs securely. If you pair it non-secure, it will not function.

The reason why I use the Iris Plugs for repeaters, is that many of them are connected to UPSs, whereas my switches are not. When the power goes out, and it does now and then, these will carry the Z-mesh while the switches are powered off.

The undetected ghost nodes to concern me, especially given the number of failures with pairing the locks I've experienced. But please understand that I just migrated from SmartThings, after years of running these exact same devices without any issue. From a scientific standpoint, I cannot blindly accept that these devices are now somehow "bad" whereas I have years of evidence to the contrary. If you have information from logs collected from my hub that you're basing this analysis on, I would truly appreciate it if you would share them.

If you are referring to my posts, those previous issues were Zigbee related, and were related to ongoing multiple, duplicate events, missed events, etc. Those are still happening.

Not what you want to hear I am sure, but my system took time to settle down after I first moved over from smart things. I remember reading at the time that adding a bunch of devices quickly would do this so I expected it. I had a less than consistent experience with smartthings for the 3 years I ran it though so I guess you could say I was tolerant.

@srwhite - I am wondering if you would consider removing a considerable portion of your Iris 3210-L outlets from you Hubitat hub (both Zigbee and Z-Wave) to see if you've simply hit some sort of limit...???

What makes your system stand out from most systems are those 50+ Iris outlets. As a test, could you reduce the number to about 10 to see how stable the system is? I know this is not a fix, but it might help in troubleshooting the root cause.

Of course, you'll need to get the Z-Wave half of the network up and running again before you can even try that. Have you tried restoring a very recent database backup to see if that gets things running again?

1 Like

I'm a very tolerant person. I'm also an engineer by trade, which is probably why some interpret my posts as being uncooperative or challenging of recommendations. When I migrated from SmartThings it was over a 2 week period. The first week was moving Zigbee devices including the Iris Plugs (and Z-Wave repeaters) but no Z-Wave devices, outside of a couple for testing. This was done very scientifically, starting closest to the hub and moving outward in a ring.

I was just posting here on Monday how stable and responsive the Z-Wave mesh has been. I guess I should have kept quiet.

I'm waiting on support for that. My instincts tell me that the Z-Wave radio is crashing, but the absence of logging makes that assertion impossible to validate.

Bruce may be on to something with respect to ghost devices. I went through a lot of challenges with trying to pair Schlage BE469 locks, and had to force remove them a few times.. But why last night after all this time?

I am an RF engineer by trade so I get it. I have found the hubitat folks very helpful. They have worked with many people including me and the end result is almost always a positive resolution. Almost always it a problem with a custom app or driver, so it's best to humor them :slight_smile:

You have time, so just wait and see what they say. If not amazon will take care of you.

You can probably appreciate this history then. I was working with Kianoosh and others at SmartThings last January advance testing their new Z-Wave firmware. I actually set up a "lab" in my basement with a hub and a couple random Z-Wave devices. Since you cannot force routes in Z-Wave, I accomplished the goal by spacing out devices and placing them inside steel electrical boxes between sheets of stainless steel for further signal attenuation. By doing so I was able to build a small 3 hop Z-Wave network using an old Aeon Gen 2 power meter to flood the network with power readings. Needless to say, it kept me busy for a few weeks, but I learned quite a bit about the Z-wave protocol during that time.

1 Like

So we have a mix of Zwave and zigbee issues.

Thanks for the clarification.