Automated Test Suite or Stable Channel

Hmm :rofl: Good luck with that.

3 Likes

Unless I’ve missed something there hasn’t been a change to the Zwave SDK on HE since they went to 7.15, so your “instability” may have other causes. Given what I perceive as a very low tolerance for disruption, I’d recommend that you consider staying at least 1 full release back, i.e. if you’re on 2.2.9.x don’t go to 2.3.0.x until 2.3.1.x is out.

7 Likes

This sounds like a good recommendation to me.

Every major release undergoes internal testing by staff, then public beta testing by users who have volunteered for that task. After a major release is let out into the wild, bugs that were not previously identified are usually patched very quickly with hotfixes that are released as bugs are reported.

Staying one release back would allow one to avoid all of that.

5 Likes

Compared to my Hubitat experience... it actually has been, now that I've gotten all my automations migrated over.

1 Like

I think @Angus_M was alluding to the notion that most people would not consider home assistant to be a hands-off solution for home automation.

To that point, here is their release announcement for last month’s platform update. Notice there is an entire section devoted to “breaking changes,” suggesting it’s not exactly an uncommon event.

4 Likes

I had a fully functional Home Assistant installation a few years ago that was irretrievably broken by a system upgrade, which made me evaluate Hubitat. I still have a Home Assistant installation, but it is ancillary to my Hubitat hubs. And Home Assistant Device Bridge makes it simple to bring devices from Home Assistant into Hubitat.

3 Likes

You can choose which version you update to. This is the key thing that is missing from Hubitat for me. How can a good faith argument be made to "just stay a major version behind", when, when you do finally update, you're forced to update to the latest release.

I very much appreciate that they explicitly call out breaking changes.

I'm not trying to get anyone else to change-- I was simply commiserating with OP and saying that I had personally found a better solution for my own use case and expectations.

A fair point. I guess a more precise suggestion would be to update to the latest version after several weeks or months have passed since the last hotfix. Most bugs are reported and fixed within days to a few weeks of a major release.

Not updating at all is still a valid option for some, since not every platform update adds something of value for every single user.

I appreciate that the HA devs call out their breaking changes too. But I appreciate more that intentional breaking changes in Hubitat platform updates are extremely rare.

2 Likes

Quite the opposite. Rule 2.5 rules still work in 2.3.0.119 ….

2 Likes

I actually have ceased being "johnny-on-the-spot" with upgrades ...almost to the point of concerned paralysis. I was in here this morning considering an upgrade because I have time right now to manage any negative consequences. That's a lousy way to look at HE ownership for the masses.

I have critical seasonal uses (like freeze protection right now) which have increased my fear of system upgrades; and with key Community Apps in the mix my fear runs both ways depending on the developer- NOT keeping up, or upgrading too soon. Catch-22

I wonder what the sentiment of HE owners is in different camps on this subject. I wonder if the majority wouldn't mind if the frequent incremental upgrades X.X.X just focused on adding devices, drivers, and material bug fixes, and X.X upgrades were when the real changes (after a lot of testing and preliminary distribution to a Developer Distribution List ) were made.

Somebody is going to step up and say, "well, that's how it is now". ... I'm not feelin that.

The other thing to keep in mind, is that unlike a lot of platforms, HE does provide an easy way to revert to a prior release (and not just the release you were on prior). So an upgrade doesn’t need to be a life long commitment.

5 Likes

Put most of the blame on Silicon Labs. Z-Wave 700 was released far before its time. Ghosts are created and only reliably removed via a secondary controller (Z-Stick). Nuking the Z-Wave database works, but is not necessary. Don't run S2 or S0 unless required (locks, etc.). Even S2 has implementation issues in some devices. The ideas behind Z-Wave are fine. But their implementation leaves a great deal to be desired. A single device falling off the network can screw up a mesh? Telling people to "run a repair"? All these issues will remain to some degree with Home Assistant. Z-Wave will have a significant marketplace issue if Thread/Matter works as promised. There's probably a reason Amazon isn't using Z-Wave for their Sidewalk technology.

One area where users have figured out a potential Hubitat specific issue is the antenna implementation. There's a great thread on adding external antennas. Many people have said the new antennas have made an appreciable positive difference in their Z-Wave performance.

4 Likes

I continue to do this with most platform upgrades and suffer essentially no consequences.

It 100% won’t be me. I understand some people have issues (and Hubitat staff work hard to try to improve on that). But that hardly means most platform updates are a train wreck for most users.

This.

2 Likes

Not to beat a dead horse here, but despite being a beta tester, I rarely find bugs or have issues. I do update and use every beta and normal release, often within about an hour of publication.

So while I have empathy for those who have issues upon upgrading, it completely astonishes me that people are encountering such bad bugs or losing whole zwave networks to the point where they are afraid to upgrade.

I will repeat that there must be something going on with certain hub setups or certain devices attached to a hub to be causing this experience for some users. For example, I suspect that the OP has some known troublemaking Zwave devices like the Zen25 crashing his mesh, or some other failing or flaky device on his mesh. We can't see what he has and he has not posted screenshots, so we are in the dark here.

1 Like

That is pretty much what happens in Beta testing. It may not be as formalized as you are speaking of, but we sure do test a bunch of stuff, especially new things. I am not sure of their behind the scenes testing (Alpha test or otherwise), but I am almost certain that they do some level of this before Beta.

When there is a public release, it IS considered stable as far as I am aware.

I think you need to start a new post and let us see your Zwave devices. You should not be having issues like that. Additionally, you might work with support to have them use the diagnostic logs to see what is going on behind the scenes.

This has not been the experience of most of the Zwave users on here, and it should not act the way you are stating. Please have the community and official support help you instead of suffering with this.

2 Likes

I'm really not going by this thread and the Zwave probs in particular.

Just looking at the Release Notes and subsequent fixes to introduced problems....; there are plenty of opportunities to say whew, glad I waited .

Not throwing shade, sh_t happens in complicated systems. I really think this is just a debate about the cost/benefit to a consequential release going out to everyone sooner rather than later.

Maybe introduce the concept of a Gamma stage, e.g. we think we've vetted this well enough through the Beta testing but for the next 30 days consider this version up for a wider audience to vet.

God knows the whole industry has gotten out of a lot of the testing process that use to take MAN YEARS of INTERNAL effort. To be fair, it use to take man years to develop stuff that now seems to be doable in a month. SOOOO there's that.... :crazy_face:

I am glad that you appreciate the speed at which our engineers resolve issues caused by bugs that have not been caught during alpha or beta testing. We know there is always room for improvement to release updates to the public that have less problems, and to limit the future problems, we have recently expanded our Beta testing group. We are very fortunate to have so many beta testers who are helping us deliver a better product, with each release.

With that being said, I would like to clarify, that a main database problem does not affect the Z-Wave database. There are two distinct databases and we patch the Z-Wave database when Silicon Labs releases an update and the update is thoroughly tested by our engineers.

Regarding your Z-Wave issues, I wish that you had reached out to our support, or here in the community, before you made the decision to reset the radio. While some persistent issues prior to last Z-Wave update may have required a radio reset, there are other ways to resolve Z-Wave issues that do not involve nuking the radio.

6 Likes

This is known as the "stable channel"

1 Like

I'll also say not much has been done with z-wave since 2.2.8 I believe. At worst updates expose corruption in the database or other things that have been problematic. @k3vmcd Why haven't you reached out to solved these issues. The fact that you say every update is breaking them is indicative of something else in your mesh or database. Posting things like your z-wave details page is a good way to help us help you. This way we can see what devices you have, if there is something else going on, we can query you and zero in on your issues. For instance, my first question would be if you have any zooz products in the mix. Do you have any beaming repeaters to help stabilize things. (just because something mains powered repeats like a switch or outlet doesn't mean it's getting very far) We'd like to see your rssi for various devices, wed like to see logs, what 3rd party apps are you running? These are all important questions for not just the community but support to help you. As others have said, stay in what works for you, but upgrading normally won't break a mesh unless something z-wave specific has been done. (which it hasn't in a long time)

1 Like

I would love to be able to do that. However, you don’t get to chose which version to upgrade to, so as soon as the new version is out, you’ve lost the chance to upgrade to the previous version. I have raised this in the past as a feature request, but was quickly shot down by @bravenel Feature request: Select what version to upgrade to

1 Like