Power Outages - Beyond the UPS. Will we need to fear this forever?

💬 Lounge
1

@thebearmay is doing great stuff again.

Not long ago I had an outage that out-lasted the UPS. Not having anything in place to sense and preempt their impending fate, the HE and datacoms just ran off the power cliff. Thankfully everything came back up unscathed (or at least I think so).

This got me thinkin.... is it impossible to get where the HE losing power is not a terminal sentence for it's configuration, database or otherwise? I thought things were A LOT BETTER than they ever have been in this regard for HE. Is post-outage stability likely to improve or always be at risk?

I'm fully understanding of the benefits of keeping things on a steady power supply (UPS) ....but in all actuality, I would not want to rush to shutting down an HE upon a mains power outage if I could help it. (As The Bear May App allows). There's battery based devices I'd like monitored for some time thereafter, and with comms on that UPS and working, I actually COULD be told some things, ...like for example, that the power outage may have been due to someone flipping an external panel switch unbeknownst to me.

The REALLY big thing to me (with backup power being sipped) ...would be "preparing for the refresh" once/if the UPS was totally exhausted. The Bear May be paving the way here.

That is, I'd love the recovery of the environment to be seamless (i.e. back to "as was", or even better "as it now should be" given the time/mode and state of devices no matter what states various devices automatically come up (on/off) per firmware design).

I realize the "as it NOW should be" is not at all something that HE might do without a whole "post outage recovery" being thought through and configured for a particular environment.

On this note, has anyone written a Welcome to Hubitat segment addressing the key things to consider when writing rules to be post-power outage-friendly? I.E. the consequences of one time triggers, mode transitions, mode states, device state checking/recovery, variable use/capture (state storage), rule re-starts, etc.

Maybe this should be moved to the LOUNGE ?

2

I think HE probably has the same susceptibility to problems due to loss of power just like pulling the plug on a desktop PC (or yanking out the battery on a laptop while running). All of these systems are R/W and could be writing to storage at the exact moment power is lost. Linux (HE's OS) is a little more forgiving under EXT4 than Windows but it still can happen.

What would be a nice feature is a USB jack on the HE that communicates with the UPS about battery levels just like Windows, MacOS, and other OSs, where the HE can do a graceful shutdown all by itself.

2 Likes
3

I think what you are asking for is what is being discussed in this thread here

That thread was started after a USB-C UPS was created for about 15 people that functions amazing for the HE Hub. It is basically a 5v battery pack that will keep the HE up for hours if power is lost.

That thread talks about taking that device to the next level with integrated Zwave/Zigbee radio to communicate state to the hub so it can be managed better. There is even talk about having a method to restart the hub after power has been restored to the ups.

The one thing that really stands out to me though about that UPS is that when I tested mine it lasted 14.5 hours. If my house is without power for 14 hours I am dealing with much larger issues then my HE environment most likely. It is amazing how little power the hub does sip and how long it can run with even a very modest battery backup.

4 Likes
4

Thanks for both posts so far.

So it's the ole "caught while write" and doing it wrong thing. Some aspects of that were the nemesis of yesteryear computing...I thought we had lots low level contingencies built into systems these days that at least gave that kind of transaction enough time to cease operations gracefully.

Thanks also for the link to that other thread. That indeed is barking up the same tree. I knew about the previous USB battery efforts but not a full fledged effort that's got some smarts to it.

5

Yea.. I am looking forward to see where this goes. I have two of the first run of that battery backups and I am very impressed. It runs great.

6

This seems like the biggest benefit offered by that project.

Anyone know if other automation hubs have this same issue? The old ST hub or Home Assistant, for example. A UPS seems like something that the "average" consumer would be unlikely to consider.

At work, our main automation systems are all on UPS power but each controller also has a module that contains a "super capacitor" that provides just enough time for the CPU to execute a controlled shutdown (if UPS power fails).

7

FWIW I recently added a UPS for my hub. It is my intention to have a rule to shut down the hub after xxx minutes of power loss. I don't know the time the UPS can hold up but xxx will be about 30 minutes before that.

Ideally I would like to have a circuit in the UPS that will sense if the hub is powered down and cycle power to the hub when the mains return. So far I've not found an acceptably simple solution.

8

I've been thinking about this. I think you could do it with a ZOOZ ZEN17 and a generic 120 volt relay. Probably less than $50 if you don't count the UPS.

Physically, the power to the hub would be routed from the UPS through both of the NO relays on the ZEN17 in parallel so that closing one or both relays would power the hub. The contacts of the 120 volt relay would be connected to the switch inputs (S1). The 120 volt relay coil would be wired to utility power. The ZEN17 would be plugged in to UPS power.

ZEN17 would be configured so that changes to S1 state are used to control relay 1 (R1). The switch would indicate if utility power was present (120 volt relay energized, contacts closed, R1 on/closed) or absent (coil de-energized, contacts open, R1 open/off). The ZEN17 relay 2 (R2) would be configured for ON after a power loss with an auto-off time of 10 minutes or so... however long a startup or shutdown of the hub takes plus some margin for error.

A rule on the hub would trigger on hub boot run every 3 minutes to send an ON signal to R2. This rule needs to run indefinitely.

A second rule would initiate a hub shutdown after seeing utility power loss based on the ZEN17 switch input.

So, when everything is "normal" both R1 And R2 are closed and the hub has power. On loss of utility power the 120 volt relay is de-energized and its contacts connected to S1 open. R1 opens but R2 remains closed and hub remains powered by the UPS but "sees" that utility power is gone because S1/R1 is open/off. The shutdown rule is triggered and the hub begins to shutdown. At some point, the rule keeping R2 on stops running and the ZEN17 begins its auto-off timer. Hub shutdown completes gracefully. R2 timer expires and R2 opens killing power to the hub. UPS remains on (powering the ZEN17) as long as batteries last.

At some future time, utility power is restored, the 120 volt relay is energized closing S1 and turning on R1. Hub is powered up and begins to boot up. At some point the rule controlling R2 is triggered and R2 closes bringing us back to where we started.

If the outage lasted long enough to deplete the UPS, the ZEN17 would see a power failure.
Restoration of utility power would cause R2 to close and power up the hub based on the configuration noted above.

There is at least one "gotcha" here. If utility power is restored before the ZEN17 timer expires, the power to the hub is never removed and it will sit idle in a shutdown-but-still-powered state. On the other end of the outage, if power is restored briefly and then drops again before the hub can finish bootup, it will be subjected to a hard power loss.

Honestly though, if a basic PC UPS can keep a hub up for over 8 hours, and @thebearmay's app linked above can shut the hub down at 7-ish hours, how likely is it that we will ever be in the situation that our hub powered down due to an outage? And is it worth all this effort to get it to automatically restart after that extremely rare event?

1 Like
9

This is why we have backup power in data centers, server rooms and closets for not just the servers but the san's and nas'. Power corrupts, power outages corrupt absolutely...

3 Likes
10

While I admittedly just skimmed most of this thread, it seems the simplest solution might be either of 1) Just have a Rule that shuts the hub down after X minutes of a sensor going to battery power and staying there, with X being a bit less than the number of minutes you UPS should last, or 2) Buy an automatic backup generator, if power outages are often a "thing" where you live (certainly the case for us).

Again, I just skimmed this, but those solutions should solve the vast majority of problems for users.

11

I have my hubs shutdown after they have been on UPS for 5 minutes. Hey if there is no power no devices are controllable in the house. Last year I moved to using a Ecoflow River Pro driving the UPS since I work from home and need to keep the wifi and internet running while the power is out. The UPS covers the period of power outage until the Ecoflow inverter picks up. It would be on the UPS for about a second or two and then report mains again. NUT UPS monitor works great for this and the Ecoflow runtime states I can keep the network running for about 10 hours when on battery.

image
image1515×396 22.1 KB

12

Yeah, in a bygone era I actually spec'd a few of those that sat in separate rooms from what we were needing to keep up. But with the shrinkage of the power requirements for everything (I can safely say)...I was just expecting that for "stuff as small but still important as what we're talking" below would be a more common built-in feature minus the "super" designation

I have multiple tombstone-ish APC UPSs, admittedly a couple are remnants from the years I didn't want PCs to go down...power conditioning aside the laptops with batteries take up some of the need. I don't just hang the HE off one, there's comms devices in locations I also have HEs. Admittedly these APCs don't have the freshest of batteries but still I think the last prolonged power outage I got at least 30 minutes from them.

I like the idea of something that JUST takes care of the HE and does so with a smart interconnect...but I'll have to admit, reading @dylan.c post above (outlining a well thought out solution) I was thinking to myself....wouldn't it be cool if a a square centimeter of board space in the HE could be dedicated to a bunch of that somehow.

I don't think I've heard anybody ever complain about the HE being "too big" of a footprint. Hell, give me another USB port to plug an optional module out the back of it to accomplish this then (sans battery but maybe not capacitor, sub-super of course :wink:).

EDIT: Maybe not, "sans battery", this is where those guys are going in that other thread and with the latest in battery technology...this really doesn't have to be too big of a unit as they've proved.

EDIT Correction: @dylan.c well.... "super" is indeed the important and necessary designation on the capacitor called for "to buy time" to gracefully close down, just as you said !

13

I think your option 1 fits about 90% of us, and option 2 works for 9%. In my mind the last 1% are those folks who have automation systems at remote vacation homes or other situations where physical access to the site/hub is an issue.

14

I'm watching where that goes also. Seems to be the most elegant solution of all these options.

15

Doesn't have a smart interconnect but the USB-C ups that @dJOS designed works very well. I'm excited for the 2.0 version

1 Like
16

yep, or a challenge to get to in a storm. Your industrial controls experience probably has you quite surprised how far "beyond the plant/facility" a lot of control capabilities have been pushed and now facilitate operations that would have otherwise required somebody putting on their boots and going out to tend to. Agriculture, Livestock, Greenhouses, etc. are a huge consumer of this now.

1 Like
17

To be fair, I didn’t design the original version, I just made them.

The new version is heavily based on that design, but we are aiming for it to be far more tailored to Hubitat.

PS, the design goals are here:

1 Like
18

That sounds like it could be helpful for users that have extended power outages. Please feel to do so. If you make it a wiki post, others can add to or edit it.

Done.

19

I really hope you guys get all the cooperation and support, if not HE internal design accomodation, you might need to succeed in integrating this.

The more one thinks about this stuff in the scope of what all the community-at-large is doing with HA it behoves the-best-in-class solutions to NEVER have a problem induced by a power outage. I'm not saying the box should have built-in "stayin alive capability" for 24hours... I'm just saying...be accomodating and orderly in how you deal with power loss.

I don't think you have to be at a Crestron level to expect this. HA has gone WELL BEYOND "playing with your lights" and despite all the safety/security warnings in making is so...I'd argue it is has become Mission Critical ...even if only for THE HUGE HASSLE it can be when it breaks after being built out and relied upon.

EDIT add:
HE prides itself in being the "Look Ma, no internet and it STILL WORKS" solution, I think the "Look Ma, no power and it RECOVERS" would make a good pairing.

2 Likes
20

At this point I don't think HE has anything to do with it. I think know it is just about the cost to add these last bits of control and features to the current design and will the cost support it.

I think the cost is really the biggest problem when you factor in ACP UPS options out there.

2 Likes