I’ve recently become aware, through controlled tests, that delayed actions and “stays-for” triggers actually survive reboots. Missed delays are executed early in the reboot cycle (~40 seconds before the startup event) and actions set to trigger later happen on time. This can be good but there are scenarios where it is not. For example, if I have a flag that says “nobody home” (a flag that might control the behavior of not only lights but also locks) which is set to true when a trigger detects motion=inactive and stays-for 45 minutes, then, if we lose power for 60 minutes shortly after an inactive event, immediately upon restart, the flag gets set to true even though there may have been plenty of motion in the room during that hour. I can think of fairly complicated solutions but don’t know of any “clean” ones.
Has anyone encountered this and is there a fairly clean solution that I’m not aware of?
Particularly because the timers fire before the restart event (I can’t just cancel rule timers on restart), the best I can think of is to specifically monitor the events that I’m worried about, set a timer when they occur and then look for a system startup event within the next two minutes that manually cleans things up. Crucially, to be resilient to restarts, the “look for a system startup event within the next two minutes” would need to use variables with timestamps or that wait itself could be fooled by a restart. That seems fairly complicated for an edge case that I haven’t even encountered,yet also one that could conceivably occur quite easily.
But there were no active events during the time the hub was down. Device events can't happen without the hub to send them, so those events never actually happened. The sensor seeing motion is not an event, it is only an event when the driver receives the change in state from the device, and then an event is sent by the driver to the hub for that reported state.
There is no way for the hub to know that the device was sending messages while it was down, and there is no event history to check for when the hub was down. When the hub comes back up, it will still have the device state that it had when the hub went down, and that state needs to change from a message from the device, for the driver to send an event to the hub about it to trigger an automation. Only a driver that polls state will change the driver state based on the current device state, when the polling schedule triggers it to check, but most drivers only react to incoming messages and do not poll the device.
Even with a rule that triggers at reboot that checks the current state of those devices, that does not tell you how long the device was in that state for, or if it is even really the current state of the device. The device driver state is going to be in the state it was in before reboot, until it gets a new message from the device. The device might have reported active many times while the hub was down, and even be in an active state when the hub comes back up, but the driver won't pick that up at reboot if it was in an inactive state when the hub went down, until a new active message comes in from the device to the driver.
So, not much you can do but make any workarounds you think will get things back into the correct state, but you may not have enough info to even do that until the driver gets a new message from the device to update to the actual current state of the device. If the device has a refresh command, that may poll current state of the device and update the driver, which would send an event. So any workaround should probably first send a refresh command to the device before checking current state in the driver, if the driver has that option.
The often recommended "solution" is to put your hub on a UPS that power it during several hours of power loss. This permits events to be received from those battery-powered sensors that are still generating them.
I would recommend the use of a UPS for this, and more importantly, to prevent hub database corruption due to sudden power loss.
My writeup was a little long-winded, so I fear not very clear. I’m not looking for a way to catch the missed events. The problem I was pointing out is that the hub design will treat a long power outage as a “stays-for”. You are right that there is no way for me to know at that point whether or not my motion sensor was inactive the whole time but it would be good if, for a particular rule, I could set a “cancel stays-for on restart” option if that is what I wanted. Otherwise, yes, a UPS that eliminates power outages or my after-the-fact detection and correction of a restart-induced stays-for trigger sound like the only options.
I think the issue is the Quartz Cron Scheduler that is used by the hub. So Hubitat is working with the design parameters of the scheduler itself that the Hub uses.
AI puts it this way:
Why it Happens
The Misfire Threshold: Quartz defines a "misfire" as any trigger that fires later than its scheduled time by more than a certain threshold (default is usually 60 seconds). [1, 2]
The "Smart Policy": By default, Cron triggers use a "Smart Policy" misfire instruction. For a Cron trigger, the Smart Policy resolves to MISFIRE_INSTRUCTION_FIRE_ONCE_NOW. [1, 2]
Reboot Recovery: Upon booting up, Quartz checks its job store for misfired jobs. Because the trigger is set to FIRE_ONCE_NOW, it immediately executes one catch-up run for the missed time before returning to its normal chronological schedule. [1, 2, 3]
How to Prevent It
If you do not want jobs to run when the machine has been offline, you need to explicitly override the default policy. You can configure your Cron schedule with withMisfireHandlingInstructionDoNothing(). This instructs Quartz to skip the past missed executions entirely and simply wait for the next chronological occurrence in the schedule
So the only other option is for Hubitat to change default parameters for the schedule to do nothing, instead of "catching up". I guess that could be a user controlled parameter, but doing nothing would not help your case either, though.
I'm not sure they can easily fix how the schedular works for cases like this, they chose a free, open-source scheduler to base everything on, and they are kind-of stuck now with how it behaves.
you are right that doing nothing is not a perfect solution either. I agree that the current behavior can be helpful and desirable in many cases (especially for brief brownouts and timers that don’t even need to “catchup” but simply keep on working as if the power outage never happened. I had been under the mistaken assumption that upon startup, all timers and rules that were in mid flight (stopped at a wait) would be killed on a power loss and that I needed to decide what I wanted the startup policies to be when I get triggered by a system startup event. Now it seems that I need to explicitly detect any misleading “stays-for” triggers and decide what policy I want with the unfortunate limitations that a). it’s a little tricky to do and b). before acting upon any stays-for trigger that I deem critical, I might have to wait an extra minute to make sure that no system startup occurs.
I have been using hubitat since before 2020 (maybe 2018? not 100% sure) and have never had that situation pop up. I have survived numerous power outages, many being more than 4 hours. Using the UPS as mentioned has allowed me to do controlled shutdowns.
One thing I would say is that your given example is one that I would never have tried in the first place given how motion sensors work. I would never use them as my only means of presence detection. Trying to think of any other situation where delays and stays in rules or apps would be the same kind of issue. The closest I have is my laundry monitor, my motion lighting rules, and possibly my rule that sets a hold on my thermostat when outside temperatures exceed certain extremes.
In the laundry monitor situation, as well as the motion lighting rules I have, power outages sort of self fix since the power goes out to the washer and the controlled lights before my hub loses power. So, they usually recover to a powered off state due to my rules.
One thing you could do, somewhat cleanly, is to have a power outage shutdown rule that resets the rules, or in your example case, starts up with a cancelation of the affected rule(s) and adds a wait of a few minutes to give any motion sensors a chance to detect motion.
A rule like that might use a virtual switch that that is turned on when you shutdown the hub due to a power outage (you could do that in a shutdown rule). Using the hub startup event with a conditional on that switch could then set variables, or do whatever you want it to do to handle special startup situations, and then turn the switch off so it doesn't run on regular reboots. (just thinking out loud here)
If you are using rules, there is also the option to evaluate required expressions on startup. I use this for several rules - specifically the thermostat one I mentioned. This effectively resets this rule (note: likely specific to my use case). However, that virtual (power down) switch being off could be a required expression for the rule to even run. If you also have the rule set to cancel all pending actions, then if the hub is shut down with the switch on, then it will disable the rule - resetting pending actions - until the switch is turned back on. So, if you just had that startup rule I mentioned set to turn the switch back off after a delay on startup, then your rules reset and restart with the toggling of the switch.
for stays the best option would be an actual option on the stay whether you want it to fire after a reboot if time was expired or not.. much more complicated for general rule schedules as there can be multiple schedules in a rule
I didn’t think of using Required Expressions as a means of canceling stays-for. That definitely works and is a great idea. But, unless I’m missing something, it only sounds useful when combined with a UPS, which, given the outpouring of recommendations, I’m now considering. To be able to shutdown the hub and then automatically restart it, it sounds like you must have a UPS that allows you to control the power to the hub itself? What UPS do you use? I’m worried that my setup is not big enough to justify the cost of a UPS that is full-featured enough to accomplish the pattern you describe.
No, I can only shut down remotely (I use the hub info driver available on HPM). For powering back on, it waits until I am there to turn it back on. You could get a wifi switch that has controls external to HE to cycle power once you have power and internet though.
I just use a normal UPS that powers my modem, router, NAS, hubitat, and one mesh node for wifi. When we lose power, I shut down my NAS if it lasts more than 5 minutes. The UPS also protects against the many instaneous outages and brownouts we have out here
