Required Expression false .. but it isn't

i do use it.. but mostly for things like time restrictions etc.. but as i have lots of rules written and not making lots of changes i leave those in place that work instead of rewritting and debugging using a new approach.. i still have more rule machine 4.x than 5.1. Just regarding the case where a required expression is not reevaluated at boot time was stating that if the expression was an if in the rule it bypeass that issue thats all.

any issue at boot is thorny at best.. i've had the same issue with std rules ie if a door opens or something comes on (and they stay that way) during the boot then some rules are and have bound to fail as the state change on given devices will be missed.. not really a way to avoid it becuase you cannot possibly go back through all devices.. at least critical devices like garage doors etc seem to internally poll periodically to avoid this issue.

1 Like

Hmmm .. I believe I was using the feature as intended and I had the moment of astonishment when it didn't work as expected. Knowing now more about under the covers, I plan to favor conditionals over required expression if the rule isn't that much more complicated, unless the frequency of state change of some condition was very infrequent.

For example, if I wanted a creaky sound made whenever motion was detected on the front porch but only on Halloween, I'd make Halloween a required expression so that the motion trigger wouldn't needlessly run a rule with a conditional "Is it Halloween?" when 364 days of the year that trigger isn't useful.

Back to furthering our understanding of behind the scenes, is it correct to say that upon creating a new rule with a required expression, or even hitting Done on such a rule, that the required expression state is evaluated at that time, and no event needs to occur to set it to true if the condition was true when the rule was created/hit Done. But state is otherwise not evaluated unless an event occurs.

3 Likes

Just thinking aloud….You could easily test that yourself by creating a rule without a required expression, running it, then edit and add a required expression for between two times that doesn’t include the current time. When you hit ‘done’, if it has evaluated the required expression, the rule would stop triggering.

If it did work that way, you could create a rule with actions to initialise your rules (where a required expression may be out of whack), triggered by system start.

Edit: Required Expression is evaluated at rule creation/initialisation, I just tried the above. On reflection, it's kind of obvious that it does I suppose, as otherwise when we created rules after sunset that have a 'between sunset and sunrise' required expression, we'd have to wait until the following day to see them start working :roll_eyes:

1 Like

Yes, I have also tested in order to reach an understanding of how it works, I believe I do understand now (again, please correct me if I'm wrong), and if I were asked to explain to others I would say ..

The Required Expression (like a condition) is a state. The state is set in one of two ways:

  1. An event occurs that potentially changes the state (e.g., a door opens would change the state of a door is open).

  2. When you hit Done to initially install a rule or after editing the rule. No event is necessary in this case.

But the state is not evaluated upon system restart, so if an event occurs while the system is down, the state won't change until the next event occurs that could change the state.

3 Likes

You could make this happen with a single rule and a Hub Variable. Create a number Hub Variable, set to any positive value. Add a Condition to your Required Expression with AND for that variable > 0. Add that Condition to the Required Expression for any rule that you want to update the Required Expression state upon system startup.

Create a new rule whose Trigger is Location event: systemStart. Its actions should be to add 1 to the number variable.

What will happen is that upon system start, that variable will change, and the Required Expression will be evaluated. The new part of the Required Expression will always be true, but the value changing will cause an evaluation. So in case something was missed while the system was not running, it will be caught up.

5 Likes

That's quite a clever workaround.

When you have a minute Bruce, you could add that lot to your RM code, with an additional checkbox in the Required Expression area titled "Re-evaluate expression at system start" :wink:

...Just kidding

2 Likes

That is a brilliant solution! I scratched my head for a minute trying to figure out why you wouldn't need to set the variable back to 0. Now I realize that the actual value of the variable is moot; it is the change of value that would cause the evaluation of the restriction.

OK, I think we've covered this topic sufficiently. Thanks for that. I would be remiss to not suggest that perhaps one day restrictions be evaluated when a rule is instantiated after system start (whatever the proper term for that would be). You've already seen that such is useful when a rule is created/edited, and making the required expression even more foolproof would be desirable. We've apparently both had moments when seeing "Required expression false" and its evaluation as TRUE was odd.

2 Likes

There is no such thing as "rule is instantiated after system start". It's already instantiated. Reboot has nothing to do with its state, that is, its state is its state as existed prior to the system going down. There is no hub mechanism to go look at specific apps upon startup (there is such a mechanism for drivers). For the most part apps don't need anything to happen at startup. However, as you've discovered, some state change that happened while the system was down will not be automatically discovered. This is a conscious and well thought out feature of the architecture, and I won't bore you with the reasoning behind it. Suffice it to say that in general trying to evaluate things that might have happened before the hub booted is a losing proposition. Any number of things might be 'wrong' with your system when it starts up. But its intelligence should be applied to dealing with what happens when its running, not dealing with an unknown number of possible issues that happened when it wasn't running.

Should RM have a feature that automatically subscribes to systemStart and automatically evaulates Required Expressions on startup? That's impossible to answer in the general case -- it may or may not be the right thing to do, and there is no magic way to know which answer is right. But, you have the tools to do it if you need it.

1 Like

It would probably cause as many issues as it solves. As Bruce previously mentioned what if a door was opened while the hub was up (but then closed while the hub was down) If it re evaluated the required expression from the state at system start it would evaluate incorrectly as the state would be incorrect temporarily. It would always work for time based required expressions though.

1 Like

The events that happened while the hub was down are irrelevant. It isn't necessary to figure out anything that happened in the past before system start. But required expression is a statement of state, and that state should be as accurate as possible. It is what it is. If there is a restriction that a rule should trigger only when a door is open, then a condition where that rule isn't triggering even though the door is in fact open .. that should be something that should be addressed if reasonable to do so. Unless I'm missing something, evaluating a required expression should be a relatively efficient process.

Evaluating a required expression won't cause a rule's actions to run without its trigger. Having the required expression state accurate AND the rule gets triggered .. then the rule will run. Yes, 100% of the time that is the right thing to do because that's the way the rule is written - if required expression is true and trigger occurs, do actions. How could that not be the desired behaviour?

2 Likes

For one, 50 or 100 rules all running at startup, when only 1 or 2 of them need to.

Example?

That you open and closed the door a million times while the hub was down is irrelevant. The truth of the required expression cannot be made incorrect by evaluating it. If the door is closed at system start then a required expression of door is open would be false. It would only be incorrect by assuming that the door was in the same state as it was when the hub went down.

2 Likes

I will look into the possibility of adding an option for a Required Expression to be evaluated at systemStart.

4 Likes

Yeah I just attempted to describe an example and failed, you're right. Let's just put my comment down to a brain fart :laughing:

On the plus side my suggestion further up (though a joke) is now receiving consideration....

2 Likes

When I read it, I thought that seems like a pretty good solution. It basically does what Bruce suggested, just simpler. As @HAL9000 mentioned it's not a trigger it's just an expression true or false, what happened while the hub was down is irrelevant, just is it true or false right now. at startup.

Also it should not be the default, but should be an item you must set for each rule if you want it.

8 Likes

sweet very good addition, thanks

1 Like

First, I must say I'm very happy with the solution. That's outstanding.

Curiosity has me wondering ... is your concern about this being the default (or perhaps only) behaviour one of performance (i.e., concern about system load during startup)? If so, I can't speak to that. I leave that to Bruce.

But in terms of logic .. are you thinking of an instance where required expression not correctly representing the current state would be desirable?

1 Like

Well my concern would be unforeseen consequences. I can't really think of reason where it would be a problem, but it's always better to err on the side of caution.

1 Like

I have been having a similar issue to the OP very regularly lately after 10+ months of my rules functioning without an error at all. Multiple rules of mine using sunset are regularly failing (close to every single day) even though I have not had any power outages at all or known reasons for my hub to have shut down. I am completely stumped by the issue since it just appeared in the last few weeks. This new feature sounds like it MIGHT help for me but I can not see it when editing my rules, has it not gone live yet? My hub has no updates available at the moment. It is on 2.3.3.140

Another one of my rules has started failing today yet I can't tell if this addition would have fixed it as it is time dependant, though it uses hard times. Some of the chat in this thread has been a little over my head so perhaps I am misunderstanding.

Do you see any reason that new feature would fix the following rule?

My biggest point of confusion comes from the fact I switched to Hubitat from ST last December and have next to NO issues at all until the last few weeks. Suddenly I am having issues daily even though I haven't added or changed any rules in many many months. My system has been setup and running great for so long until it just decided to regularly have rules not functioning.

1 Like