Rule machine 4.0 is too hard to use (Part II)

Ok, now I understand that we're not going to get significant improvements to the UI anytime soon. My second question/suggestion goes deeper. Note that I started writing this simply to get myself more organized around how to write rules, but ended up... someplace else. :slight_smile:

A rule is fundamentally simple:

when (trigger) { operation }

where (and sorry about the formatting, I wrote this on my text editor and couldn't figure out a better way to make a hierarchical list):

# A. A trigger is basically an event whose occurrance will cause the
#    operation to be executed. Events can be organized into four categories:
#     1. Activity from a device
#          i. switch turning on/off, thermostat activity, motion sensor, etc
#          ii. sensor data (like temperature, battery) state change - e.g.
#              crossing a threshold, etc
#     2. Hubitat system events, including scheduled events (periodic or at
#        a specific time), mode changes, etc
#     3. Signals about other rules:
#          i. the trigger conditions for another rule
#          ii. that another rule fired
#          iii. when a rule is paused or resumed
#          iv. another rule's "private boolean"
#     4. Complicated triggers:
#          i. Conditional triggers, representing modifiers on the above
#          ii. Compound triggers, representing a collection of the above
# 
# B. An operation is a collection of one or more actions, which fall into
#    four basic types:
#     1. Act: 
#         i. change a switch, activate a scene
#         ii. delay/pause/wait (state) - wait for something to happen
#                 delay: wait a fixed amount of time
#                 pause: wait until externally interrupted
#                 wait (state) - explicit test per the tests described 2(i) conditional below)
#         iii. restore device state
#         iv. send/speak a notification
#     2. Logic:
#         i. conditional - if (test) then {operation} else {operation}
#             (test can evaluate device state, global variables, or the
#              nameless boolean... or another rule's "private" boolean)
#         ii. loop - repeat {operation} (periodically or by count)
#     3. Capture state: capture/poll device state, set global variables,
#        set the nameless "private" boolean
#     4. Control other rules (run actions, pause/resume, cancel delays, set
#        another rule's "private" boolean)

I struggle with a few things here.

First, providing a single boolean to track state seems arbitrarily limited. People need state. Working around this results in hacks such as people defining a virtual dimmer switch simply to emulate a scalar variable, or using global variables with curated variable names to work around the lack of good variable scoping.

Second, the interaction between rules [sections A(3) and B(4)] seems quite ad hoc. We can steal another rule's trigger, we can run another rule's operation. We can read or set another rule's private boolean - which most definitely means it is not private, at least not in the sense that most programming languages use the term. We can fire, pause, or resume another rule, or use another rule's being fired, paused, or resumed as a trigger ourselves. All in all, it seems like a twisty, intertwingled opportunity to accidentally hurt myself all the way down.

Third, the implementation of delay/pause/wait is tricky and I think needlessly complex. First, the implementation here seems muddled and mixed around with other concerns; we have three primitives that do largely the same thing based on how processing is restarted: "delay" waits for a fixed amount of time; "wait" responds either to an event or sensor data state change; and all can be interrupted by another rule (while "pause" waits only for that explicit interrupt). In the user interface, delay and wait are organized together, but they're somewhat mysteriously mixed in with repeat; meanwhile, pause is mixed in with setting variables and "run rule actions." But all of this suggests an intriguing thought to me: consider that this interaction basically introduces the concept of a trigger in the middle of an operation. Right? Delay is a one-time scheduled event; wait is just sticking a trigger in the middle of the rule; "pause/resume" is a trigger aimed only at that one rule.

Ok, so let's take a step back. I think RM could reorganize things to (1) make the interactions between rules easier to reason about, and (2) unify delay/pause/wait. Furthermore, I think I can propose a way to do that which would simplify the whole interaction model, and I don't think would require changing much of the existing, underlying code (although the operating system appears to not be what I thought it was).

First, I want to suggest the idea of a "named" trigger. I want RM to allow me to define a complicated trigger [per (A(4) above, either conditional or compount] as its own entity with a name. So if I have a collection of a dozen motion sensors, where I care when e.g. the external are active and the internal are not, I can define a trigger that fires appropriately and reuse it multiple times as needed. Then when I add another motion sensor, I only need to add it to one place. A named trigger should be able to use another named trigger, which means we can mix and match creatively - and this of course gives RM a nice place to detect cycles [a triggers b triggers c triggers a], which I don't think it can do today. And I'd want RM to show me all my pre-defined complicated triggers in a nice little list. Obviously this also gives me the ability to give a semantic name to a simple trigger, if I want. Seems like a small but pretty useful change to me.

when (gym_idle) 
    Off: Treadmill

But I want to be able to create named triggers ad hoc, and I want to organize all the wait/delay/pause logic explicitly around them. What I wait for is a trigger, always and only. If I want to delay, I attach the delay to the trigger, not the operation. If I want to repeat, I make the repetition part of the definition of the trigger, not the operation. And if I want to cancel it, I can cancel the trigger, presumably by name. (And if I haven't given it a name, I can still operate on it similar to how it works today - but then I can't reuse it with other rules.)

when (gym motion sensor *changed*)
    if (gym Motion Sensor inactive)
        send gym_idle delayed 0:05:00
    else 
        cancel gym_idle

Ok, that's basically the idea. Why would you care? What does this do?

I. We can get rid of sections A(3) and B(4) entirely, the stuff that distorts the interaction between rules. It's just named triggers; anybody who wants one can send one, and anybody interested can catch it. I don't need to run some other rule's operation, I can send a trigger and then add it to the triggers that rule cares about. Much more tidy, and a lot easier to figure out a couple years later.

II. We can unify all the logic around state and repetition into a simpler implementation around the triggers. B(1)(ii) [wait] and B(2)(ii) [looping] and B(4) [control] unify. Repetition and delay is entirely about a trigger. Waiting is simply about a trigger. Pausing is entirely about a trigger.

So what would the organization look like? It's a lot simpler. I don't think there's anything you can do today that you can't do with this approach, but curious if I'm missing something. (I would really prefer a different programming UI for all of this, but I'm resigned to not getting it.)

# A. Triggers
#     1. A trigger is: 
#         i. Activity from a device (like before);
#         ii. Hubitat system events (like before);
#         iii. A signal (explicitly sent by some process, typically 
#              another rule)
#         iv. Complicated triggers, conditional and compound.
#     2. Triggers can be named.
#     3. Triggers can be scheduled:
#          i. be delayed, recurring, periodic, or at a specific time);
#          ii. scheduled triggers can be paused/resumed, or cancelled.
# 
# B. Operations
#     1. Act: 
#         i. change a switch, activate a scene, or send a (named) signal;
#         ii. wait (for a trigger), which just means stop processing until a 
#             (named) trigger arrives;
#         iii. restore device state;
#         iv. send/speak a notification.
#     2. Logic:
#         i. conditional - if (test) then {operation} else {operation}
#             (test can evaluate device state, global variables, or the
#              nameless boolean... or another rule's "private" boolean)
#     3. Capture state: capture/poll device state, set global variables,
#          set the nameless "private" boolean
# 

That's the environment I want to be coding in.

Thoughts?

1 Like

I am going to sound really dumb here, but what are you trying to do exactly? Are these suggestions for improvements, or are you trying to write an app?

I am not good at all with code, but it looks like this could be done if you want to author an app of your own.

If you are trying to write rules in Rule Machine, what doesn't seem to be working correctly?

Otherwise, I guess I am lost...

Very lengthy post. You’re indeed frustrated.

If you’d like, throw a couple situations on what you want to do, and I’ll show you some rule machine examples.

This might help get your thinking on the right track.

3 Likes

You can create a local variable to track this. You'll just need to make sure you change the state accordingly

These are suggestions for improvements to Rule Machine.

I'm a pretty experienced developer. One of the ways I learn a new coding environment is to write careful notes; writing things down often leads to clarity. In this case, it pointed to what I believe is a flaw in the underlying model.

Private Boolean hails from the days when there were no variables at all and this was one way you could "track" something (actually, I think it was also a restriction that would cause the rule to not run if false...been a while). Virtual dimmers and other devices were workarounds some people used before global variables, and it's something you still can do even though they exist now (including booleans and other types), but...

to work around the lack of good variable scoping.

There are local variables now, too. You do not need to worry about careful naming of global variables for the purpose of avoiding confusion in multiple rules. I'm guessing this addresses the "single boolean to track state seems arbitrarily limited" issue.

There are times when these features are nice to have. I recently suggested one for a use of PB in another thread (and actually, a local variable wouldn't have worked here since those can't be accessed by other rules; but yes, "private" boolean might be a bit of an odd name for something that is, in this sense, public).

In any case, if you don't find a good use for this features, don't use them. :slight_smile: (I have never had a good reason to steal another rule's triggers, and only rarely have I had a reason to do anything like the above. I actually had to verify that some of the rest was possible--it is, indeed.)

I can understand this: triggers for things like "motion changed" or "contact open" create device subscriptions. A "wait for event" or (if not already true) "wait for condition" does the same ad hoc in the middle of the rule actions. These can be nice to simply rules and minimize device subscriptions (not a huge concern but still not something you should do more than really needed to maximize performance). If I recall, the "Wait..." actions are a bit newer. They also do behave differently. Consider the following equivalent ways of writing the same rule:

Trigger: Motion changed

Actions:

IF (Motion active) THEN
  Cancel Delayed Actions
  On: Lights
ELSE
  Delay 0:05:00 (cancelable)
  Off: Lights
END-IF

...or:

Trigger: Motion active

Actions:

Cancel Delayed Actions
On: Lights
Wait for event: Motion inactive
Delay 0:05:00 (cancelable)
Off: Lights

The original (and I think current) Rule 4.0 docs have a lot of examples that look like the former. Staff have lately been suggesting things like the latter to make rules "simpler," though it is not clear to me if that means for the user or for the hub (for the hub, the biggest difference I can see is that the trigger will only happen for one of two events, though mid-rule a second subscription, which I suppose you can think of as a certain ad hoc "trigger" to continue with specific actions, does get created). In both rules you have a "Delay" that creates a scheduled job and causes the rule to wake up again and continue with the remaining actions, as well as a "Cancel Delayed Actions" action that will cancel that delay (which will not get cancelled unless you do this--or go way overboard and "Cancel all timed actions" on this rule from another rule, which is again probably one of those things that would be very rare to need to do) if the rule wakes up in the meantime and happens to be executing a code path where that action would occur. Note that you don't need to explicitly "cancel" the wait in the second rule; any trigger matching will cause any in-progress wait to stop (and no actions after it to run).

The existence of all of these is probably partly a product of how RM has changed over time, but it is worth noting that they all have particular uses that become apparent by seeing more examples and writing more rules.

I might be missing something, but I honestly think you can already do this today. Use global variables, then create one or more rules to manipulate those global variables however you want (could be a complicated set of IF THENs in your actions). As you might know, you can trigger rules based on global variables changing or becoming equal to (or other operands, depending on the type) a specific value. Use that as the trigger for your "real" rules.

That being said, I'm not sure I've seen anything like that before, though I think it should work. I second the suggestions above to provide specific examples of automations you want to achieve, and perhaps a seasoned RM user can provide an example that does what you want (or suggest a native or custom app that does the same; I would not turn to RM for all of my automation needs unless it's clear a native app doesn't do what I want and I don't want to use custom code, which there is indeed an element of caution you should be aware of when using).

There's no denying that RM has a bit of an awkward UI and it takes some getting used to. I resisted for a long time but am quite comfortable with it now. I'm sure staff value the comments from "new eyes" who haven't already become used to these oddities, which it's probably hard for longtime users to even identify anymore. I'm just not sure I'd count on drastic RM changes anytime soon. :slight_smile:

It's a bit unlike "normal" development (apps aren't constantly "running," there is a difference between apps and drivers, and the environment is not super-well documented but not terribly different from the also-not-super-well-documented-but-better-documented SmartThings "classic"/Groovy environment; there's more to know than just Groovy), but perhaps you'd enjoy writing custom apps more than using RM. The classic ST developer docs are good start, though a SmartApp would be "Apps Code" (and just an "app") rather than the IDE in Hubitat land.

3 Likes

If you’re experienced, why not just develop for Hubitat?

I’m starting to learn myself. I don’t write code on a normal bases, but I’ve written plenty of scripts and understand the core of writing.

1 Like

You are correct. I use PB to determine if power is either above X watts or below Y Watts. This is used in a couple rules so the secondary rules do not flip-flop or needlessly trigger. If a rule for my wash machine ran every time power changed, the hub and myself would want to kill ourselves from the false messages!

1 Like

It seems to me you are complaining that your Chevy is not a Corvette. For good or bad the platform is/was evolution not revolution. To say you would like something else is not very useful.

Had you attempted to create some automation back in the X-10 days you would feel what Hubitat offers is amazing.

just my 2¢

2 Likes

Nah. Never let perfect be the enemy of good, but also don't let good enough be the enemy of better. :slight_smile:

Like I said, this started as me just figuring out how RM works. Then I had an idea how it might be improved. Asking whether others agree that it would be an improvement isn't complaining.

To torture your analogy, I'm trying to suggest that this IS a Corvette, but I feel like they put a limiter on the accelerator. I suspect that what I'm describing makes RM almost as easy to use as Simple Automation Rules. Seems worth a discussion at least.

5 Likes

I have to agree with the OP. I am a long time Hubitat user and have used all the iterations of rule machine. It is now very powerful, but the UI is not ideal.

Having relatively recently switched all of my logic control to Node Red on a Pi and comparing it's ease of use to RM4 makes RM4 look very clunky indeed.

I think the Hubitat team know that RM4 UI is sub optimal but they are hamstrung by the architecture they use as I understand it.

1 Like

I agree with OP and obviously RM has morphed over time. I presume some things were left in for backwards compatibility which is completely understandable. However maybe re-thinking this from scratch isn't a terrible idea either. At some point you need to let go of old ideas. I actually miss some of the old days of RM3.0 where some things were infinitely easier to do. Lately I've been forcing myself to write my own apps to eliminate RM rules that were just getting too complicated. However now starting to have all these individual apps isn't very organized either. Then I wrote a Parent app to organize all my smaller apps under to organize things a bit better. But yes... I'm sure I'll get slammed for ... but I still miss webcore... it was so much easier to use and the interface was really great, the rules (pistons) could be shared between users, etc, etc.

Again though...that platform was running on hosted servers with little concern for overhead vs our now local platform. So everything is relative.

I think it's good people giving ideas for how to improve the platform.

3 Likes

How do you define a local variable? I've looked through the documentation and must be missing it.

You should have this option at the bottom of every rule:

This screenshot is from the latest release of Rule 4.0. If you happen to be on really old firmware for some reason, the feature was added last summer with hub firmware 2.1.3.

2 Likes

Thanks for the ideas, @dondo.

This all reminds me of the genesis of CoRE:
Phase 1: Adrian - "I've got a cool idea to replace RM"
Phase 2: Community - "You're in way over your head, boy"
Phase 3: Adrian - "Gonna do it anyway, even if just for myself"
Phase 4: Community - "Adrian, you are a god !"

3 Likes

If people think webcore is so great, they can do that today in Hubitat...

So that seems like an odd comparison, as if that is what "good looks like" to you, it is already an option to use. Right now. :man_shrugging:

No need to make rule machine into webcore.... Just use webcore.

5 Likes

Sorry for the long delay; busy week. I've been playing around with pause/wait/delay. I've discovered that the eventing model is not at all what I expected.

Inspired by the thread @bertabcd1234 referred to above, I tried to come up with a concurrent approach for hysteresis (that is, "run once and then hold off for a while.") In this case, the intent is to run once in a particular time period. I'm using a pretty short period (couple minutes or less) for testing.

Here's the baseline rule, using time variables. This works, but it's ugly. It's also going to be vulnerable to the midnight boundary (across which a time a few minutes in the future becomes 24 hours in the past) and run twice, which isn't ideal. [ed: this is why most programming languages provide timestamps, which aren't vulnerable to those oddities.]

I've tested it and sure enough, it basically works. But it's icky, so I decided to try to use the RM pause/wait/delay primitives directly.


This experiment was pretty informative, showing that things don't work at all as I expected.

image

Here I'm setting a boolean with a delay. My expectation was that everything would stop until that delay completed (since the rule is sitting on its hands, waiting for the timer to fire). But that's not what happened at all. The logs show a very interesting interaction (remember, read from bottom to top):

07:46:55.447 pm info Delay Over:     Set ready to true --> delayed: 0:00:45
    [ten seconds later!]
07:46:45.859 pm info Action: END-IF
07:46:45.856 pm info Action:     [stuff skipped here]
07:46:45.841 pm info Action: IF (Variable ready(false) = true(F) [FALSE]) THEN (skipping)
07:46:45.734 pm info ActOnce Triggered
07:46:45.522 pm info Office Motion Sensor is active

07:46:34.506 pm info Office Motion Sensor is inactive
07:46:11.063 pm info Deskside lights is on [digital]
07:46:10.274 pm info Action: END-IF
07:46:10.224 pm info Action:     Set ready to true --> delayed: 0:00:45
07:46:10.184 pm info Action:     On: Deskside lights
07:46:10.153 pm info Action:     Set ready to false
07:46:10.147 pm info Action: IF (Variable ready(true) = true(T) [TRUE]) THEN
07:46:09.950 pm info ActOnce Triggered
07:46:09.702 pm info Office Motion Sensor is active

So that's pretty fascinating. Each event gets its own distinct instance of the rule. So all of the wait/delay/pause primitives will operate on a unique chunk of running code - there's no underlying thread with which we can interact! So this does works, but not at all in the way I expected. I thought I could just wait; instead, I have to set the guard, and then remove it after the delay.

Still, a useful approach, so I thought I'd share.

Last time I checked that wasn't officially supported...and obviously a pretty big resource hog (yep, there is a tradeoff). Plus separate servers and stuff.

Honestly though...if you really want to start writing rules that are getting complicated...just start writing groovy...in some ways...it's actually easier/faster than RM4.0. It's also editable so there aren't all these clicks and drops downs and stuff.

leave RM4.0 for the rules that fit it. Plus once you start writing your own apps, it's copy paste quick to take code from a previous project. It's not that hard. In fact...A thread or spot with some good basic apps to get people started would't be a half bad idea. The staff at HE show examples all the time...like this one:

I used to start an MQTT driver for my garage door...sometimes it just takes a little help to get started.

1 Like

I've only followed some of what is being explained. It's 8am I've been up since 5 and only had one coffee. However it seems to make sense and I think it may bridge the gap for users going from the built simple automation and motion apps to rule Machine, which I feel isn't quite there for the average user.

There is merit on opening a dialogue with the powers that be. Maybe someone with more clout can tag them in, or point them into this post.