You raise some interesting thoughts there, thankyou , I'll digest.
My reasoning behind (only) supporting the inbuilt HE drivers for adhoc input to HE is that it helps define and restrict the devices I support and avoids me getting asked for xyz support. I really don't have the time or to be frank interest in authoring and then supporting a load of drivers for peoples individual needs. My code is available for people to either create pull requests for additions or edit for their own individual needs and use (only)
For outgoing devices HE -> MQTT I support effectively every driver.
Great input .. thanks for taking the time to understand the issues
This is a tough problem. We had a similar issue with the Lutron integration, because sending a command to Lutron generates what looks like an external event from Lutron -- indistinguishable from one. We do use state to resolve this, to avoid looping.
Here's a fairly easy example .
I have a HE virtual dimmer currently off at level 0 linked to an MQTT device
The remote device updates it's status via MQTT to off at 50%
I know I need to send a level change command to the HE virtual device but want the off status to remain.
I can't do this without thought because I remember the level change turns the light on in HE
This level change then generates a switchedDim 'on' event and a level change event
I didn't expect the on event of course but anyway I can turn it off again. Some devices have a presetDimLevel command I believe.
Conversely setting the level to 0 does not turn the light off and a light can be turned off an retain its dim level. (which I think is the right approach)
But in actioning this I have received both an on and an off event and a level change, and importantly I didn't send an 'on' command so I didn't expect it eventing . This actually happens even if the light is already on, it again sends another on event.
So.. I'm messed up by attribute interactions getting back events I didn't expect and really not having a way to know if these should be sent back to the remote device. Automatically knowing what I might expect back (by keeping a note of the commands I've sent) isn't easy. I never sent that 'on' command. Also action order of these commands for example off/55% or 55%/off is important the off must go last or the device is left on. Knowing which attributes when updated can internally effect other attribute values is key to knowing in which order to apply commands ... hhhmmm
It's amazing even a simple dimmable bulb can be so awkward !! I dread implementing an arbitrary MQTT thermostat.
@bravenel I assume when a device generates several near simultaneous events the various event handlers can run concurrently and are not deterministic in sequence e.g. switch before level ? I think I read somewhere events can get re-ordered before delivery.
For anyone interested I've implemented a reasonable solution to this It partly uses tagging that @tcp77 suggested, state or atomicState and a very brief 100mS enforced 'deaf' period to events, in my app only and for the one specific device.
When an MQTT status message comes in I tag the device involved and first check if the current attribute value matches - if it does I don't need to update so I drop that command thus avoiding an event (although it might not get sent anyway)
If I do need to update a device then I get right to the point of issuing the command and set atomicState.block = device.displayName , then issue the command and kick off a runInMillis(100,"unblock") call. This will remove the block in 100mS marking atomicState.block='FREE'. Each command I have to send (there could be several in a json payload) I update the atomicState.block, and send another runInMillis(100,"unblock") which resets/prolongs the unblock timer.
Now in my event handlers I immediately get the events device.displayName and compare it against atomicState.block and if they match I ignore (drop) the event.
This has the effect that I am deaf to all events from that specific device for 100mS after the last command I sent to it for MQTT originated 'status' update messages. Other devices subscribing to the device will receive events as normal and any other commands I send to the device are actioned without blocking.
The only potential issue is that if another app e.g. Rule Machine sends a command to that device during that 100mS I will not see the event. I know how to avoid that situation by keeping a list of expected events and their reported value - and then RM's command would be 'unexpected' but that adds significant overhead. Anyone creating a RM rule that triggers off the device change and updates back to the same device just needs to delay the response by say 250mS and it will all work fine. (I hope)
P.S. 100mS was a best observation based on the time for an event to fire following the command being sent. On my system I could get away with 50mS but on a more sluggish system it might need more. The good thing is that on a sluggish system the unblock command gets delayed too so it sort of self adjusts .
Seems you've got a reasonable workaround, but thought I'd throw this out anyway.
Does device.updateDataValue() work from an App?
It was mentioned here - Device routine returns null - that you can manipulate a device Data object (update / get) from an App. Not sure if that's just getDataValue() though or whether you can write to it ....
If you can then couldn't you set something in there each time the device is updated from your App to show what events you want to ignore afterwards. Then ignore any subsequent events until it's cleared?
MQTT Status Change ON, 50% ->
virtualdevice.updateDataValue("MQTTEvents", "ON, 50%")
virtualdevice.on()
virtualdevice.setLevel(50)
Then as the events come back from the Virtual Device compare them to what you've stored and remove them until there's none left. When there's none left, or a non-matching event comes through, you know that this was from somewhere else and you should send a CMD out.
(I've not played with HE Groovy for a while so this may be b0ll0x in which case just ignore it)
What you suggest is actually what I was meaning with 'significant overhead'. I do use data values within devices for storing the MQTT topic values and mappings but it is quite slow.
What also deflected me from this approach is that some unexpected commands come back - for example setLevel() returns a switchedDim 'on' event. Now I know that I can expect it, but there are other devices where this is going to happen and I will get unexpected events from those. What I like though is that there are no artificial delays imposed in this approach.
There is also the question of intelligence in a 'virtual device' Should a virtual device just effectively be a display / attribute store template or should it act like the real device ? After all in the MQTT situation the real device and it's intelligence is often remote.
Take for example a blind (shutter) or a garage door. In the virtual devices a 'close' command starts the mechanism working but the device does not return the closed event for some time - maybe 10 seconds (often settable). Being able to set this delay to 0 would be ideal of course.
Your approach actually caters for this and mine doesn't because my 100mS expires long before that event arrives. For me I shall probably handle such 'delayed response' devices specifically or hope that I can set the closed state directly without using the 'close' command. After all the remote device will report 'closed' itself eventually in a status update.
PS I did initially try an approach as you suggested, initially with a Map object and later device data and it worked fairly well (except for unexpected commands) but it did add about 800mS to the processing as several write/reads have to be made to the devices data value. It was quicker with the map object but more awkward to code.
Also with concurrency it's hard to protect the integrity of that data value without passing it via atomicState - hence the map object again helped. I haven't totally discounted that as in practicality the device has already been updated so it's not impacting speed there as such - more in the the tidying up afterwards.
What would be better would be for Virtual Devices to perhaps have a method where you could update their attributes without triggering an event. They are a bit of an oddball I guess because as you say often the action is remote so the Ack that a "normal" device would get (to confirm its state has changed) never arrives.
You really need:
setLevel()
--- generates an even that you can react to
setAttribute("state", "on")
setAttribute("level", "50")
--- doesn't generate events but the device is now updated to show the current values
Not sure how much you'd need to petition Mike / Bruce to get that though!
All OK here thanks, mostly just lurking on forums nowadays unless something interesting catches my attention I rarely speak
Even that is problematic as other apps subscribed to the device do need the events or they wont realise it's updated - it's just me that doesn't want them !
I think both these approaches are viable - i hate using 'delays' as a solution in code so I prefer the other approach .. I'll see how this works out. I'm still concerned over concurrency if two devices are involved simultaneously as this solution won't handle that. But I believe the parse{} method in HE's MQTT driver is single threaded and I use parent method calls now (not events) so that might avoid this happening.
@bravenel Having something returned in events that identified 'what/who' originated the state change would be so useful though... This could even help identify which user turned the device on/off which was a request in another topic for a history log. Sorry to tag you Bruce but this may be missed otherwise. Maybe it's just too awkward as it works internally currently.
or identify RM interactions specifically...
or log a specific users name
.. an alternative being pass a token with a command that gets included in any associated events but that probably requires all drivers updating and supporting it - non starter
Most events are generated by devices, not apps. The current architecture of the hub does not support this concept.
I don't follow what RM has to do with this. Aren't you concerned about the response from the external device from a command sent to it, distinguishing that response from one generated externally? Isn't this something that should be handled in the driver that sends the command and processes the response?
The initial issue was filtering out 'caused by me' expected events that came back from a device I had just sent a command to. (By this I actually mean I am updating the HE virtual device status to match a new status reported by the remote MQTT device, not sending a command to the remote device). They are really state updates to the HE virtual device, implemented via the commands.
The RM reference just refers to me within in my app being able to see that RM had sent the command to a device that resulted in this event.
If I change a few attributes (via commands) on a device then if RM triggers off these and sends other commands back to the device immediately then the events get interleaved in my app and I have difficulty determining what the final state should be.
Assuming this is your driver in use, simply create a separate command for RM to use (with Custom Action).
If state is too slow for your use, what about using @Field? I'm assuming you need to set a flag in the driver when you send a command so that you know the next response is a result of that command.
By that you mean the next events I receive ?
I am sending commands to the HE virtual device (from my app not driver) to update its state eg 'switch' ' on' . I want to ignore the events that come back caused by my command , but to receive any that come form elsewhere e.g. RM
Yes essentially but it could be that RM or any other app that subscribes to one of these virtual devices also tries to control this same device at the same time. The events get interleaved. I don't know how to recognise that situation. These devices are all using Hubitats included virtual drivers each as a representation of the remote MQTT devices.
I don't think you can solve this problem using built-in drivers. The only way to attack it is in the driver for the device that represents the remote MQTT device. It's the only place you can distinguish and stand a chance of knowing which response is the one to ignore.
OK - The only driver in my app is the one single child MQTT driver - I don't have 'drivers' for any remote MQTT devices - I wanted to use your standard virtual ones !
So it's just not feasible for an event originating from a (any) device to return an extra property saying which appID triggered this event ? (a feature request)
- non starter
