Need MQTT for Hubitat Elevation


#21

Ahh - you missed my point. I was meaning companies like Shelly and Sonoff are creating very low cost plug and play peripheral devices that can easily be integrated into controllers supporting MQTT. I was meaning the advent of novice users buying such devices and hoping, even expecting that Hubitat could easily support them. Not needing a device driver for every specific manufacturer.


#22

I agree, I don't think that's really necessary or advisable. Keeping HE as a top performer is important and all MQTT users ATM have their own broker. Just basic client functionality would be great, even if it just supported some basic device types like onoff, dimlevel, text perhaps. I think Rule Machine might be able to assist here if people needed more.


#23

Ummm I’m new to all of this & already on my 2nd HE after the power port feel out with the power cord provided. I WOULD gladly try out the MQTT client but that won’t connect to my logged in Mosquitto server. I am sure you get a thousand requests for stuff & didn’t mean to start all of this about a post. I’ll consider it a mute option.


#24

It can be made to work - as I use it currently but it is just a kludge via http, and the sad part is that the kludge about because of the deficiencies in ST lcal network capabilities.

Integrating MQTT to HE via HubConnect is another possibility to investigate, but it would be an external rather than integrated solution


#25

TL; DR version

I'd like the MQTT Client API exposed to Driver/App authors in the short term - without having an(other) intermediary party (NodeJS, Python et-al) to deploy, manage and run.

Short term, I don't need HE to host a MQTT Broker, Mosquitto on my 3yr old rPi trucks along fine with a significant event volume.

Background

I came from a hybrid (federated) world of openHAB (5 yrs), and Vera (10 yrs). I federated these to get the benefits of both worlds (richer/active Rule deveopment in openHAB, stronger Z-Wave support in Vera), and to ease the transition/conversion process.

I also got access to my Alarm Panel which, until recently, was also served by my Vera.

No single / complete HA solution - open enough to let us glue it together

One solution couldn't "do it all", so I brought in other bits to handle the broader parts of a whole home automation, monitoring and control system.

eg. Flexible dashboards, iPhone control devices, Long term data archival, Diagnostics, multi-modal Notifications etc.

Data collection

I collect a lot of data about my house, mostly for off-line understanding and planning, but also for curiosity about the overall behavioural patterns for the system components, and for the house itself - ultimately to work out optimizations, maintenance, etc.

For these, I've used different databases over time, most recently InfluxDB (Event data) and rsyslogd (Log data)

Separation of Producers / Consumers for the data

With Influx, I originally directly integrated, and in more recent years I did it over MQTT - allowing me to swap out Databases without impact on the upstream/producer systems. It also meant I wasn't querying Influx (history) to get current state.

eg Data publication:

  • btmon (for Brultech Energy Monitors) pumps my energy production/consumption into MQTT, and Telegraf configured to push the MQTT events into Influx.
  • openHAB publishing change-events to MQTT, and similarly into Influx
  • openHAB subscribing to energy events, and alerting when they go above limits/thresholds

All my data for the last ~4yrs is in InfluxDB, as events, and rsyslogd as raw logs. Every light switch, every temperature change (2x Nest + Outside weather), all the energy usage (5s samples for 30+ household breakers)

... except for the stuff I moved to HE, as I'll have to custom author something to push the data there correctly.

Separately, I swap out household components on the regular. The Hubs isolate my automations from the Device changes (eg. Wemo -> ZWave) and the MQTT helps isolate my data collection & dissemination from my Hub(s)

HE has been great at simplifying the common Automation tasks from openHAB/Vera, but I've lost a LOT of visibility to the data I used to collect... not because it can't be done, but because it takes a long time to (re)write functionality that's available, but HE doesn't enable.

Use Case examples

  • Want to know if the washing machine is on, I can get that back from MQTT.
  • Want to know the Espresso machine is heated and ready to go :slight_smile: ...
  • Want to get alerted when the outside lights having a wiring fault, I can get the data from MQTT

In each case, I don't care where the data is (ultimately) coming from, just that it's centrally available, and historically archived.

Could I do these with other things, sure. Would I have to change those each time I changeover a physical Device, a Home-Automation Controller (etc)... probably.

Don't get me wrong, I went into HE with my eyes open about it's limits, and I'm ok to wait... I like it's more declarative nature for the simple stuff.

Examples

  • Owntracks (linked above)
    I stopped using this just prior to the transition, so I'd get used to it not having it.

  • openHAB MQTT Driver (linked above) - implementing both Homie and HASS's MQTT form.

    I was going to redo my Paradox Security Alarm Panel driver in the Homie convention (Linked above), and then it could be integrated once for all home-automation platforms.

    I've recently re-written my Vera Alarm system plugin (Lua) for HE (Groovy) but I'd rather have written it to MQTT / Homie (or HomeAssistant's equiv) to avoid doing the same thing repeatedly, and to chose the language I'd prefer to work in.

  • Threshold alerting for Energy use/anomalies
    This one is fun, since I recently had an electrical fault in one of my circuits.... something that an outlet-based monitor would never have picked up.

  • My transition to Hubitat
    If the MQTT Client API parts were available, I would have bridged/federated openHAB (using Homie convention) to get up and running (much as I did with Vera -> openHAB originally). It would have been a lot less disruptive, and you see folks here federating their ST and HE devices (as well as multiple hubs)

    Luckily the Mrs was away for 4 days when I did the primary conversion, and the house was dark :wink:

Expectations:

Low level (API for Driver/App authors) ...

  • Connect to one-or more MQTT Servers at the same time.
    One per top-level Device would be fine, using TCP (most common) hostname and port, preferably with TLS capability for remote service security.

    In my use-case, one is my "local" MQTT Server, the real Hub of my system, and the others would be for connecting to services like owntracks. Initially, I'd setup my MQTT Server to daisy chain the Topics to the owntracks stuff, which is how I do it now, but I'd prefer clients to connect (declaratively)

  • (Bulk) Topic Publication to the Device's (configured) MQTT Server.
    For my use-case, I'd write an App and "register" the HE Devices I want to watch, sending their change events to my MQTT server for archival (etc).

    The form of this is where it starts to get interesting, since there are many data formats, Topic-naming conventions (etc).

    This is why I'd prefer to see the integration available in API form initially, as it's likely the initial implementations would need to perform transformations on the data before sending it off (eg lights can be "on"/"off", 1/0, true/false, 100/0 etc). The HE App, in concert with it's Top-level Device, will need to acquire the data and perform these transformations before delivery.

    API-wise, this is the functional equivalent of publish.multiple in Paho

  • (Bulk) Topic Subscription to topics on an MQTT Server.
    I think of this as being akin to a "MQTT" Capability, where there's a set of methods I could call upon to setup the subscriptions, and then everything would be delivered via MQTT-Capability specific methods (akin to parse for Protocol.TELNET, but delivering an object representation instead of plain strings)

  • Receive events to a HE Handler (like parse, kinda, but more structured)

  • LWT Support - letting everyone know you've died, they've died, we've all died.
    MQTT has a concept of a Last-Will-and-Testiment that "both ends" can usually sign up for. Each entity with data (typically) uses the LWT to signal they're alive. Homie convention uses this, and we'd want to be notified (evented) if they detach. In a simplistic manner, it's akin to the random exceptions you get with the "Telnet" Capability, but it's a little more formal in what you'll see/get when you use them.

Medium level...

  • This post is long enough, but...

The Medium stuff will pan out once the short-term stuff is available.

For the commodity stuff, I suspect it'll be around the auto-inclusion/detection/configuration of devices that use Homie and/or HASS MQTT... and making that experience point-n-click

ie. common, cross-hub, expressions of components and ready discovery and federation/inclusion.

Want to join two HE's together, it'd be rolled up the same model as ST-HE, openHAB-HE (etc) integrations.

Want to move from one to the other, federate first and slowly cutover... but let's not marathon before we prove we can crawl :stuck_out_tongue:


#26

There's hints there may be an alpha MQTT implementation in the works (Hope I can say this!).
Whether it makes it into the final cut is another question entirely!


#27

You mean the reddit post?