Need MQTT for Hubitat Elevation

I use HE for local control & debating going back to ST as it integrates with Home Assistant. I stupidly got my family excited to be able to use HomeKit which all my devices were pushed to from Home Assistant. Which was fed to ST from HE. But no color control so I eliminated ST so I was dealing with 2 “hubs”. HE & Home Assistant. Once those worked 100% I have plan on adding the rest back in. But aggravating as something that works sorta if you want on/off. Or I go buy 36 more Hue bulbs.... It’s that or my family sends me off the plank (lesson learned no hints about features until 100% tested.

Well I would like that option but I believe it’s just not possible to build such a thing currently. The current hack is dreadful architecturally. I think only Hubitat could do it.

As for overlap with volume users look at companies like Shelly and Sonoff with devices directly supporting MQTT... it’s on the increase and avoids every platform needing their own drivers.

K

Alternatively, support Groovy's @Grab and let us get Paho (etc) ourselves :slight_smile:

With that, community members can do the cross-tech integrations and show how it'll benefit (with an open, event-based and documented delivery system)

Having the API exposed would allow cross controller federation (vs building them one-at-a-time) and ease the process for those wanting to migrate between the various Home-Automation platforms.

Some examples:

  • a) Potential to build Support for Homie and/or HomeAssistant conventions for federation/conversion (currently supported by Hass, Home-Assistant, Homie, openHAB )
  • b) Simpler data push/pull with InfluxDB / Telegraf.
  • c) COTS Phone/Device tracking (via services like owntracks )
  • d) One step closer to AWS MQTT, Google MQTT (etc) for paid services

The above wouldn't eliminate the need for Drivers on Hubitat, but they could be used to lessen the amount of code that's required.

eg. Alarm systems integrated via the Homie Convention vs one-at-a-time (as HE specific Drivers)

Initially MQTT would be for the advanced users, but it's often those folks that are writing the AddOns, and then inviting their (non-tech) friends to use the platform.

BTW, I agree that on-boarding new, typical, users should first priority, before some of this, but a few enablers (like @Grab support) could pave the way :slight_smile:

I didn't think HE's WebSocket interface, and underlying protocol atop the WS, was supported. Has that changed?

2 Likes

No, it hasn't changed. However, @chuck.schwer did state in another thread that the devs were going to make a websocket server official at some point in the future (with no ETA).

1 Like

Ah, so it’s a Logitech Harmony style integration atm...

LOL I think I’ll skip it :wink:

1 Like

Not really a comparison to a home automation hub. Shelly is open source cloud-based. No support staff like here. Sonoff is hardware, with very minimal cloud based software. No support staff writing code and supporting these products at the level a full automation hub needs.

Sorry, got your post out of order when moving this to new topic.

1 Like

Ish? The Harmony integration uses the Groovy websocket client. The websockets I'm using (for various things) are hosted by the HE hub itself (ws://[hubitat]/eventsocket and ws://[hubitat]/logsocket).

@corerootedxb,
My reference was to using an undocumented API, hosted by the (local) Logitech Hub... and then people going nuts when it changes, and their integration(s) break...

A style of integration I casually refer to as Logitech Harmony style, given they've had the most recent / widely publicized failure for doing it.

ie. Move along, nothing to code-against here people :wink:

1 Like

Ohhhhh!!! Yeah, I see what you mean now.

True on all counts.

Sure, you make it sound easy... What API methods would you expect to see or please show me how another hub has what you already want?

Doing an MQTT client that would connect to a server / broker to get and send messages similar to our websocket or telnet capability isn't that hard, but that basically means a driver to map the capabilities to for each MQTT "device" or signal you want to send.

Making Hubitat a full blown MQTT broker is a completely different argument. Which, for the advanced users, who already known what MQTT is, are already going to have a main server to communicate to.

So, please point me to an implementation of MQTT that facilitates the examples you provided that would provide any insight as to how to approach external MQTT interaction beyond just a client interface to the drivers as a communication layer for pushing and getting messages from a broker?

Ahh - you missed my point. I was meaning companies like Shelly and Sonoff are creating very low cost plug and play peripheral devices that can easily be integrated into controllers supporting MQTT. I was meaning the advent of novice users buying such devices and hoping, even expecting that Hubitat could easily support them. Not needing a device driver for every specific manufacturer.

1 Like

I agree, I don't think that's really necessary or advisable. Keeping HE as a top performer is important and all MQTT users ATM have their own broker. Just basic client functionality would be great, even if it just supported some basic device types like onoff, dimlevel, text perhaps. I think Rule Machine might be able to assist here if people needed more.

Ummm I’m new to all of this & already on my 2nd HE after the power port feel out with the power cord provided. I WOULD gladly try out the MQTT client but that won’t connect to my logged in Mosquitto server. I am sure you get a thousand requests for stuff & didn’t mean to start all of this about a post. I’ll consider it a mute option.

It can be made to work - as I use it currently but it is just a kludge via http, and the sad part is that the kludge about because of the deficiencies in ST lcal network capabilities.

Integrating MQTT to HE via HubConnect is another possibility to investigate, but it would be an external rather than integrated solution

TL; DR version

I'd like the MQTT Client API exposed to Driver/App authors in the short term - without having an(other) intermediary party (NodeJS, Python et-al) to deploy, manage and run.

Short term, I don't need HE to host a MQTT Broker, Mosquitto on my 3yr old rPi trucks along fine with a significant event volume.

Background

I came from a hybrid (federated) world of openHAB (5 yrs), and Vera (10 yrs). I federated these to get the benefits of both worlds (richer/active Rule deveopment in openHAB, stronger Z-Wave support in Vera), and to ease the transition/conversion process.

I also got access to my Alarm Panel which, until recently, was also served by my Vera.

No single / complete HA solution - open enough to let us glue it together

One solution couldn't "do it all", so I brought in other bits to handle the broader parts of a whole home automation, monitoring and control system.

eg. Flexible dashboards, iPhone control devices, Long term data archival, Diagnostics, multi-modal Notifications etc.

Data collection

I collect a lot of data about my house, mostly for off-line understanding and planning, but also for curiosity about the overall behavioural patterns for the system components, and for the house itself - ultimately to work out optimizations, maintenance, etc.

For these, I've used different databases over time, most recently InfluxDB (Event data) and rsyslogd (Log data)

Separation of Producers / Consumers for the data

With Influx, I originally directly integrated, and in more recent years I did it over MQTT - allowing me to swap out Databases without impact on the upstream/producer systems. It also meant I wasn't querying Influx (history) to get current state.

eg Data publication:

  • btmon (for Brultech Energy Monitors) pumps my energy production/consumption into MQTT, and Telegraf configured to push the MQTT events into Influx.
  • openHAB publishing change-events to MQTT, and similarly into Influx
  • openHAB subscribing to energy events, and alerting when they go above limits/thresholds

All my data for the last ~4yrs is in InfluxDB, as events, and rsyslogd as raw logs. Every light switch, every temperature change (2x Nest + Outside weather), all the energy usage (5s samples for 30+ household breakers)

... except for the stuff I moved to HE, as I'll have to custom author something to push the data there correctly.

Separately, I swap out household components on the regular. The Hubs isolate my automations from the Device changes (eg. Wemo -> ZWave) and the MQTT helps isolate my data collection & dissemination from my Hub(s)

HE has been great at simplifying the common Automation tasks from openHAB/Vera, but I've lost a LOT of visibility to the data I used to collect... not because it can't be done, but because it takes a long time to (re)write functionality that's available, but HE doesn't enable.

Use Case examples

  • Want to know if the washing machine is on, I can get that back from MQTT.
  • Want to know the Espresso machine is heated and ready to go :slight_smile: ...
  • Want to get alerted when the outside lights having a wiring fault, I can get the data from MQTT

In each case, I don't care where the data is (ultimately) coming from, just that it's centrally available, and historically archived.

Could I do these with other things, sure. Would I have to change those each time I changeover a physical Device, a Home-Automation Controller (etc)... probably.

Don't get me wrong, I went into HE with my eyes open about it's limits, and I'm ok to wait... I like it's more declarative nature for the simple stuff.

Examples

  • Owntracks (linked above)
    I stopped using this just prior to the transition, so I'd get used to it not having it.

  • openHAB MQTT Driver (linked above) - implementing both Homie and HASS's MQTT form.

    I was going to redo my Proprietary/Closed Security Alarm Panel driver in the Homie convention (Linked above), and then it could be integrated once for all home-automation platforms.

    I've recently re-written my Vera Alarm system plugin (Lua) for HE (Groovy) but I'd rather have written it to MQTT / Homie (or HomeAssistant's equiv) to avoid doing the same thing repeatedly, and to chose the language I'd prefer to work in.

  • Threshold alerting for Energy use/anomalies
    This one is fun, since I recently had an electrical fault in one of my circuits.... something that an outlet-based monitor would never have picked up.

  • My transition to Hubitat
    If the MQTT Client API parts were available, I would have bridged/federated openHAB (using Homie convention) to get up and running (much as I did with Vera -> openHAB originally). It would have been a lot less disruptive, and you see folks here federating their ST and HE devices (as well as multiple hubs)

    Luckily the Mrs was away for 4 days when I did the primary conversion, and the house was dark :wink:

Expectations:

Low level (API for Driver/App authors) ...

  • Connect to one-or more MQTT Servers at the same time.
    One per top-level Device would be fine, using TCP (most common) hostname and port, preferably with TLS capability for remote service security.

    In my use-case, one is my "local" MQTT Server, the real Hub of my system, and the others would be for connecting to services like owntracks. Initially, I'd setup my MQTT Server to daisy chain the Topics to the owntracks stuff, which is how I do it now, but I'd prefer clients to connect (declaratively)

  • (Bulk) Topic Publication to the Device's (configured) MQTT Server.
    For my use-case, I'd write an App and "register" the HE Devices I want to watch, sending their change events to my MQTT server for archival (etc).

    The form of this is where it starts to get interesting, since there are many data formats, Topic-naming conventions (etc).

    This is why I'd prefer to see the integration available in API form initially, as it's likely the initial implementations would need to perform transformations on the data before sending it off (eg lights can be "on"/"off", 1/0, true/false, 100/0 etc). The HE App, in concert with it's Top-level Device, will need to acquire the data and perform these transformations before delivery.

    API-wise, this is the functional equivalent of publish.multiple in Paho

  • (Bulk) Topic Subscription to topics on an MQTT Server.
    I think of this as being akin to a "MQTT" Capability, where there's a set of methods I could call upon to setup the subscriptions, and then everything would be delivered via MQTT-Capability specific methods (akin to parse for Protocol.TELNET, but delivering an object representation instead of plain strings)

  • Receive events to a HE Handler (like parse, kinda, but more structured)

  • LWT Support - letting everyone know you've died, they've died, we've all died.
    MQTT has a concept of a Last-Will-and-Testiment that "both ends" can usually sign up for. Each entity with data (typically) uses the LWT to signal they're alive. Homie convention uses this, and we'd want to be notified (evented) if they detach. In a simplistic manner, it's akin to the random exceptions you get with the "Telnet" Capability, but it's a little more formal in what you'll see/get when you use them.

Medium level...

  • This post is long enough, but...

The Medium stuff will pan out once the short-term stuff is available.

For the commodity stuff, I suspect it'll be around the auto-inclusion/detection/configuration of devices that use Homie and/or HASS MQTT... and making that experience point-n-click

ie. common, cross-hub, expressions of components and ready discovery and federation/inclusion.

Want to join two HE's together, it'd be rolled up the same model as ST-HE, openHAB-HE (etc) integrations.

Want to move from one to the other, federate first and slowly cutover... but let's not marathon before we prove we can crawl :stuck_out_tongue:

3 Likes

There's hints there may be an alpha MQTT implementation in the works (Hope I can say this!).
Whether it makes it into the final cut is another question entirely!

You mean the reddit post?

Sorry to beat a dead horse but id like to add my 2 cents. I am actually leaving Home Assistant for Hubitat because I need proper Radio RA2 support. The main part of my system relies on a bluetooth presence detection that sends messages to an MQTT server. GitHub - andrewjfreyer/monitor: Distributed advertisement-based BTLE presence detection reported via mqtt

It would be nice to be able to continue using this. Basically having a way for Hubitat subscribe to an MQTT server and allow me to make a sensor based on MQTT topics would be perfect. Being able to send custom MQTT messages to a broker would also be very helpful.

2 Likes

Apparently it's coming, according to @patrick on https://www.spreaker.com/episode/17682937?utm_medium=widget&utm_term=episode_title&utm_source=user%3A4487293

The whole podcast is well worth a listen.

2 Likes