MQTT Client - Beta

🚧 Developers Code Share
68

While MQTT is great, I must defend Zigbee: it supports group messaging, which usually also eliminates the "popcorn effect." But that's only part of the story. Most (maybe all? I haven't used a lot but all I have do) Zigbee bulbs support scene clusters, and each bulb remembers it setting for a scene, which can also be recalled with one command. This is what what makes Hue scenes so fast (a recent Bridge update might have changed this, but traditionally, accessories like the Tap and Dimmer have just broadcast this command and the bulbs would immediately respond--they knew what to do--without Bridge communication being necessary) and Hubitat scenes sometimes a bit slow (the hub is emulating this, bless its little heart, with individual commands to each bulb for things like color and level). Theoretically, MQTT could do this too (broadcast a command like "activate scene 28"), but I'm not aware of any bulbs or lighting systems that support this.

In other words, Zigbee handles this quite well, and the "popcorn" effect has nothing to do with the protocol. :slight_smile: (It's also not proprietary; it's an IEEE specification. Anyone can make a proprietary profile on top of the standard protocol, but most devices people here use are ZHA 1.2 or Zigbee 3.0 using the standard clusters and whatnot as intended. Z-Wave is a similar story. Wi-Fi is...a mess beyond the fact the protocol itself is standardized, which helps in no way with how people actually use it for devices.)

(Z-Wave might have something similar; I know nothing about Z-Wave bulbs except that color implementation is generally a mess, and I have low hopes for the rest so I'd doubt it.)

2 Likes
69

Yes you are correct. If someone invests in a homogeneous Zigbee 3.0 enabled network it is about the same. BUT that isn’t the case with many people such as myself. Using low cost solutions like Shelly, Tasmota, SonOff devices that are MQTT enabled actually provide a lower cost but more robust solution over the standard Zigbee and other proprietary (which Zigbee is even though it is an open protocol) systems.

2 Likes
73

You need an MQTT broker somewhere (anywhere) - a Pi is a good place

79

I'm a proponent of MQTT's use for home automation and it's been a reliable technology for my applications. However, I wouldn't go as far as to make the claim you've stated.

For starters, Zigbee and Z-Wave are mesh networking systems with defined messaging protocols and operating frequencies. In contrast, MQTT is a messaging protocol, full-stop. It's most commonly implemented as a messaging protocol over a Wi-Fi network (but it could be used with other networking standards). So the "robust" aspect you stated is a function of the Wi-Fi network and not the MQTT protocol in isolation.

Only a limited number of home automation devices currently ship with built-in MQTT supported. This is in contrast to the broad array of products available using Zigbee and Z-Wave. The approach HA hobbyists have used is to replace the firmware of whatever device allows it (typically devices using an ESP8266) with ESPeasy, Tasmota, ESPHome, Espurna, etc to allow it to operate via MQTT. This is a boon for HA hobbyists but a steep hill to climb for casual HA users who are better served by devices that work 'as-is'. More so when you consider how many different parameters can be tweaked in, for example, Tasmota to optimize the device's functionality and performance.

Using MQTT over Wi-Fi means it is not well-suited for battery-powered devices, such as motion/contact/water/environmental sensors, because of the higher power consumption of Wi-Fi-based devices. In contrast, Zigbee and Z-Wave were designed to operate with low-power devices so are likely to dominate this space for some time to come.

I didn't quite understand what you meant when you said Zigbee is both proprietary and an open-protocol. Perhaps you mean that some manufacturers have put their own spin on the ZHA standard (or ZLL) thereby making their products not entirely compatible with Zigbee coordinators (hubs) from other manufacturers. They may do that due to a misinterpretation of the standard or to simply constrain users to buy their products. Ultimately, their implementation is 'exclusionary' and the Zigbee protocol remains non-proprietary.

To put it in perspective, one can argue that the various implementations of MQTT are 'exclusionary' because they are incompatible with one another. A device flashed with Espurna won't readily communicate with a device flashed with ESPeasy. One needs to massage the payloads and map the respective topics before any valid exchange of commands takes place. To avoid integration issues, one could simply flash everything with the same flavor of firmware. However, one could say that sounds a lot like buying Zigbee devices from the same manufacturer.

I believe the addition of an MQTT client will be a boon to Hubitat's users. However, its advantages should not be misrepresented as being superior to purpose-built standards. It is very useful but not optimal for all applications.

7 Likes
80

I know Insteon and UPB technologies support scenes and, perhaps only of historical interest, even some X10 devices support it (the ones manufactured by SmartHome). It's probably a safe bet that Lutron (Radio Ra2 and Caseta) supports scenes as well .

I agree with you that scenes are an efficient way of controlling groups of devices. Instead of sending a dozen commands to control a dozen devices, only one command is needed. This places less demand on the network (be it wireless or wired) and produces a more pleasing end-result (no 'popcorn' effect).

FWIW, I've used UPB dimmer switches in my home for almost a decade. Each dimmer will respond to a scene command as per its individual programming (i.e. upon receiving scene #1, one dimmer brightens to 50% at a rate of 5 seconds, another to 25% immediately, a third to 75% over 10 seconds, etc). All UPB dimmers also have a 'blink' feature. So I could send one scene command to make them all blink (for alarm purposes). This is far more efficient than sending a continuous, repeating stream of on/off commands to dozens of dimmers!

Theoretically, all of this possible using MQTT. The responsibility lies with the device's implementation of MQTT. It must subscribe to a topic dedicated to scene control and then react accordingly. I'm not aware if any of the popular firmware that's available (Tasmota, Espurna, etc) implements it.

81

Okay, so after struggling for a while to get any functionality between the Hubitat and Home Assistant via MQTT (using the Generic MQTT Driver), I have finally gotten it to work! Yassss From a device created as a Generic MQTT Driver, I can subscribe/publish to topics from Hubitat and from Home Assistant (via Node-Red). I am seeing updates on the Hubitat side after injecting strings/values from Node-Red. Similarly, in Hubitat, the device added as a Generic MQTT Driver has a Publish MSG command which I am able to use to send custom strings (published to topic in the MQTT Broker) successfully. By doing this, I am able to receive the string in Node-Red and use the message to trigger certain events on the Home Assistant side.
Now, although this is a big step and I am satisfied with my progress, my end goal is to use the Samsung Multipurpose V5 sensor (paired in Hubitat) to trigger effects to my WLED app (in Home Assistant) when acceleration becomes active. This is where I am now running into problems, as I have spent most of the night into this morning attempting every conceivable method (albeit being a complete newb to this) to use the acceleration active status as a trigger to either publish a state or MSG to the MQTT Broker. I have tried using Rule Machine and Modes but have found no way to achieve this.
Does anyone have any idea or known method of doing this? Do I need to be using a certain community app or is this something that should be done by editing the actual MQTT Driver code, as I have tried playing around with the Salt Tank MQTT Driver .groovy file(published by developer Aaron - [PROJECT] Salt Tank monitor using Arduino & MQTT ) where he implemented a switch into the Driver but it appears to be taking events in to the Hubitat to control the switch, which I'm trying to do the opposite? If anyone has any experience or could point me in the right direction, it would be greatly appreciated as I don't have experience writing/editing .groovy files and my next step was to just blindly start modifying the .groovy MQTT Driver code.

82

I have written an extensive MQTT app that is currently available 'on request' in an alpha version. I am just finishing another release which has some major revision to the way virtual device drivers are implemented. It will be available as alpha5 and then assuming there are no issues it will become beta1 which I'm trying to make available before Xmas.

It will do what you need.. and additionally supports auto discovery of HE devices into HA and HA devices into HE.

I would suggest waiting a couple more weeks and then trying out the beta app...

Kevin

PS The alpha version is aimed at existing competent MQTT users with devices already using a broker who can provide feedback and bug testing rather than people just venturing into the MQTT world who might need more general help.

83

Since I am a noob when it comes to MQTT, I'll wait.

I do have an application. I now have a generator monitoring and control app, GenMon (GitHub - jgyates/genmon: Generac Generator Monitoring using a Raspberry Pi and WiFi), running on a raspberry pi and connected to my Generac. I have another pi now running Mosquitto and I am able to pass generator events over to the Mosquitto server and see them there with MQTT-Viewer. Logical next step is to extend this over to Hubitat.

1 Like
84

Looking through this thread and various code examples, I don't see anybody handling re-connections if the connection is lost. Can anyone point to code examples of handling connection drops and auto-reconnecting with this mqtt client?

85

My alpha code does handle reconnects - but more by implementing something that seems to work than design. The people who had early issues now seem happy with later versions. I haven’t seen any other code examples anywhere. Public beta soon although currently under a restrictive licence.

86

7 posts were merged into an existing topic: [Alpha] MQTT application

91

Does this api support tls? I want to try to reverse engineer something using azure iothub which requires tls for mqtt. @chuck.schwer I want to make sure it’s even possible before I start trying to figure out all of Microsoft’s over engineered requirements!

92

I wouldn't call requiring tls "over engineering". I would call that good basic security when the traffic is leaving a trusted network.

93

That isn't the part that is overengineering. They require you to jump through a lot more hoops than anything I've ever looked at. If Hubitat doesn't support TLS, there is no reason for me to investigate all of the other things (you have to do a multi-step OAuth 2, plus shared secrets, plus X.509 certs, plus HMAC256 hashes, etc.) since I'd be dead in the water if TLS over MQTT doesn't work. I agree TLS is important here... It's everything else that's killing me!

94

Yowza. That does sound a little over the top/over zealous/over engineered... Over SOMETHING anyway. :slight_smile:

95

Yeah it's fun... I've been trying to reverse engineer the Kohler Konnect app. I'm close, I just can't figure out how to get the shared secret. Unfortunately their android app is pretty heavily obfuscated making it hard to just decompile it and find out what it's doing. I'm in it for about a hundred hours so far :frowning:

96

Have you tried past versions of the app to see if they are also obfuscated? I've found in some cases the first versions of some apps aren't obfuscated (like Rapid Ring).

97

Just install Webscarab on a laptop and point your phone at it as a proxy. Then intercept all of the communication.

98

That won’t work. The app has the api websites https certificate embedded to prevent mitm attacks. So unfortunately I can’t use a proxy

99

You still can MITM but it becomes much harder. If you are lucky the certificate is not in compiled into code and it's just a resource. You replace it which breaks the signature on the apk but then you run the app in a phone where you can tell it to ignore signatures or you resign it.

If the certificate is embedded in code (those devils) then you can decompile, replace it and recompile. This is a giant headache so you have to decide how bad you want it.

Another fun option is if they use one of the recommended methods of certificate pinning... you can install the Xposed framework and then use one of the unpinning modules. I've had mixed success with this.

If you don't have an Android phone anybody can have an Android emulator.