[Beta] MQTT beta 3d (released 5th July)

I have a question....

Not many people post here yet there were a 100+ alpha access requests. I do get a lot of PM's from people I've never heard of but I have really no idea how many people use my app.

If I was to include a 'feature' that just let me know how many people use this app by advising me of your HubUID would that worry you ? This is not the IP or MAC address but seems to be a random but unique identifier (the only one) I have available to use. I don't believe in any way it identifies you to me. I'm wanting this really to know how much work I should put in, in the future, to this app.

This app is free to use and always will be from me, yes I have a PayPal coffee donation link but my incentive is for many happy users and pushing HE to be the leading integrated hub in the MQTT world, which I believe it now is... 'probably' ... but I'm just not sure how many of you there are..

2 Likes

Would you support me including a fully anonymous feature that advised me how many people use my app ?

The last option has an inference you would and so defeats my objective.

  • No
  • Yes
  • Yes - as long as I can opt out

0 voters

Thnx for the beta3c release. I wanted to get started with the virtual MQTT devices and json payload (re)mapping.

I tried installing but ran into several issues. After some fiddling around I decided to remove the app, driver, devices and app/driver code, reboot hubitat and start from scratch.
Still, the following happens:

  1. import driver code from github: check
  2. import app code from github: check
  3. creating app: hangs in loop (browser window waits indefenitily), log shows some action (see screenhots), but when refreshing the browser, there is no MQTT app created. However, the mqtt child device driver is created.
  4. Second try to create the mqtt app seems to succeed (the browser shows the hubitat mqtt app). However the log shows errors.
  5. Setting Hub name and MQTT Broker address results in the following entries in the log
  6. No connection to the broker

It took me a while to get back to you, because I tried the above several times to be sure it's reproducible and no trivial issue.
Could it be that there are some leftovers of objects/code somewhere in hubitat that could cause this behaviour? I really tried to delete everything and rebooted several times.

I wanted to downgrade to beta3b to check whether I could get that version working again but can't find the files in the repository.

Any suggestions welcome...

UPDATE: I manually deleted the MQTT Child device driver. Then again configured broker in the mqtt app and now it's connecting to the broker

So I definitely would choose option 3 but allow it at least initially. It just seems like the classy/fair thing to do even though it can mess with your #'s.

Would have the default setting to "opt out" unchecked - maybe a blurb explaining why you want to collect data, how it is useful etc etc. Also maybe collect the version of MQTT in use at that time as well.

For legality purposes (especially seeing how someone will throw out GDRP, CA Privacy laws), the default option should be to opt IN, not OUT... either that or force a "yes I have reviewed this" somewhere...

and make it VERY clear what it's collecting, how often, etc...

Personally I'd have no problems with on every "save" it sends some information such as # of devices pushed to HA, number of devices from HA, etc...

1 Like

Continuing my question from here: [Alpha] MQTT application

@kevin So I tried to set up the device as a "Virtual Temperature Sensor" since I get both battery and temperature data from my RV battery. However when I choose that as the device type it doesn't give me the option to choose any attributes:

When I choose the edit option:

Then click "Update" it takes me back here:

EDIT: Nevermind: I forgot to select it as a device first in the first selection menu before proceeding to "Edit". My bad. Now I've got it to where it let's me enter the attributes:

I thought Temperature Sensor would make the most sense... As I thought it would have a battery attribute as well as a temperature attribute, both of which are things my BMS tracks for my main battery pack. I guess the virtual temp sensor doesn't have a battery attribute?

You mentioned the Text driver... I wasn't clear on what that was for exactly or if I needed that?

-Jeremy

@kevin, this is what I was talking about some weeks back (probably in a PM) when I noted the same behavior.

One additional question I have:

The Victron MQTT broker has an odd behavior: In order to avoid sending unnecessary messages it automatically shuts down after 60 seconds. That means if a client wants to continue receiving messages from the broker it has to publish a message once every 60 seconds to do a "keep alive". The example Victron provides is to run something like this every 60 seconds:

mosquitto_pub -h localhost -t "R/04a316c3eb1f/system/0/Serial" -m ""

My thinking was to create a virtual button or switch and just set up a rule that pushes the button every 60 seconds, and map a button push to publish a message on that topic. So I created a virtual button like this:

Do I need to add something to tell it what "message" to publish? It just needs to be an empty string.

Hmmm, @kevin's app sends homie/steve-s-hubitat/$implementation/heartbeat every 20 seconds. I wonder why that doesn't do the "keep alive"? Does every subscriber have to publish in addition to the publisher?

The editor is ‘quirky’ in that it sometimes requires two updates to accept a value but the behaviour is as intended.

The app does a few things before ‘taking you back’ . It adds the value you have just edited to the data value within the device and it makes sure MQTT is subscribed to any needed topic you have just added. The values from my dynamic page were getting mixed sometimes if I did not return to the entry screen. I will revisit this but not immediately.

Correct, it has neither the capability or the attribute. My MQTT Text driver does have the battery capability and a text attribute that could display anything including temperature and you can include a degrees suffix. It does not have a temperature capability.

A non workable implementation. You need to use a proper 24/7 MQTT broker like mosquito. Are you sure this is the broker and not a client ?

Regardless as Steven mentions my app does send a heartbeat every 20 seconds so this should suffice but that is a feature I will likely deprecate.

I was likely going to send the hub UID, or a hash of, every time done was clicked. I am not interested in any other information except how many hubs run this app. If it is just a dozen or so then I will likely not evolve the app any further feature wise but just release any bug fixes. However 100+ say and I have groupies to satisfy.

I am not a programmer and have other commitments so have spent over 6 months in the evenings/early mornings so far , but learned a lot too. I just want to know if people are, or are not using it.

The ‘can opt out’ is always likely to be the final choice but I was trying to gauge how many would actually do this.

Indirectly any existing app that accesses a remote url can stealth implement this any way by IP. Even loading a remote image or code or controlling a cloud device.

Yes, unfortunately it's the only option if you want to interface with Victron equipment via MQTT. (The other available protocols are Dbus, CAN-bus, or MOD-TCP, none of which are available in Hubitat that I'm aware of). Here's the documentation that explains the keep-alive: GitHub - victronenergy/dbus-mqtt: Venus OS service mapping the D-Bus on Venus OS to MQTT

It should be workable though right? I just need to be able to publish a message to the broker every 60 seconds?

Yeah this doesn't seem to work for the keep alive. Maybe because it's not being published to a topic that the broker is aware of? Because my Virtual Temperature Sensor only updates the battery stats if I manually publish a message via the "PublishMsg" button on the "MQTT: Child device driver" page.

This isn’t a battery device ? I can only assume they kill the broker to stop too many bridges to their cloud broker. :slight_smile:

This doesn’t allow an alternate broker to be configured ?

By default, dbus-mqtt will connect to a Mosquitto MQTT broker running on the GX Device itself.

Another option would be to run a local MQTT broker 24/7 that is bridged to the Victron (or their cloud). I’d recommend this if no other solution.

Please don’t use a non 24/7 available MQTT broker with the MQTT app is it is not (yet) definitively robust for frequent disconnects and implements an increasing back off’ timer on failure that can ultimately give up.

Well, this is where it gets hairy. So the page I linked to are docs for "Venus" which is the open source OS (Based on Linux) that Victron developed for their own products. They have a commercial product called the "Color Control GX" which runs the Venus OS.

If I were building my own Venus OS based device (IE: if I took a raspberry pi for example and installed Venus on it and built everything from scratch), I could use whatever broker I wanted. Because I'm using a commercial off the shelf Victron product, they flash the storage device every time you apply any updates to it. So making changes that "survive" a firmware update is a tricky area and something I'd really like to avoid.

I'm also trying to avoid having to add an raspberry pi as a bridge, as adding components just adds even more complexity and parts that can fail.

I guess I'm not seeing the concern with just publishing an empty message every 60 seconds? That seems like it should be simple enough to accomplish? There are quite a few people who are doing it that way and it's working for them, just not necessarily with Hubitat.

Default keep-alive interval is 60 seconds

This implies it’s configurable. Can it be set to 0 for permanent ?

My keepalive won’t work because it needs to go to this topic

Topic: R/e0ff50a097c0/system/0/Serial
Payload: empty

Where I send my heartbeat just add another line after it that sends to this topic... but the broker must be connected or it will fail.

I can send you a line number if that helps, it’ll be in the MQTT client driver. Every time I update though you will have to add it in again.

PM me if you want more help

It is configurable if you're building your own Venus device. On the Color Control it's not (As of now) from what I've gathered reading through the various posts of people who've integrated with it via MQTT. Pretty much everyone just runs some sort of loop to publish a message every 60 seconds.

Thanks! I'll give this a shot. But I'm curious: On the button where it has the option to do "Pushed MQTT command topic": I was under the impression that would allow me to have a button press event publish a message to that topic. Is that not what this is for?

The reason I ask is there are certain things on the Victron that can be controlled by writing commands to certain topics. For example I could switch the inverters off to save battery power if it's not hot enough to need to run the A/C, or send a command to start the generator when the weather forecast shows that it's going to be cloudy (No solar). I was hoping I could map virtual buttons and/or switches to functions like this?

Thanks again,
-Jeremy