First attempt with Node Red - Help with MQTT

I'm brand new to this so please be gentle! I'm trying to get Node-Red up and running. I've had limited success, stumbling along. I currently have an MQTT broker on my Pi. This receives topics published by my alarm system. For the last year I've been using those topics to switch virtual devices in HE to mirror my physical wired alarm devices. This was using a community MQTT app which is in beta. That's currently unsupported/has developed a bug so I'm wanting to set up Node-Red as an alternative as I'm currently unable to add more devices or edit existing devices in the current setup.

I've got Node-red installed on my Pi along with the Hubitat Palette which is configured and I managed to get a single virtual device switching a HE virtual device using the full topic path. However I have multiple individual devices in that path and I'm trying to avoid using multiple switches in the flow. This is the start of the flow:

The full path to the MQTT Lounge PIR is selfmon/vmod.010fbd/prio/inputs/read/1047 (1047 is the alarm device zone number) If I set the 'MQTT In' node with that full path including the specific zone:

And similarly with the 'Switch' as follows:

That works and switches the attached virtual device. However I want to shorten that topic path (to 'selfmon/vmod/.010fbd' and add outputs for multiple devices and states in that switch (so that I don't need an individual switch for EVERY MQTT device). I'm struggling with the topic path and syntax>. I've tried various combinations but nothing seems to work. Any ideas?

I suggest putting a Debug Node set to Complete Message Object. Then, trigger all of your MQTT devices and see what the debug message says.

image

That should help you with the syntax.

Also, as an experiment, I changed the listening topic to RFBridge/+ instead of RFBridge/RFDevices where the "+" is a wildcard. Then, I triggered something and, in the Debug Node, msg.topic was RFBridge/rfDevices and the payload was the Device ID. That would seem to indicate that you can use the wildcard but then need to use msg.topic in your first switch, not msg.payload.

I am an MQTT novice but try listening to selfmon/vmod.010fbd/prio/inputs/read/+ and trigger all of your devices and see what appears in the Debug Node.

1 Like

Thanks for the suggestions @stephen_nutt - the debug might be what I'm looking for as I needed to see what was happening in the background and didn't know how. To be honest, even if I have to replicate it many times using the full topic path - I've just realised that I can drag over the 4 nodes, do a copy/paste and then just change the one number in the nodes as below - though I'm not sure whether that will have cause any performance issues:

I'm not 100% sure I follow what you're trying to do, but if you want to have a single MQTT-listener and then pull out the various subtopics, you can use this in a function node:

subtopic = msg.topic.split("/")[2];

This splits the topic on every '/' and in this case takes the third section, so for

selfmon/vmod.010fbd/prio/inputs/read/1047

msg.topic.split("/")[0] = "selfmon"
msg.topic.split("/")[1] = "vmod.010fbd"
msg.topic.split("/")[2] = "prio"
msg.topic.split("/")[3] = "inputs"
msg.topic.split("/")[4] = "read"
msg.topic.split("/")[5] = "1047"

2 Likes

In the Debug Window, the button indicated by red will copy the path. The button indicated by blue will copy the value in that path. It will work for msg.topic, msg.payload, etc. You can use the copied values in your Switch Node to ensure accuracy.

1 Like

Hi @Alan_F . Yes I wanted a single listener (and preferably a single switch). The topic is the same until after read/ where that 4 digit number is specific to a single device on the alarm (PIR, Door Contact etc). Regardless of the device type on the physical alarm system - it sends 'OPEN' or 'CLOSED'. I can also monitor outputs on the alarm programmed as Alarm, Power Fail etc - those will be ...prio/outputs/read/1234 and send 'ON' or 'OFF'

Edit: Struggling with exactly where to add that code in the function node (On Start, On Message, On Stop). It just gives an error to say the message has not been defined.

The path I have is correct and it just shows the payload simply as 'CLOSED' or 'OPEN'

Single MQTT listener to the top level topic
--->
   Function node to split the topic and get the device number
      For example: 
       msg.device = msg.topic.split("/")[5]
        return msg
    --->
       Switch node to route the message based on msg.device
       --->
          Switch node to route the message based on msg.payload
          --->
             Hubitat Command Node (for that device and payload)

Just curious, what bug did you run into in the hubitat MQTT driver? I am using it and haven't noticed any problems (yet).

There's an issue where if you try to add or edit any virtual device, the window won't pop up allowing you to see the topic and edit it. You can add a device but not edit it so I'm in limbo. Everything previously created works still but I can't add any further devices or make changes. Another user reported the same issue. Kevin is aware but hasn't addressed it as yet (I don't want to bother him). The other user rolled back the Hubitat platform and found that it was a HE update that broke it.

On start is code that runs every time the flow starts or restarts. You don't need that here. I use it sometimes to make sure a flow or global variable always has a default value on a restart (for example: "flow.var1 = 'Initialized'" avoids any issues with referring to flow.var1 in the On Message section.

On Message is what you want. That is the code that is run for each message received. Within the On Message section you can refer to all the message parts, msg.payload, msg.topic, etc.

I've never used On Stop so I'm not sure exactly when that runs.

I'm using this one:

Which as far as I can tell still works even though it hasn't been actively maintained in a long time. The setup is pretty bare-bones, but it seems to get the job done.

1 Like

I'll take a look at that thanks. I only want to do one way - MQTT Broker > Hubitat Virtual Device. Most of what I came across while searching was for biased toward exporting HE devices via MQTT to HA.

Edit:

Note: This project is deprecated, not under active development. As noted below, please consider GitHub - xAPPO/MQTT: MQTT client for Hubitat for integrating MQTT functionality into your Hubitat projects.

That note references the app and driver that I've got issues with

I'm pretty sure this one is one-way.

I handle the Hubitat -> MQTT end of things using Node-Red. I use the Hubitat Events node and connect it to an MQTT out node. I filter out any devices named "Ping" as I used to have pings running from the Hubitat (all of those have been moved to Node Red now, so I really don't need that step anymore). Then I use a function node to modify the displayName to remove characters that don't play well within MQTT topics, set the message topic and payload, and pass the message to an MQTT out node.

I don't actually use the Hubitat -> MQTT flow for anything at the moment, but you never know when it might be useful to be able to pull Hubitat states out of MQTT.

If you decide to try out the Kirk Rader version and need help, let me know. It isn't that complicated, but I found the process of creating the child devices a bit confusing.

I think I'll stick with Node-Red and try it out. If I go ahead and just use multiple connections as shown in message 3 (as my head is scrambled by the additional function nodes and switches):

  • Do those multiple MQTT In connections have a performance impact over a single one with multiple function and switch nodes?

  • Are they ok to be in a single flow, or should they be done as separate flows (as they're not interconnected)?

I don't know if there is any performance impact of having multiple MQTT-in nodes. It might even be less demanding on the system since you may be able to skip a lot of splitting and switching downstream from a single node.

I put everything that is related on one flow, otherwise I would have dozens and dozens of flows and it would be unmanageable. So everything related to my Hubitat is on one flow, everything related to Blue Iris on another, everything related to Google Home on another. There can be some cross-over, but in general I know where to find things.

I only recently discovered the grouping function, which makes things more organized. Connected nodes are grouped, and I usually add a comment node to describe what they do.

1 Like

Thanks Alan. I'm on setting it up now separately. For some reason using 'selfmon' as the main topic just wouldn't output anything and I gave up. From a maintenance point of view it might be better to keep each alarm zone separate, as it's easier to amend, add and delete links to zones without risk of affecting other zones.

Out of interest I switched on info logging for both a virtual device linked to the broken MQTT app and the new virtual device linked to node-red. Both virtual devices are mapped to the same topic on the same broker. The quick test showed that the node-red version completed the actions around 0.3 seconds faster. Bearing in mind that the virtual devices are used for motion lighting, that is welcome. 'Lounge PIR' is the node-red link, '1047 Lounge PIR' is the previous MQTT app link:

Ugh :frowning_face: I thought I was sorted but I've hit a major hiccup. I'm not sure how to sort it with node-red. I'm sure there must be a way but it's beyond me...

The way that the alarm system works is that it's split into blocks of 8 zones (separate Remote Input/Output boards or 'RIO's per 8 zones) . So in my example above - the 1047 zone address equates to:

  • Data Bus 1
  • RIO board 04
  • Zone 7

With regard to MQTT when any one Zone changes state on a RIO, the states of all 8 zones are updated. IE - If zones 1041 to 1048 are 'CLOSED' and 1047 goes 'OPEN', it sends:

1041 Closed
1042 Closed
1043 Closed
1044 Closed
1045 Closed
1046 Closed
1047 Open
1048 Closed

The problem with the current flow(s) is that because I'm using the command node, when one PIR changes state from Closed to Open, the other seven zones command their respective virtual zones closed. Even though the actual device is already closed, that 'close' command is sent to the virtual device. Those non 'changes' are logged and it's creating a huge amount of traffic.

I need a way in node-red to only use the command node IF the topic payload has changed:

Closed > Closed - Do nothing
Open > Open - Do nothing
Closed > Open - Send the change to the virtual device
Open > Close - Send the change to the virtual device

Maybe a function that on start looks at the initial value (OPEN or CLOSED) then on each message only continues the flow to the switch IF the value is changed. That is how it would have worked in the other MQTT app I was using.

Edit: I've just discovered the 'filter' node. Perhaps I need that between the MQTT node and the switch?

For some reason using 'selfmon' as the main topic just wouldn't output anything and I gave up.

If I understand correctly... nothing came out of the MQTT node when you tried to just use the first topic. You need to use the MQTT '#' wildcard, so subscribe to "selfmon/#". This works at any level, so topic/level2/level3/# will subscribe to everything level 3 and below.

1 Like

Filter may works, but it sounds like you may need to set a flow variable to store the state, and then compare to that variable to decide whether to pass the message.

You may be able to do this in a change node or using a simple function. Remember that everything is message based, so unless you store something in a variable, you can only use what is in the current message. So to compare to the past state, you need to store it.

I would probably do something like this:

Assuming the message payload is either Open or Closed, for each message:

// get the zone from the MQTT topic
var zone = msg.topic.split("/")[5] 
// compare the payload to the stored flow variable holding the previous payload
if ( msg.payload == flow.get("zone_" + zone + "_state") )
{
// do not send the message - that state has not changed
return null
}
else
{
// set a variable named "zone_X_state" (where X is the zone number)
flow.set("zone_" + zone + "_state", msg.payload)
return msg
}