Rpi for Nodered and MQTT

Was planning on trying my hand at Nodered on a Rpi with the Rpi also serving as the MQTT broker. Reasons: just want to learn something new, want to “future proof” (and this is purposely in air quotes) support for my Tasmotized devices that are used in Hubitat automations (currently using T4HE but don’t know how long that will keep working as I update the Hubitat OS) and I want to be able to have future flexibility for adding in devices that support MQTT but are not integrated with Hubitat.

Question: for those of you that have done this, what type of Rpi are you using (how much RAM, sd card size, and Rpi model is needed to comfortably run Nodered as well as serving as a MQTT broker)?. Also, do you find that a cooling fan is desired as this will run 24/7 or is CPU usage low enough in this application that convection cooling with heatsinks are all that are necessary? TIA for any help!

I have an old RPi 3 running Mosquitto broker for my personal weather station. It seems to work fine. The only issue is I don't think you get full network speed with the interface due to it routing through the usb2 bus. For my use-case this is okay.

I also have several RPi 4s in operation - I use the 4 GB variety but you can get away with less. For storage I've used 32 GB and 64 GB cards whichever is cheaper. You don't really need much. Also consider getting a similarly sized usb drive as a backup.

I have a basic cron script that backs up the card to the usb and can failover if needed.

In terms of a fan - have used both with and without, generally 5-10 (F) degrees cooler but not really all that noticeable to me. If you get a metal case then that can help although the fans that have come with the kits are pretty quiet.

Also if interested using wireguard on the PIs as well. This is a nice PiVPN script I found:

Be sure to do the following if installing WG:

sudo sysctl -w net.ipv4.ip_forward=1

edit: Also I am using ubuntu's RPi server distribution which is 64 bit.

2 Likes

Sorry one more post..

On my RPi4s am running Node-RED, Homebridge & Wireguard. Node-RED has a limited built in MQTT broker which I am using for internal NR stuff.

1 Like

Erik,
Thank you so much for taking the time and effort to post such a detailed response (your willingness to help is one of the primary reasons I switched over to the HE community). The information you provided is extremely helpful for a newbie in this area and greatly appreciated! I will incorporate your suggestions into my new “project”. Thanks!

2 Likes

Cool!!! Happy to help!! Make sure you post back if having any troubles or need ideas. There are a lot of really good knowledgeable people here on these forums.

Running NodeRED and MQTT on a Pi3 seems fine but the temptation is always to add one more thing and then things can slow down - I have a small HA test system running on mine too. But my expansion plans should be really to utilise other Pi3's or just use Pi4's the latter is probably my future intent for "future proofing".

I haven't had problems with overheating but I use good metal cases or just leave them open. The two plastic case kits I bought with included fans do spin up pretty frequently.

1 Like

Would you mind sharing the script? How do you find rpi-clone - no issues?

Does the 64 bit OS provide any tangible benefits over RaspiOS? I'm happy with the stock but curious to know.

Pi 4s generate a lot more heat - I used a Pi 3 previously with a printed plastic case (vented) and heatsinks on the chips for a long while. When I switched to a Pi 4 had to go metal case and fans to keep the CPU temp reasonable.

Hi everybody!

I just configured a MQTT broker on a docker image, running on a Raspberry Pi Zero W, part of a project that I'm developing:

To be sure that everything is working fine with the MQTT broker, I'd like to capture, at the HE, data sent from a Tasmota device (through MQTT) and routed through the RPi. Like this:

  • Tasmota device ==> MQTT broker ==> HE

and, of course (and obviously), send commands all the way back, from the HE to the Tasmota device:

  • HE ==> MQTT broker ==> Tasmota device

I did some research and, if I'm not wrong, I cound't find a solution for the HE that do both ways, send and receive data/commands at the HE, meaning, having a Tasmota device mapped at the HE using a MQTT broker as a man-in-the-middle.

Am I wrong?

Thanks in advance!

RPI Clone seems to be working fine - have it installed on 3 different systems. The script is just a cron call...

30 11 * * 7 /usr/local/sbin/rpi-clone sda -U

The only reason I went with 64 bit is that since I have 4GB of ram thought it might be better with memory allocation / performance / low latency security features etc.. I also like the ubuntu distribution - Node-RED, HomeBridge, PiVPN etc install just fine.. There may be more apps available as well.

1 Like

You could subscribe to wild cards topics in a small groovy program on HE that just spits incoming to the log. However, you won't learn much, Habitat is very reliable with MQTT. As is Mosquitto. You can and should learn about the command line tools mosquitto_pub and mosquitto_sub. Those can be used for all sorts of ad-hoc testing and debugging.

sudo apt install mosquitto-clients

I have a 2GB Pi that runs all my home automation docker images (Home Automation Docker Containers (Node-red, Hubconnect, Homebridge, Apache, Alexa Cookie)). The beauty for me is that if something requires an old version of python, it'll always work, even if I move the image to a newer Pi, and if I change OS's, I can just run the commands to re-build my container and use the data files.

Much like @erktrek I use rpi-clone to keep those Pi's from crapping out. I've had one too many SD cards die. I wrote about this last year if you want more info on backing up your PI or using an external SSD (or thumb drive) on a Pi: Proactively Preventing SD Failure on a Raspberry Pi. I’ve relied on backups several time since and it just works. One of my pi’s lives outdoors in my generator and I’ve been through multiple SD’s but I’ve not had to do any reconfiguring. It’s also a good safety net if you upgrade something and it breaks - just swap cards and restore.

2 Likes

I don’t use Tasmota but am I right that the payloads are nested json, ie a json payload with more json within. If so my MQTT doesn’t currently handle that. If it’s just a basic json payload it will be ok.

Hi Brian,
Thanks for your input. I realize that SD cards are not really designed for multiple read/writes and I certainly agree with your advice to use Rpi clone to back up! In your home automation Rpi, just how frequently do you find that the SD goes bad on you (months between failure?). Have you set up an alert via HE so let you know when this happens or do you just have to deduce the failure if some part of your automation stops working? Again, TIA for your input.

All mine are booting from SSD’s now. They aren’t much more than a SD card. The one that still has an SD card lasts about a year.

1 Like

For now I clone my sdcard to a usb stick and that can be used as a failover. My cron entry actually has 2 entries (I only listed one in my prior comment) - I do a backup to the usb and then the sdcard. The one that is currently active will fail which is okay. If the sdcard fails then the usb will automatically take over on a reboot. If a new sdcard is put in then it will automatically get restored.

I like the ssd option though...mmmm... :thinking:

New-ish bootloader (https://www.tomshardware.com/how-to/boot-raspberry-pi-4-usb step 12) on the 4’s let you say boot from USB first, then SD if they fails. So my cronjob is weekly for SD, and I locally rsync my docker directories. If I yank my SSD out, it’ll boot off SD by default (using regular rpi-clone command). There was once a requirement to boot from SD and that config pointed to SSD, but that’s no longer required. I think I got a 128GB SSD for like $20 and a external case for $7.

1 Like

Yep we are thinking alike except I'm booting from the sdcard first.. probably better to do it the other way.. thanks for the ideas!!

Do you guys use rpi-clone to get your backup image onto an SSD? I've used a spare pi with SD copy in the past but it's a bit of a hassle.

It can handle a direct backup to the SSD via usb including boot. I currently have an SD card and a usb stick of the same size (usually 64 GB). I'm booting off the SD card and clone everything to the usb stick. If the SD Card fails you simply remove the SD card and the usb takes over.

I think @brianwilson does the opposite + uses an SSD which makes more sense since an SSD is more reliable long term.

1 Like

Works both ways. I cloned SD to SSD and weekly go the other way.

2 Likes

Download the Hubitat app