Raspberry PI companion for HE

Hi everyone !

I know that it's easy to one make suggestions when the person itself does not know how to implement it, but nonetheless I'll make this one.

When we at HE community have tools like Hubitat Package Manager (HPM) available, it makes everybodys life easier - I think there's no question about that.

However, there are some solutions like local WebCore server, Alexa TTS, VPN, local e-mail client (some that I can think of now, and certainly there are much more), that requires a Raspberry PI to run. I have, myself, plans to develop some apps that would require a web server and a Raspberry PI is the logical place to put it.

Talking about the obvious, using a Raspberry PI to host some solutions locally is on par with one of the main advantages of HE: local processing!

That being said, what I'd like to suggest is:

a) creation/definition of a basic Raspberry PI software configuration (name it RPI4HE maybe?)

b) for each app/package/solution that a developer wants to add to such basic configuration, the developer woud create a script where all software components necessary for the developer app/package/solution to work would be installed, configured, etc

c) if any app/package/solution needs user specific files (e.g. configuration options), the installation script could ask the user during its run where to find such files (could be at GitHub or - even better: local files on HE !)

d) copying the brilliant idea used at the implementation of Hubitat Package Manager (HPM), the developer should register its installation script in a centralized repository, from where it would be easy for an user to select the apps/packages/solutions that the user wants to install on the user RPI

e) at the RPI, a HPM like script would handle the selection and execution of the developer scripts desired by the user; this same RPI script could, also, handle usual maintenance tasks, such as making RPI backups, updating/removing apps/packages/solutions, updating RPI software packages and so on

Please note that the ideas above are here for the sole purpose of testing the interest of the community in such solution. Of course new suggestions are expected and would be very welcome - after all, it would be a community project.

For me, someone that barely can use Unix vi to edit a file this solution would be a blessing - and I'm sure that for a lot of users on our community it would be too.

Trying to stir the community interest, imagine a RPI solution that, at scheduled times, makes a local backup of HE, encrypt it and, using an e-mail client, send it to your e-mail. Wouldn't it be nice?

Again, it is easy to propose ideas where other people would do the hard work. However, my very limited knowledge of the Unix enviroment - being honest, kind of almost none ... - it would be impossible to do it myself.

Well, I hope this proposal gets some attention from the community.

Thanks for the attention.

Best regards,

Marco

8 Likes

would be cool to use a RPi as a hub link type device to utilize the compute power. when running the alexa speaks on HE, it lags the hub so much

Hi @dadarkgtprince !

That's why I initially called this proposed solution as a "companion" for HE. Using RPi we could increase processing power and memory as needed - all we would need is change the Pi model (at least I think ...).

1 Like

Great idea.
I used the image from @csteele as my PI base to achieve a version what you are talking about.
Specifically making it easy to setup Node-Red and HomeBridge with HE. Raspberry PI Image for Homebridge + Node-Red

2 Likes

When I initially created that Image, I was planning on including the HubConnect NodeJS proxy Server, since it would be 'free' but ultimately I decided it was too niche, and there's be questions about "Is THAT causing a problem with..." :slight_smile:

Then I thought, Oh, I'll put DNS and an NTP server on it too.. same too niche result. The thing is, installing bind (dns) and ntp is trivial, it's the setup that requires significant effort.

But I do agree, it's as good a starting point as any other. :smiley:

1 Like

@SoundersDude, thanks for pointing to @csteele image!

@csteele that's what I was looking for. If you agree, we could talk a little (or a lot !) about what you have and what could be done.

We could discuss on how to address questions like this:

Or these:

What do you think?

Marco

1 Like

I am using an RPi as a "companion" server as well. I have Node-RED, MQTT (currently NR-MQTT) and watchguard VPN running on it. It also does a backup of HE every so often. The issue with an RPi is the stability of the storage. The sdcards may be problematic but there are workarounds like using an external flash drive. I am using an SD card but running an image backup script to a flash drive. An alternative system is the oDroid.

The other thing to consider is using a UPS for both HE and the companion server. TalentCell has a good "mini" ups that I've been using for 2 different installations.

3 Likes

That’s one of the things that I love in this community: ideas & contributions, and a lot of them!

Even if we use a SD card from a good brand as SanDisk, Lexar or Sony?

This could be an option, one of the "features" to activate/install.

And this another one!

Let's keep seeing what the community bring us.

Thanks!

@csteele,

Which packages do you use to install DNS and NTP? Do you use apt-get?

I am using Samsung SD cards and they have been reliable. I am using RPI-Clone to clone to a flash drive..

For the HE backup something like this (taken from a simple script I wrote). The ${} are variables I initialize via parameters (I have multiple hubs in use currently). Use it in Cron scheduler

wget http://${HUBIP}/hub/backupDB?fileName=latest -O `date +${BAKDIR}/${HUBNAME}.%Y%m%d.%H%M.lzf`
2 Likes

Yes, apt-get, of course.... :slight_smile:

sudo apt-get install -y ntp ntpdate bind9 bind9utils dnsutils

As I said, install is trivial... it's building a viable config that is 10x the work...

sudo systemctl stop systemd-timesyncd
sudo systemctl disable systemd-timesyncd
sudo /etc/init.d/ntp stop
sudo /etc/init.d/ntp start

To test:

sudo service ntp restart
ntpq -pn
ntpdate -d {ip of raspberry}

NTP will just come up and work, it's not picking the best source servers of time, but it's good enough to test and then start modifying into a viable config.

sudo nano /etc/ntp.conf

and start adjusting :smiley:

But DNS (bind9) is a different story. You try this initially, you'll get plenty of errors:

sudo service bind9 start

To get a SAMPLE DNS working, there are 4 files to edit:

BIND9 Sample Configs

sudo nano /etc/bind/named.conf.local

zone "home.lan" IN {
type master;
file "/etc/bind/db.home.lan";
};
zone "1.168.192.in-addr.arpa" {
type master;
file "/etc/bind/db.1.168.192.in-addr.arpa";
};

sudo nano /etc/bind/db.home.lan

home.lan. IN SOA raspberry.home.lan. hostmaster.home.lan. (
2017081401 ; serial
8H ; refresh
4H ; retry
4W ; expire
1D ; minimum
)
home.lan. IN NS raspberry.home.lan.
home.lan. IN MX 10 raspberry.home.lan.
localhost IN A 127.0.0.1
raspberry IN A 192.168.1.31
router IN A 192.168.1.1

sudo nano /etc/bind/db.rev.1.168.192.in-addr.arpa

@ IN SOA raspberry.home.lan. hostmaster.home.lan. (
2017081401 ; serial
8H ; refresh
4H ; retry
4W ; expire
1D ; minimum
)
IN NS raspberry.home.lan.
1 IN PTR router.home.lan.
31 IN PTR raspberry.home.lan.

sudo nano /etc/bind/named.conf.options

forwarders {
8.8.8.8;
};

sudo service bind9 restart

To test:
$ dig raspberry.home.lan @{your Raspberry IP}

and expect:

; <<>> DiG 9.8.2rc1-RedHat-9.8.2-0.62.rc1.el6_9.4 <<>> raspberry.home.lan @192.168.7.34
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 22898
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 1, ADDITIONAL: 0

;; QUESTION SECTION:
;raspberry.home.lan. IN A

;; ANSWER SECTION:
raspberry.home.lan. 86400 IN A 192.168.1.31

;; AUTHORITY SECTION:
home.lan. 86400 IN NS raspberry.home.lan.

;; Query time: 4 msec
;; SERVER: 192.168.7.34#53(192.168.7.34)
;; WHEN: Wed Oct 7 16:54:28 2020
;; MSG SIZE rcvd: 66

1 Like

Sweet - I've been looking for a way to back up my headless pi without having to yank the SD out!

1 Like

If you just want a straightforward backup as an image, something as simple as this works:

dd if=/dev/mmcblk1 bs=1M | gzip | dd of=disk.img

2 Likes

Thanks - When I was researching this some time ago I seem to recall reading that doing a dd on a 'live' sd card wasn't recommended (if you had a desktop installed on the pi and can use the built in backup utility this is a different story) and that you should do it on a separate linux pc.

(Had a look at the github script and it uses dd also (guess there aren't any other options))

So I don't have a Pi - I prefer SBCs with eMMCs. But I have been using dd to backup a live eMMC for several years. And have restored the image on an unformatted eMMC several times with no issue.

Typically I will do an fdisk before creating an image

Edit: Linux supports loopback mounts - so on occasion I have loopback mounted an image before restoring it .....

1 Like

I'll run a test on a non production pi first then :wink: !

1 Like

My 2c worth - if you are maintaining your network and devices using a commercial UPS then NUTS is an essential setup on a companion pi.

Don't forget MQTT, CATT, zigbee2MQTT, Deconz too!

1 Like

I must admit I was sceptical at first with the proposal of a pre-defined setup for a raspberry pi. I tend to think something like a rpi is used in so many different ways, even if you just focused on those who use it as an add-on to HE, that a standard setup is probably a hard thing to achieve. I probably still have that same view, but proposing this idea has certainly lead to a healthy conversation, so credit to @maffpt for raising an interesting topic and to those who have contributed to the conversation.

In terms of my contribution....

While I have used Unix / Linux in a limited capacity in my professional career, I would put myself in the category of someone who is not completely at home in that environment, more so in a Windows PC world. That said, one of the main reasons I purchased my rpi was the fact a number of likely solutions for my home automation were going to include instructions involving a rpi, whether they be here on the community or more generally on the Internet. In terms of this thread, I think the major benefit that can be gained is from people fleshing out the details of their setup on a rpi for specific solutions, more so than a pre-defined technical setup. Just my opinion.... Certainly there could be benefits for specific applications / solutions, but I feel a broad "companion" setup may be too broad...

Again, great topic for discussion and great contributions by all, would love to see it continue.

Simon

I guess it is how you look at it. Like you mentioned an RPi can be used for many things on a local network not necessarily related to HE.

Some HE Companion Ideas:

  • Offload Rules/Apps for reduced HE overhead using systems like Node-RED.
  • Separate UI / Visual for alternate rule apps like WebCoRE.
  • External Logging Server.
  • External apps that work in conjunction with HE apps to provide missing functionality or connectivity like HubConnect [server], 3rd party dashboard apps, email notifications etc.

Tangentially related stuff that benefits a companion server (and the lan as well):

  • MQTT
  • NTP Server
  • VPN
3 Likes

I totally agree, and I didn't mention in my earlier post, but I did contemplate posting some of my thoughts early on, and am glad I didn't cloud the early conversation with my boundless pessimism :slight_smile:

My questioning of the proposal is really a pedantic one. I whole heartedly support making it easier for people to make use of the power you get from a rpi, I've even posted myself along similar lines. Coming from an IT project background, I guess my only concern is scope, which is a really boring an uninspiring one to bring up during the fostering of an idea. I think if we want to harness the collective efforts of those who use a rpi, then knowledge sharing is probably the most effective outcome, again IMHO....