Hubigraphs Alternative

Hubitat package manager streamlines the process of pulling custom apps and drivers from community developersā€™ GitHub repos and downloading to your hub. When it has done its job, you will see the custom app or driver appear in your apps code or drivers code pages of your hub.

You still have to install an instance of the app (now that you have the code available on your hub thanks to HPM) in order to use it.

All of the above is true for Hubigraphs, and other apps/drivers downloaded by HPM.

4 Likes

See I didn't have an issues at all getting it working with Smartthings. That is where my setup originated from and i just used the directions I linked above. They should still apply with only two exceptions. The first being InfluxDb is now in the 2.x versioning and for authorization reasons I think you ned to load 1.8. The second would be you would likely want to specify Hubitat for your database instead of Smartthings.

The big problem is where to put the influxdb and Grafana environments. It needs to be a 24/7 system so it can always get the device updates. That can be as simple as a rasbperry pi for someone that doesn't have a always on system, If you have a 24/7 solution though it could be a VM, or a docker as well. Some systems like Unraid Server also have packages that cover both InfluxDB and Grafana. So it really depends on your home setup.

1 Like

I am going to see about updating codersaurs documentation and will post it here if i can get the minor stuff updated to be valid.

HubitatData Visualization using InfluxDB and Grafana

POSTED ON 2016-04-11 BY CODERSAUR updated Doc for Hubitat by mavrrick

Original Doc at codersaur.com

Overview

There are three key components to the solution:

  • Hubitat Hub: Hubitats home automation controller. Since youā€™re reading this article Iā€™ll to assume youā€™re already reasonably familiar with this platform. To get data out of Hubitat weā€™re going to use a custom SmartApp, more on that later.
  • InfluxDB: A free open-source distributed database for managing IoT time-series data. Itā€™s super-easy to install and natively supports compression, retention policies, and down-sampling.
  • Grafana: A free open-source platform for visualizing time series data. Grafana makes it easy to create dashboards and charts. As well as InfluxDB, it works with several other data sources such as Graphite, Elasticsearch, and OpenTSDB.

Installation

For this solution Iā€™musing Ubuntu Server 22.04 running in a virtual machine,however you could substitute this with a Raspberry Pi running Raspbian, or any other Linux distribution of your choice. If youā€™re not using Ubuntu or another Debian-based distribution you will need to use different commands to those shown below, follow the links to the official installation instructions for more information.

Installing InfluxDB

At the time of writing, the latest version of InfluxDB is v1.8.10, and the official installation instructions can be found here. If you use a different platform please confirm with that link the correct install directions for your platform.

Submit the below commands.

wget https://dl.influxdata.com/influxdb/releases/influxdb_1.8.10_amd64.deb

sudo dpkg -i influxdb_1.8.10_amd64.deb

Start the InfluxDB service:

~$ sudo service influxdb start

To verify InfluxDB is up and running we can run the influx CLI:

~$ influx

Connected to http://localhost:8086 version 1.8.10
InfluxDB shell 1.8.10

We can also browse InfluxDBā€™s web interface on port 8083, *http://localhost:8083/*:

InfluxDB Web UI

Next, we need to create a database for Hubitat and a user account for Grafana. From the InfluxDB CLI, issue the following commands (obviously, replace password with something a little more secure):

CREATE DATABASE "Hubitat"

CREATE USER "grafana" WITH PASSWORD 'password'

Installing Grafana

At the time of writing, the latest version of Grafana is v9.0.0, and the official installation instructions can be found here.

To install the latest OSS release:

sudo apt-get install -y apt-transport-https
sudo apt-get install -y software-properties-common wget
wget -q -O - https://packages.grafana.com/gpg.key | sudo apt-key add -

Add this repository for stable releases:

echo "deb https://packages.grafana.com/oss/deb stable main" | sudo tee -a /etc/apt/sources.list.d/grafana.list

After you add the repository:

sudo apt-get update
sudo apt-get install grafana

Start the Grafana service:

Start the Grafana server with systemd

To start the service and verify that the service has started:

sudo systemctl daemon-reload
sudo systemctl start grafana-server
sudo systemctl status grafana-server

Configure the Grafana server to start at boot:

sudo systemctl enable grafana-server.service

We can verify Grafana is running by browsing to the web interface on port 3000, *http://localhost:3000/*:

Grafana: Log In Page

At this point, Irecommend Logging into Grafana and changing the default admin password.

So far, so good. We now have InfluxDb and Grafana running, but before we can start constructing some charts and dashboards, we need some dataā€¦

Installing the InfluxDB Logger SmartApp

To get data from Hubitat into our InfluxDB database, Iā€™ve written a custom Hubitat SmartApp. Once installed, we can use this SmartApp to subscribe to the devices we want to monitor and it will send data to InfluxDB using its HTTP API. The SmartApp uses the hubAction command,which means it will support the InfluxDB server being on the same privateLAN as our Hubitat Hub, so there is no need to make the logging server accessible from the internet.

Follow the standard procedures to install the InfluxDB Logger SmartApp in the Hubitat Hub. The raw link to the latest code is below

InfluxDB logger on Hubitat Community Github

InfluxDB LoggerSettings

Once weā€™ve added the InfluxDB Logger to the user app section of Hubitat, we must configure it. Enter the host name and port of the InfluxDB server(note that the InfluxDB HTTP API uses port 8086 by default, whereas the web interface uses 8083). Additionally we must specify the database name we created above.

The InfluxDBLogger also allows us to configure a Soft-polling interval. If enabled, this will cause the attributes of all devices to be sent to the InfluxDB database on a periodic basis. I implemented this feature because Hubitat will only generate an event when a device attribute actually changes in value.For attributes that donā€™t change very often, this can lead to big gaps in the data series, which in turn can lead to gaps in our Grafana charts. For now,I recommend setting the soft-polling interval to 5-10 minutes.

Finally, we must configure the devices we want to monitor. When finished, press Done.

Verifying Data Logging

The InfluxDB Logger SmartApp should now be sending events to InfluxDB via HTTP, we can verify this by checking the Hubitat logs and by running a query from the InfluxDB CLI.

From the Hubitat ,navigate to the Logging tab. After a few minutes, we should see some info events from the SmartApp, corresponding to changes of device attributes:

Hubitat: Logging

From the InfluxDB CLI we can query data in the database directly using the InfluxQL querylanguage. Remember to set the database first using the

use

command first.For example, to query switch measurements:

use Hubitat

SELECT * FROM switch

name: switch


time deviceId deviceName groupName unit value valueBinary

1460400903900824163 xxxxxxxx-xxxxx-xxxx-xxxx-xxxxxxxxxxxxEn-suite Towel Rail Bedroom switch off 0

1460401204143438152 xxxxxxxx-xxxxx-xxxx-xxxx-xxxxxxxxxxxx AVPower Lounge switch off 0

1460401204169860754xxxxxxxx-xxxxx-xxxx-xxxx-xxxxxxxxxxxx Computer Power Office switch on 1

We can see above that the InfluxDB Logger maps device attribute names (e.g. switch or power)to a measurement name in InfluxDB. Additionally, deviceId, deviceName and groupName (i.e.room name) are added as tags. Device attribute values are mapped to the value field. In cases where an attribute value can also logically be represented as a binary value, thenan additional field called valueBinary is also included. E.g.for a switch; off = 0 and on = 1. We will see later how these additional tags and fields are useful when building charts and dashboards in Grafana.

If you want to learn more about querying data using the InfluxQL query language,you can find a guide here.

Now we have data in our database we can begin building charts using Grafana. Happy days!

Configuring Grafana

To configure datasources, dashboards, and charts we must use the Grafana web interface on port 3000, http://localhost:3000/.

We will first need to login using the admin password that we configured earlier. Once logged in as admin you may wish to create additional users from the Grafana admin menu.

Data Sources

Next we need to define a data source that corresponds to our InfluxDB database. Click on the gear in the lower left corner of the browser and select Data Sources. Now click on the Add Data source button in the window that came up.

Grafana: Add Data Source

Select InfluxDB from the presented screen and you should be put on the page to configure the Data Source. Provide a valid name for the data source like Hubitat, Update the URL to point to our InfluxDB instance on http://localhost:8086. Scroll down to the bottom of the setup screen and set the database name to "Hubitat" and the username and password to what we set earlier. Save and test the connection.

We can now begin using the Dashboards sectionā€¦

Dashboards

Grafana supports the creation of multiple dashboards. Each dashboard can contain any number of graphs, tables, stats, and text panels.

Create a new dashboard from the top menu, give it a name, then click +Add Row to add a row to the dashboard.

Letā€™s start by adding a graph to our dashboard. From the row menu select Add Panel and then Graph:

Grafana: Add Graph Panel

Graph Panel

We are first presented with the Metrics tab of our new graph. In the bottom-right corner, ensure the data source for the graph is set to Hubitat.We should find a drop-down menu of available measurements when we click on Select measurement. In this example Iā€™ve selected temperature:

Grafana: Graph Query

If we have multiple devices with temperature attributes, we can add the tag deviceName into the GROUP BY clause, which will create an individual data series for each device. Additionally, Iā€™ve specified the deviceName tag in the ALIAS BY clause above so that each series name matches the device name. We end up with a chart looking something like this (obviously we have to wait a while for multiple data points to be logged):

Grafana: Basic Temperature Graph

Grafana supports a wealth of graph settings, most of which should be self-explanatory. We can customise the graphā€™s title, axes, legend, generate aggregate values, and overlay data series from multiple devices and measurement types. For the time being Iā€™ll leave you to experiment.

In future posts Iā€™ll share some of the more-advanced charts and dashboards Iā€™ve created, including use of the templating and annotation features. Weā€™ll also look at configuring data retention policies and down-sampling data in InfluxDB.

7 Likes

The above directions will walk you through the process to install Influxdb and grafana on a computer running ubuntu. You can install it on allot of different platfoorms, but the steps are basically the same.I just did it to validate the changes i made to the original document creatd by codersour. The longest part was installing Ubuntu on a VM The rest maybe took 20-30 mi since it was just running a few commands I hope this helps folks needing a walkthrough to do it.

2 Likes

The latest version of InfluxDB logger has been added to HPM to make it easier to install.

I have also found a fork that was updated to use InfluxDB2.x, but after testing it out i am not sure it is a wise direction to go. It isn't nearly as user friendly to setup initially or use in Grafana.

5 Likes

Thank you very much. I will take a look at it. However, I will be using my Rpi, which will have a different Linux version, and the previous instructions I have tried all were using containers etc. so obviously were more involved (and prone to confusion).

I'll see how far I can get with this :slight_smile:

You know what. Give me a little bit of time this morning. I have been kind of wanting to try to build a image for stuff. let me see if i can do that. and just post it for download.

2 Likes

I have never used any other Influx besides 2.x, the web interface and setup seems good to me. In Grafana if you connect using the 2.x API then yes it is more complicated because you have to write the queries in Flux by hand basically. But you can also make a DBRP mapping and then connect Grafana to the compatibility 1.x API and it is much easier to build the queries using that. I use both in my setup, the Flux allows for more flexibility once you learn it.

I feel like using containers just ads more unnecessary complexity. I just installed everything direct on a VM running DietPi and it works great.

Here is a link to a Raspberry pi image. Like most images you will want to run the commands to expand the image to use your full SD card/storage device once the image is burned. I hope it works.

Raspberry Pi Image for HE

It uses the standard pi password which you probably should change, The influx DB is using a DB name with Hubitat as the database and hubitat/hello123 for the userid/password.

It also has Node-Red if you want to use it. I also loaded docker and installed NGINX and Portainer in case you want to use it.

http://(Pi_IP):81 will take you to Nginx Admin Page (Userid: admin@example.com Password: changeme)
http://(Pi_IP):1880 will take you to Node-Red
http://(Pi_IP):3000 will take you to Grafana log in page (Userid: admin Password: admin This should be changed once logged in)
http://(Pi_IP):9000 Portainer.io log in page (will make you set credentials upon first login)

2 Likes

Honestly using the latest version would be awesome. I have it running, and created a single dashboard. Trying to write queries in flux is a pain though. Do you have an easy write up on how to setup the DBRP mapping.

The thing with sticking with 1.8.10 is that it just works and is fully matured with grafana. The Flux database connector is still in beta. When I make that statment i am thinking about difficulty of entrance into getting this setup for folks.

Fantastic! Highly appreciated. I will check it out as soon as possible and report back :slight_smile:

1 Like

Let me know how it goes. This is my first attempt at building a raspberry Pi image. So :crossed_fingers: it works with no issue. Everything works here on the Pi i built it on so.

This explains the DBRP mapping. You do not need the V1 auth for Grafana, but you would need it to use the Hubitat Influx DB Logger app. If you want to use it with Grafana be sure to give write AND read access. ----> Influxdb2 and HE integration - #10 by jtp10181

To connect to the v1 API in Grafana I am using an Authorization header with the token. If you setup a v1 Auth (with read access) you should be able to use that with the "basic" toggle in Auth section instead of using a token if you want.

See screenshot below: (1) you have to add the custom HTTP Header EXACTLY as explained (case sensitive). I suggest building the Value in a text editor with the "Token" key word and the actual token so you can make sure it is exact, then paste it in.
Header: Authorization
Value: Token TOKEN_WITH_READ_ACCESS

(2) Database: This is the name you gave in the --db parameter when creating the DBRP mapping.

Then you just select the V1 data source in the editor, to get the query builder.

I found a Influx DB logger port that was actually upgrade to use InfluxDBv2 with a token. That is how I tested it. So I don't think that part of the config would be an issue for others to setup, but having to use Flux in Grafana is what I would consider a pain point for many. I will see about setting it up.

Maybe there is something going on between version 9 of grafana and what you have running, but it just won't work for me. Is there something that needs to be done to add a user to access the DBRP. I found the documentation for it and it matches what you said, just wont work on what i setup for some reason.

After this I ended up adding the v1 auth for read access. so i have it now working. It wouldn't take the token in the header for nothing..

The token is tricky. I had a hard time with it the first go around. I think somewhere in a doc I found it shows to use the word TOKEN all caps, but that is wrong, it only works as "Token" in that exact case. So the Value should be set like this:

Token<space>YOUR_TOKEN

UPDATE: I just updated to Grafana 9.0.0 and cleared my saved v1 token header, and reapplied it using my instructions and it works perfectly.

Just curious, How did this go. Is it working for you?

Also if you have a better suggestion for the host name let me know.

@mavrrick58

Thank you so much for this excellent and detailed post. Also, thanks for the fork of the InfluxDB logger.

I'm a total NOOB but in the past week I've managed to:

  1. Revive an old PC (from 2006!) and install Ubuntu. Linux is new to me.
  2. Install Influx DB (actually a couple of times - I started with the latest version but down-rev'ed it after some challenges).
  3. Install Grafana and setup Influx user per instructions.
  4. Install the Influx DB logger and configure it correctly.
  5. Verified that data can be viewed in Grafana.

Now I just need to learn how to configure dashboards and other plots in Grafana.

I actually struggled quite a bit, mostly because my knowledge is so limited. I'll spare you the details.

Thanks again! :smiley:

4 Likes

I am glad it is working so well for you. I can't take creadit for the InfluxDB Logger though. That was done by some other folk