For those of you who can do simple programming under Unix/Linux, you can get quick-and-dirty gnuplot graphs of your Hubitat device event data from the real-time exported JSON records sent out by the Maker API App. [EDIT: A more efficient way to get JSON records is by connecting to the undocumented, unsupported websock event socket. See my posting later in this thread.]
You don't need to install node-red, javascript, or any databases, and you don't need to poll your hub. All you need to do logging is a machine somewhere that is running a web server, that lets you run CGI scripts, and lets you write to a file.
The Maker API App lets you set a URL to POST device events to. That means that for every device you select under Select Devices, every event those devices produce will be sent in JSON format to that URL in real time. That URL can be anywhere in the world.
Create a CGI script on the web server at that URL. Make the CGI script simply append the posted JSON records to an output file on the machine. No polling is needed, assuming the devices you are logging generate their own events periodically.
Since the JSON event logs produced by Maker API are missing a time/date stamp, I had my CGI shell script add one at the beginning using the sed program:
#!/bin/sh -u
# My URL for this script is: http://192.168.1.1/~idallen/makerapi.cgi
echo "Content-Type: text/plain"
echo ""
# Change the output file below to be where you store all your event data.
# Use $(date +%s --date='now UTC') if you want UTC time stamps (see `gnuplot` below).
exec sed -e "s/\"name\"/\"date\":\"$(date +%s)\",&/" -e "s/\$/\n/" \
>>/idallen/makerapi.json 2>&1
As soon as you create the above script, make it executable, and put the URL for it into the Maker API App configuration, you'll start seeing JSON event data being appended to the file in real time for the devices you selected in the Maker API App configuration.
If you don't see anything, make sure your device is configured to generate its own events fairly frequently, and check your web server logs and web error logs for script and file permission errors. Make sure your web server is configured to permit CGI scripts in the directory you are using.
If you already have a database or existing graph infrastructure (e.g. Grafana), you could make the CGI script add the event data to the database directly, instead of putting it in a file. Perl has database tie-in functions; you could also use your favourite CGI language to do the same. I won't cover that here.
Here is a sample line from my generated .json output file (note the leading date field added by my CGI script above):
{"content":{"date":"1653841789","name":"power","value":"426.974","displayName":"Aeotec Energy Meter Gen5","deviceId":"1","descriptionText":null,"unit":"W","type":null,"data":null}}
Make a note to roll and empty this log file occasionally, or else you'll fill up your file system with device events.
With the JSON data being logged into a file in real time, all you need to do is pull the values you want to graph out of the file.
You could use the command line tool jq to convert the JSON records to CSV for input to your favourite CSV graphing program:
$ jq -r '."content" | [.date, .name, .value] | @csv' \
< /idallen/makerapi.json >data.csv
I wanted to combine three JSON power entries onto one line for input to gnuplot, so I wrote a small Perl program to read the JSON records and output one line with the three power readings on the one line:
#!/usr/bin/perl -w
# Read the makerapi.json file and consolidate multiple power
# reports onto one line based on leading epoch seconds timestamp.
use strict;
use warnings;
use JSON;
my @powers = qw( power power-Clamp-1 power-Clamp-2 );
my %triple = ();
my $prevdate = 0;
while(<>) {
next unless /^[{]/;
my $objr = decode_json($_);
die unless $objr;
my $cont = ${$objr}{'content'};
next unless defined($cont);
die unless ref($cont);
my $dn = ${$cont}{'displayName'};
# only look at desired device
next unless $dn eq 'Aeotec Energy Meter Gen5';
my $name = ${$cont}{'name'};
# only look at desired power fields
next if ! grep(/^$name$/,@powers);
my $date = ${$cont}{'date'};
# if more than a second apart, start over
if ( ($date-$prevdate) > 1 ) {
%triple = ();
}
my $value = ${$cont}{'value'};
$triple{$name} = $value;
my $size = keys(%triple);
# When we have all three keys, we can output them on one line:
if ( $size == 3 ) {
print "$date $triple{'power'} $triple{'power-Clamp-1'} $triple{'power-Clamp-2'}\n";
%triple = ();
}
$prevdate = $date;
}
The above Perl script produces one line with three power levels on it:
$ ./i.pl /idallen/makerapi.json | head -n 4
1653841789 426.974 381.851 45.123
1653841794 427.62 382.428 45.192
1653841799 428.802 383.64 45.162
1653841804 430.294 385.181 45.113
That output can be put into a file and fed into gnuplot:
$ ./i.pl /idallen/makerapi.json >mydata
$ gnuplot -persist -e "
set xdata time ; set timefmt '%s' ; set format x \"%Y
%m/%d
%H:%M\" ;
set grid ;
set title 'Power consumption' ;
plot 'mydata' using 1:2 t 'power',
'mydata' using 1:3 t 'clamp1',
'mydata' using 1:4 t 'clamp2' ;
"
The X-axis graph times will appear in UTC unless you apply a time correction for your local time zone. One way to fix this is to adjust UTC to EDT by subtracting four hours in the gnuplot script itself:
[...]
plot 'mydata' using (column(1)-4*60*60):2 t 'power',
'mydata' using (column(1)-4*60*60):3 t 'clamp1',
'mydata' using (column(1)-4*60*60):4 t 'clamp2' ;
A better way to fix the UTC problem for gnuplot is to write UTC times in the CGI script. In the CGI script, change the command substitution $(date +%s) to $(date +%s --date='now UTC') and then you don't need to do the subtraction in the gnuplot script, nor do you need to keep track of Daylight Savings Time. This time adjustment may or may not be good for other plotting programs.
With appropriate scripting or gnuplot plot limits, you can limit the data plots to date ranges:
$ start=$( date '+%s' --date="May 29 3pm UTC" )
$ end=$( date '+%s' --date="May 29 5pm UTC" )
$ gnuplot -persist -e "
[...]
plot [$start:$end] 'mydata' using (column(1)-4*60*60):2 t 'power',
[...]
Suggestion for Maker API: add a time/date stamp to the JSON records.
