Hubitat > NodeRed > MySQL > Grafana (LONG READ)

Hi @corerootedxb - I'm new to node red, but I've been logging to influx using the SmartThings logger (initially) and then the port for Hubitat, for 3 years now.

To my surprise, I got this working right away on a new influx db!
Screen Shot 2020-10-03 at 5.37.30 PM

But when I try and use it with my existing influx database (with the 3 years of data), I'm running into some data type inconsistencies.

Ex. Temp was stored as a float, but your flow wants it to be a string

Error: A 400 Bad Request error occurred: {"error":"partial write: field type conflict: input field \"value\" on measurement \"temperature\" is type string, already exists as type float dropped=1"}

and

Error: A 400 Bad Request error occurred: {"error":"partial write: field type conflict: input field \"valueBinary\" on measurement \"thermostatOperatingState\" is type float, already exists as type integer dropped=1"}

Like I said, new to node red, but it looks like the Set Influx Data node is where any conversion happens.

If i'm reading right, Line 171 already has a catch all to push attributes like temperature in as a Number

// Catch any other general numerical event (carbonDioxide, power, energy, humidity, level, temperature, ultravioletIndex, voltage, etc).
else {
    v.value = Number(msg.value);
    v.isBinary = false;
}

With this in place, any idea why I'm getting the error above saying temperature is a string?

edit: scratch the above request - learning that there are a lot of nuances to the device data. Will try and overcome in code. Thanks again for the primer.

Hi, im new in SQL so i need your help.

In Node-Red i become this error:

3.1.2021, 22:04:32node: c3c2bc48.01bc5
INSERT INTO events(source,name,displayName,value,unit,deviceId,hubId,locationId,installedAppId,descriptionText) VALUES('DEVICE','presence','Keller - Server - Tür ','not present','null',2,0,undefined,0,'null') : msg : Object
object
source: "DEVICE"
name: "presence"
displayName: "Keller - Server - Tür "
value: "not present"
unit: "null"
deviceId: 2
hubId: 0
installedAppId: 0
descriptionText: "null"
_session: object
_msgid: "17a42e0c.079432"

topic: "INSERT INTO events(source,name,displayName,value,unit,deviceId,hubId,locationId,installedAppId,descriptionText) VALUES('DEVICE','presence','Keller - Server - Tür ','not present','null',2,0,undefined,0,'null')"

error: object

message: "Error: ER_BAD_FIELD_ERROR: Unknown column 'undefined' in 'field list'"

source: object

stack: "Error: ER_BAD_FIELD_ERROR: Unknown column 'undefined' in 'field list'↵ at Query.Sequence._packetToError (/home/user/.node-red/node_modules/mysql/lib/protocol/sequences/Sequence.js:47:14)↵ at Query.ErrorPacket (/home/user/.node-red/node_modules/mysql/lib/protocol/sequences/Query.js:79:18)↵ at Protocol._parsePacket (/home/user/.node-red/node_modules/mysql/lib/protocol/Protocol.js:291:23)↵ at Parser._parsePacket (/home/user/.node-red/node_modules/mysql/lib/protocol/Parser.js:433:10)↵ at Parser.write (/home/user/.node-red/node_modules/mysql/lib/protocol/Parser.js:43:10)↵ at Protocol.write (/home/user/.node-red/node_modules/mysql/lib/protocol/Protocol.js:38:16)↵ at Socket. (/home/user/.node-red/node_modules/mysql/lib/Connection.js:88:28)↵ at Socket. (/home/user/.node-red/node_modules/mysql/lib/Connection.js:526:10)↵ at Socket.emit (events.js:314:20)↵ at addChunk (_stream_readable.js:297:12)↵ --------------------↵ at Protocol._enqueue..."

payload: "HE Logging has thrown an error"

my Create Events insert:

var query = "INSERT INTO events(source,name,displayName,value,unit,deviceId,hubId,locationId,installedAppId,descriptionText) VALUES(";
//var name = msg.name.replace(/'/g, "'");
//var msg = msg.msg.replace(/'/g, "'");

var displayName = msg.displayName === null ? 'null' : msg.displayName.replace(/'/g, "\'");
var descriptionText = msg.descriptionText === null ? 'null' : "'" + msg.descriptionText.replace(/'/g, "\'") + "'";

query += "'" + msg.source + "',";
query += "'" + msg.name + "',";
query += "'" + msg.displayName + "',";
query += "'" + msg.value + "',";
query += "'" + msg.unit + "',";
query += msg.deviceId === null ? 'null,' : msg.deviceId + ',';
query += msg.hubId === null ? 'null,' : msg.hubId + ',';
query += msg.locationId === null ? 'null,' : msg.locationId + ',';
query += msg.installedAppId === null ? 'null,' : msg.installedAppId + ',';
query += msg.descriptionText === null ? 'null)' : "'" + msg.descriptionText + "')";

msg.topic = query;

return msg;

Does anyone have a solution? Big thx

Has anyone created a flow that uses Graphite as the database to store events in instead of MySQL/MariaDB or InfluxDB? I'm asking because the Grafana Cloud service only supports Graphite and Prometheus metrics formats. I've found the Graphite node-red-contrib-graphite output node. Does anyone have any experience with this node?

I've successfully got Grafana Cloud working with metrics exported from Netdata on my RPi 3 B. I use Netdata to monitor the RPi itself and some network devices over snmp.

Hey guys trying to get to grips with node-red as a new user.... As such I am trying to save data to an InfluxDB.

I have an existing database reading sensors which I populate from a python script and hence I understand that the database is very fussy around how data is passed to it so suspect I am not quite understanding something correctly.

Essentially I have a bespoke device handler within Hubitat that I use to pull data from an external vendor using a simple api handler. Eventually I would like to pass this data to the database for saving.

However in an effort to baby step my final intention I am trying to save the status of a simple virtual switch using the device node. I have applied the set influx data functio provided by @dan.t (thanks) but I get the following error:

"TypeError: Cannot read property 'match' of undefined"

looking at the payload from the switch:

{"name":"switch","value":"on","displayName":"Node Test Switch","deviceId":"261","descriptionText":"Node Test Switch was turned on","unit":null,"type":null,"data":null,"currentValue":"on","dataType":"ENUM","values":["on","off"]}

I think the function should parse it correctly but getting stuck with the error.

Below is the simple flow. Can anyone point me in the right direction

You'll need to provide the code in the function node (set Influx data), and also what format you want the data to be in for the next Node.

Thanks for putting this together! I have my setup as described in the first post, however, none of the values are being assigned. In the debug tab I get the message below for every message that comes through the WS. I don't think that I have made any typo's but I pasted the function below just incase my eyes aren't catching something. Any help with this is much appreciated.

4/29/2021, 9:27:33 AM[node: 30458c1a.88e6a4](http://10.0.0.195:1880/#)INSERT INTO logs(`name`,`msg`,`id`,`time`,`type`,`level`) VALUES('undefined','undefined',undefined,'undefined','undefined','undefined') : msg.payload : string[160]

"{"name":"Zooz Double Plug-RIGHT Outlet","msg":"Zooz Double Plug-RIGHT Outlet power is 0W","id":357,"time":"2021-04-29 09:27:33.813","type":"dev","level":"info"}"
var query = "INSERT INTO logs(`name`,`msg`,`id`,`time`,`type`,`level`) VALUES(";
//var name = msg.name.replace(/'/g, "\'");
//var msg = msg.msg.replace(/'/g, "\'");

query += "'" + msg.name + "',";
query += "'" + msg.msg + "',";
query += msg.id + ',';
query += "'" + msg.time + "',";
query += "'" + msg.type + "',";
query += "'" + msg.level + "')";

msg.topic = query;

return msg;

It looks like the "double quotes" on the return msg.payload may be throwing things off? If I put the msg.payload in your post through a JSON parser, this is the error message.

However, if I remove the opening and closing "double quotes", it parses it correctly.

Thanks for the inputs! I am new to Node-Red so forgive the questions. I am not sure where the extra quotes are coming from in the function. In the debug message, I thought that the bottom line was the output of what was coming from hubitat.

I think what is happening is that msg.payload is coming out as a string (for the entire JSON object) and because of that it is enclosed in quotes.

Screen Shot 2021-04-29 at 10.42.11 AM

If you put a debug node right after the Hubitat Logs WS node and look at msg.payload in the debug panel, I think it will show you the attribute you are getting is msg.payload (as a JSON string) and not msg.name, msg.msg etc.

I don't use websocket data (I use the Hubitat node) so not sure why the attributes are being sent as msg.payload (string) vs. msg.name, msg, msg etc.

I just created a test flow and confirmed that what is coming from the WS listener node is a string and not a parsed object. What you need to do is modify the configuration node that the WS listener is subscribed to and change the "send/receive" selection from "payload" to "entire message"

The output will then be a correctly formatted JSON string. Hope this helps.

Awesome! That was it. Thanks for your help! I've been racking my brain over this for two days. :smiley:

1 Like

when i try to import I get this error msg:
SyntaxError: JSON.parse: expected ',' or '}' after property value in object at line 188 column 18 of the JSON data

I think dan's code is built to load from a websocket query. After reading around for a couple of hours I decided to go down this same path as you. Maker API > NodeRed > InfluxDB. Seemed to make the most sense to just use NodeRed as the middle man between Hubitat and Influx instead of a separate dedicated node.js module running. This is pretty simple but painfully difficult to implement due to all the different ways to do this and the only clear instructions are using very old and outdated methods.

I know how to code and I am working on fixing it myself right now, will edit this with a fix when I figure it out.

EDIT: It appears I have it working, due to there being so many different options I am going to have to write it up on a separate post I think. I will put a link here when I have it up.

1 Like

@stephack and @btk

I’m trying to get these flows working with influxdb 2.0. I have everything working except for reinjecting the battery/temp/humidity data. I have not been able to figure out flux well enough to query the database for the correct values. Do either of you have any insight on how to do this?

thanks

replying to myself here. I figured out he flux query I think:

msg.query  = 'from(bucket: "hubitat")';
msg.query +=  '  |> range(start: -12h)';
msg.query +=  '  |> filter(fn: (r) => r["_measurement"] == "' + msg.measurement + '")';
msg.query +=  '  |> filter(fn: (r) => r["_field"] == "value")';
msg.query +=  '  |> last()';
msg.query +=  '  |> yield(name: "value")';

return msg;

with @stephack’s flow:

f/u question though. I have 4 hubs dumping data into my influxDB database. they are tagged with different hub IDs. how would I edit this node to maintain my hubID data when reinjecting the info?

thanks

Is this solution still actual?

what solution are you wondering about? as far as flux 2.0 goes, I never got it fully working.

I am so happy I found this post. Until now I manage to save my thermostats data to influxDB using eventsocket as source -> node-red -> influxDb. Tomorrow I will add Grafana
Thanks for effort :slight_smile:

Hello,
I am new to this thread and I'm getting an error when I try to import the flow:

The error that it's mentioning is in there twice but I don't know what to do about it. I got this from copy/paste and upload of the file. I did it twice each way to be sure and I received the same error on importing the flow each time.

I get the same error. You may just need to wait until @corerootedxb posts it again.

1 Like