Hello,
I am looking for a way to store all logs for mode changes, lights on/off, etc to an external logging source. I have searched for hours.
What are my options here? I can't find any guides or definitive answers on the best way to do this?
If I need to post this somewhere else, let me know!
Thanks!
There's a WS stream called Log Stream
ws://<yourIP>/logsocket
If you have a handy WS client... That will demonstrate the log stream but you'll need to find a way to capture that and store it. There have been a few Community based offerings for that, but it so completely depends on what you have available as an always-on-computer to capture and store.
I guess I’m just the opposite of you. I have all logging turned off that I can find. Then when there is an issue, I can just turn on the logging I am needing to troubleshoot the issue. I don’t see any value of what happened yesterday, or last week. What is your reason for this much information?
If I throw my Zwave door locks into Hubitat, I'd like to see when the door opened, and which specific code was entered. This will tell me who unlocked the door and when.
I have a datacenter room built in my house and it would be nice to log all the information based on temps as well.
That's a few reasons.
Thanks! Any good offerings you could recommend?
If that's all you need (or things along that line), Hubitat's built-in event history might work. On the device page, look at the "Events" button. You can change the defaults or per-device settings, but I think the defaults save the last 100 events per attribute (e.g., lock: locked or unlocked) or the last year, whichever purges sooner. For anything reasonably recent and events that don't happen super-often, this may be enough.
It sounds like you might want something more long-term, but since I didn't see this mentioned--it is distinct from logs, which conventionally (but not necessarily, especially for the various community drivers) log similar information plus some debugging information if enabled--I just wanted to make sure you were aware.
This might work for now.
I'm hoping for something more long term. I have a few Elasticsearch stacks I could send this too and handle modifying the data with a Grok filter. Even Influx+Grafana might work, but I'm not as good with that.
Community options are fine. I did see there was something with IFTTT? And WebCore?
I'm using this in combination with Node-RED and mySQL to store the logs. I keep 4 weeks of data in the database and every week a Node-RED flow exports any data older than 4 weeks to a file and then deletes that from the database.
I also keep all device events (different than logs) also via Node-RED. I filter out certain event types (typically ones that are very frequent - lux reading, motion activity) from the events that are stored. Below is a link if you are interested.
Hope this helps - feel free to reach out if you want additional details.
I would LOVE additional details on this!
@bdp are you currently using or looking to use Node-RED? The solution that I use, requires Node-RED and a SQL database (I use mySQL). My set up is:
The Node-RED sequences for logging:
The websocket node listens for data from HE on ws:///logsocket, converts it to a SQL insert statement and stores it in the database. I also check if the log message type is anything other than info, warn or debug and send an alert .
Websocket node config:
Function node config (to create SQL statement):
I use the Node-RED dashboard to show the data and also allow selection by date range, log type and device:
If you (or anyone else) is interested in the flows, I can post them here - Ineed to do some clean up before I can do that.
@rakeshg Hi, looking to implement Node-RED. I was going to spin up a new VM in my lab and use that.
I'm a bit lost with Node-RED, I hadn't even heard of it before googling around for storing my logs!
Your log setup is exactly something I am looking at.
Great! Let me know when you have NR and a SQL database set up and I can put up the flows and walk you through the logic/process.
@rakeshg
Node-RED installed and up with mySQL up and the SQL script ran.
How do I enable the websocket?
Which SQL script did you run? If it was from the original post, I don't think it creates a "Logs" table. Can you confirm? If it did not create a "logs" table, can you create a table with the following columns?
I don't think you need to do anything special to enable websocket (for logs). On the Node-RED side, try importing this flow:
[{"id":"7a12636a.8b7024","type":"comment","z":"7b994e31.f24a6","name":"Hubitat Log","info":"2021-05-30: Created","x":174.3332977294922,"y":2063.999755859375,"wires":[]},{"id":"36966e59.07b402","type":"websocket in","z":"7b994e31.f24a6","name":"Hubitat Log Socket","server":"96044230.8127a","client":"","x":197.6666259765625,"y":2119.333251953125,"wires":[["57aaa540.2a5b84","9fdfa8df.0fde48"]]},{"id":"57aaa540.2a5b84","type":"function","z":"7b994e31.f24a6","name":"Log SQL","func":"//Fix for SQL insert errors on HPM name\nif(msg.id == 133){\n msg.name = 'Hubitat Package Manager'\n}\n\nvar sqlString = 'INSERT INTO LOGS (NAME,MSG,ID,TIME,TYPE, LEVEL)VALUE ('\nvar quote = \"'\"\nvar comma = \",\"\nsqlString +=quote+msg.name+quote+comma\nsqlString +=quote+msg.msg+quote+comma\nsqlString +=msg.id+comma\nsqlString +=quote+msg.time+quote+comma\nsqlString +=quote+msg.type+quote+comma\nsqlString +=quote+msg.level+quote+\")\"\n\n//sqlString = sqlString+quote+msg.name+quote+comma+quote+msg.msg+quote+\",\"+msg.id+\",\"+msg.time+\",\"+msg.type+\",\"+msg.level+\")\"\nmsg.topic = sqlString\nreturn msg;\n","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":431.99993896484375,"y":2119.333381652832,"wires":[["c0a6c5c4.10901"]]},{"id":"9fdfa8df.0fde48","type":"switch","z":"7b994e31.f24a6","name":"Not Info, Debug or Warn","property":"level in [\"info\",\"debug\",\"warn\"]","propertyType":"jsonata","rules":[{"t":"false"}],"checkall":"true","repair":false,"outputs":1,"x":482.33331298828125,"y":2181,"wires":[["97cb985e.2d933"]]},{"id":"c0a6c5c4.10901","type":"mysql","z":"7b994e31.f24a6","mydb":"2d73f9f7.c404d6","name":"Create Log DB Entry","x":972.6666564941406,"y":2119.66650390625,"wires":[[]]},{"id":"96044230.8127a","type":"websocket-listener","path":"ws://192.168.1.113/logsocket","wholemsg":"true"},{"id":"2d73f9f7.c404d6","type":"MySQLdatabase","name":"Hubitat_Event_Prod","host":"192.168.1.86","port":"3306","db":"hubitat_logging","tz":"","charset":""}]
It should create the websocket config (you will need to modify the IP address to that of your HE hub) as well the the mySQL database config (again, you will need to modify to your setup).
If the websocket config does not come over, the format is:
The flow above is through the node that filters out 'warn, info, debug" - I have not included the nodes that create the email/SMS since I am not sure what you will be using (I use the gmail node in NR).
Feel free to PM me if you have any questions. Hope this helps
Hmmm, yeah, even simply having the Hub relay the normal Logs over to a syslog server would be a pretty handy thing. Real easy to manipulate logs that way (grep, awk, etc).
Just checking if you were able to import the flow and if it worked? There are a couple of other components to this process (restart Node-RED flows if HE restarts, export and delete old logs, dashboard to show the logs) but I wanted to see if you were able to get the basic data to flow from HE to mySQL.
Download the Hubitat app
