Scrolling through tens of thousands of events one screen at a time

They don't make the list. This falls squarely within the domain of a custom app or on an outside system.

5 Likes

And therein lies one of the great advantages of a system like Hubitat.

For edge use cases, anyone that’s sufficiently motivated can do as they please.

4 Likes

@idallen

This is the approach that most people who need that kind of data have taken. And it is pretty easy to implement using the websocket endpoints that I indicated. And while those are unofficial, staff have indicated that they are not likely to go away.

You could even do it as a little NodeJS app running on a Mac/Windows computer or an RPi.

3 Likes

The Node-Red flow above actually uses the log websocket to capture the data.

3 Likes

Using the Maker API URL I don't need to install any other packages, not NodeJS or NodeRED or anything. I'll read up on these websocket things, but even a little beta-version command-line program to dump the websocket feed is 495 lines of Java: Releases · ianparkinson/helog · GitHub My CGI script to save the JSON record from Maker API is three lines of shell script, one of which defines a variable and one is echo. Even the C program I wrote to replace the script under xinetd is only 72 lines. I'd love a much more efficient interface, but do I have to use 495 lines of Java to get it?

Unofficial is a good reason not to spend a lot of time messing with them. If I can find a small command-line program that connects and dumps the unsupported feed, I'll consider switching away from the supported URL I'm using in Maker API.

Honestly i don't see your point. Just because you dump something doesn't make it usable. Otherwise why ask the questiin at the beginning of this thread. The procedure I provided gets everything that hits the live logging first not device events. Then translates that to a good format for a database that is easily queried. Then you can use a software like grafana to visualize it.

The most important part is that it really is usable.

You will find huge differences between simple event logs and the live logging.

If what you want is event logging only you can also use the InfluxDB Logger smartapp to load all your events for selected devices into InfluxDB.

I have a feeling a interesting question could be what is you expectation with this. This seems more like it is just because you can not because of any real use case. I know why i have these working, but the reality is i dont use it to frequently. Perhaps a old quote from Jurassic parks makese sense here:

"You spent so much time on if you could, you did't stop to think if you should".

2 Likes

I don't see any event time stamps on the raw output from helog. Am I supposed to do something to turn on time stamps? Why do you see them, but I do not?

This is what Maker API sends for an event:

{"content":{"name":"current","value":"4.686","displayName":"Aeotec Energy Meter Gen5","deviceId":"1","descriptionText":null,"unit":"A","type":null,"data":null}}

This is what helog dumps from the websocket for the same event:

{ "source":"DEVICE","name":"current","displayName" : "Aeotec Energy Meter Gen5", "value" : "4.686", "type" : "null", "unit":"A","deviceId":1,"hubId":0," installedAppId":0,"descriptionText" : "null"}

I see no date/time to be found in either place, but I haven't gone over the 495
lines of Java to verify that helog is dumping all the data being sent by Hubitat.

This is what I get from logsocket .....

{"name":"Bar Light","msg":"Bar Light was turned on [digital]","id":9,"time":"2022-12-09 15:01:04.598","type":"dev","level":"info"}

I have no idea what helog is, so that's up to you to figure out.

2 Likes

Maybe i don't understand Maker API enough, but it seems like it is very much the wrong tool for this. As i understand it the purpose of maker API was to do 2 thing: provide a web interface for devices in hubitst to be sent commands and updates from a remote system, and the for those devices to also send event updates to a external system to provide a secondary automation. I dont think it was ever ment for this kind of logging and frankly this concern over timestamps does matter for simply passing status updates back and forth.

That said I have seen folks use it to do so with Node-Red, but Node-red has the ability to add the timestamp so thag wouldn't be an issue.

1 Like

That doesn't look like a Hubitat event JSON record. It's missing Hubitat event fields such as descriptionText, deviceId, and displayName and it has extra non-Hubitat fields such as id, msg, and level. That JSON is not coming from the Hubitat websocket. Some other program is adding the time stamp, not Hubitat. You've posted a logsocket JSON record, not an eventsocket JSON record.

I get this from logsocket:

{
  "level" : "info",
  "id" : 3,
  "msg" : "Received event from ZeeRadioUpper\/Office WallSwitch: [switch, on null]",
  "name" : "ZeeRadioUpper<span style=\"color:green\"> Online<\/span>",
  "time" : "2022-12-09 13:38:06.648",
  "type" : "app"
}

same results.
Websocket is similar but of course no timestamp.

{
  "hubId" : 0,
  "source" : "DEVICE",
  "value" : "off",
  "installedAppId" : 0,
  "deviceId" : 111,
  "displayName" : "Office WallSwitch",
  "name" : "switch",
  "type" : "null",
  "descriptionText" : "Office WallSwitch was turned off",
  "unit" : "null"
}

I'm on a Mac and use Websocket Client to capture.

2 Likes

You're incorrect. The issue is that you haven't used the links I posted.

You mean eventsocket :grinning:

1 Like

I concur completely.

I've just compared about 100 lines of logging between the on-hub event log, the Maker API URL external log, and the websocket (as dumped by the alpha version of helog) method, and both the on-hub event log and the Maker API logging method are missing events. It seems to be when a bunch of events happen "at the same time" (on the same second), some of them get lost, both in the list of events recorded on the hub and in the events sent out via Maker API (which makes sense, since Maker API reads the on-hub event stream). It doesn't explain why the websocket would have more events than the on-hub log.

Since the on-hub event stream is missing events, that may explain why some automation isn't working the way I want it to. The events I'm logging (and losing) are from an Aeotec Energy Meter Gen5 and it produces a lot of events. In my small sample, I also see lost events from the Hub Info driver.

Lucky for me, I have all these past log records to compare with. :slight_smile:

In any case, it means I have to give up on using Maker API URL logging, and give up on the idea of downloading events from the hub, since they are both incomplete. Only the (UNSUPPORTED) websocket appears to have all the events.

I'll have to upgrade my Java skills (and learn the Gradle build system?) so I can modify the 495-line Java program helog to add the missing date information and append log lines to my event log file. Or maybe I can just find a C program that does the same thing in 50 lines.

You should consider changing the device’s parameters to decrease the frequency of events.

It is capable of measuring changes in the data it reads that are so small, the resultant flood of events serves no purpose.

Sort of like this thread.

2 Likes

I did that when I set up the device, using the three categories of event timing. It's not the only device losing events.

Best of luck to you with all of the plans you’ve discussed above.

1 Like

Aye. I wish events had time stamps. Neither the events sent through Maker API nor the events sent through eventsocket have time stamps. I have not enabled logging for most of my devices and apps. I prefer to leave the logs for errors and warnings.