Write event and debug logs to local PC

🚧 Developers
1

I couldn't find a convenient way to write logs from the Elevation to a local file on a PC, so I've built a very simple command-line utility which connects to logsocket and eventsocket, formats the live logs for human readability, and writes them to stdout for your piping or grepping pleasure.

I've published it on GitHub as helog.

You'll need a Java 11 runtime installed. Download helog-v1.0.0.tar or helog-v1.0.0.zip and uncompress the archive. The executables are in the bin directory. You can then run:

  • helog events 192.168.1.200 to stream eventsocket
  • helog log 192.168.1.200 to stream logssocket

...substituting the IP address for that of your own hub. As I understand it, eventsocket and logsocket aren't actually supported by Hubitat, so this tool might break at any time!

Sample output:

> bin/helog log 192.168.1.200
Connected to ws://192.168.1.200/logsocket
2022-12-29T11:15:58.226Z info   dev 36 Bathroom Underfloor Heating  Bathroom Underfloor Heating temperature 18.5 C
2022-12-29T11:16:24.804Z info   dev 36 Bathroom Underfloor Heating  On command, lastRunningMode: heat
2022-12-29T11:16:24.911Z info   dev 36 Bathroom Underfloor Heating  Bathroom Underfloor Heating is on
2022-12-29T11:16:25.200Z info   dev 36 Bathroom Underfloor Heating  Bathroom Underfloor Heating temperature 18.5 C
2022-12-29T11:16:34.911Z info   dev 36 Bathroom Underfloor Heating  Bathroom Underfloor Heating heating on

> bin/helog events 192.168.1.200
Connected to ws://192.168.1.200/eventsocket
2022-12-29T11:15:58.226Z DEVICE 36 Bathroom Underfloor Heating: temperature 18.5 C
2022-12-29T11:15:58.482Z DEVICE 36 Bathroom Underfloor Heating: thermostatMode heat On
2022-12-29T11:15:58.490Z DEVICE 36 Bathroom Underfloor Heating: switch true On
2022-12-29T11:16:25.910Z DEVICE 36 Bathroom Underfloor Heating: temperature 18.5 C
2022-12-29T11:16:34.911Z DEVICE 36 Bathroom Underfloor Heating: heating on

It can also filter by specific devices and apps, and write the output in CSV or raw JSON format. Run helog --help for details.

I hope this of use to the community. There are a few additional features I'm hoping to add as and when I get time, but let me know if there's a feature that would be particularly useful to you.

(I did find this post by @JohnRob - I'm not particularly familiar with Node-RED so it may be that his is a better solution and I've been wasting my time!)

(Edited to update links and sample output to v1.0.0)

7 Likes
2

This Java program was very useful to get started with websockets and verify that my Hubitat eventsocket works just fine. Alas, I don't speak Java.
Here's a little Perl script that does the same thing, dumping in raw JSON format:

2 Likes
3

I've found a bit of time to add a few features:

  • Filter by device or app (--device and/or --app)
  • Output in CSV format (--csv)
  • Improved log format, especially to better represent apps in the events stream
  • Now writes a message to stderr when it's connected to the hub, to save you from staring at a blank terminal until something happens.

I've edited my post above with new download links and sample output, and you can grab the new version here: helog-v0.0.2.tar or helog-v0.0.2.zip.

4

Nice work. Just installed it on my dev rpi and can see logs coming through from my dev/test HE hub. I'm going to play around with it some more so I can start capturing logs from each of my HE hubs.

5

I may not end up using this feature.... but could the device / app filtering cater for both inclusion and exclusion, i.e. only capture specific devices / apps (I believe this is the current behaviour), or, capture all devices / apps except XYZ? I expect if I choose to use the events option I may want to exclude certain "chatty" devices, but not need to nominate the devices / apps I want to capture. Does that make sense?

Thanks again.

6

@sburke781 Good idea - done! Try helog-v0.0.3.tar or helog-v0.0.3.zip.

This should let you exclude devices and apps with --xdevice and --xapp respectively.

1 Like
7

Hi @user2669,

I noticed the events don't include the date/time in the output. Is that not provided as part of the stream you are reading from? Seems odd if it isn't.... Though I do see you included a sample as a comment in the EventsJsonStream.java file and there is not one there... odd.

8

Yeah, surprisingly the event stream doesn't include a timestamp. You can also use --raw to see the JSON streamed from the hub. I do plan to generate a local timestamp, as it seems like it would be useful.

1 Like
9

Certainly for those who plan to use the data in platforms like Grafana... :slight_smile:

While I haven't looked at it in detail, does your Python setup include the date/time @idallen ?

10

Yes, my little Perl script adds a UTC date field (seconds since epoch) to websocket records as they are read. It understands both eventsocket and logsocket event streams. It's easy to modify the little script to use a different date format.

  • Y'all should also add your voice to ask Hubitat to fix the event stream to include time/date stamps.
  • Y'all should also ask that the event streams be standardized between eventsocket and Maker API and the on-hub copies that you can download using this awkward URL, which each contain different fields and none of which have a proper date field.

The on-hub download list has an HTML element in it that you can dig the date out of with some work, and the on-hub event listings in the GUI also have dates, but those dates aren't sent out by either of the websocket or Maker API interfaces.

2 Likes
11

I've just uploaded a new version - helog-v0.0.4.tar or helog-v0.0.4.zip - which includes the local time in both logs and event output, and also has some additional filters:

  • Filter debug log by level, using --level or --xlevel. For example, helog log 192.168.1.200 --level=error,warn.
  • Filter event log by event name, using --name or --xname. For example, helog events 192.168.1.200 --name=temperature.
  • The --help message is tidied up a bit.
  • Include local time, at which an event was received by helog, in all logs:
    • The events output now includes the local time. Previously it did not display a time at all.
    • The log output uses the local time instead of the hub's reported time.
    • CSV output (--csv) includes a new column, localTime. For the debug log, the hub's reported time is still carried in the time column.
1 Like
13

Thanks Ian, that all seems to work nicely (when I run the right version :wink: )

1 Like
14

So I found an actual bug and, in the process of fixing it, realised that helog's internals were far more complex than they needed to be. I've pushed a v0.0.5 (helog-v0.0.5.tar or helog-v0.0.5.zip) with a substantially simpler implementation.

I'm not planning to add any more features so sometime over the next couple of weeks I'll publish a v1.0.0 and call it complete. If you see any problems please let me know!

2 Likes
15

I've just published v1.0.0 (helog-v1.0.0.tar or helog-v1.0.0.zip). Only one change since v0.0.5:

  • --csv no longer converts null literals provided by the Hubitat Elevation into an empty string. In practice, this means that the events log will have null instead of empty fields when rendered as CSV. This brings CSV output closer to the actual information sent by the Hubitat, and is consistent with the handling of absent integer fields (which get written as 0).
16

This topic was automatically closed 365 days after the last reply. New replies are no longer allowed.