[BETA] CSV Visualization - Completely Local Data Graphs

Companion app to [BETA] Device Attribute Iterative Storage (RRD) to allow some simple, but completely local, visualizations of the CSV data utilizing the chart.js library (thanks to @dandanache for the inspiration):

In HPM or at
(https://raw.githubusercontent.com/thebearmay/hubitat/main/apps/csvVisual.groovy)

HPM will install chart.js into the File Manager, but the app will check upon load and prompt you to install the file via a button if not found.

Data sets with attribute values that are reasonably close in scale graph best, i.e. don't expect a good graph if you're using freememory and cpuPct.

Graphs can be seen from inside the app or externally (including via the cloud) using an endpoint.

14 Likes

Now that is slick!
Works great, very nice.

Dashboard display?

1 Like

Use an iFrame tile device and the app endpoint.

Thank you, it does not get any better than this.

1 Like

Curious Question:

Something like this (resident to the hub) has been asked for going on a few years now.

What were the key elements that enabled it to be realized now?

File writing capability, storage capacity, tile capabilities, ...etc. ?

Thank you for addressing the need !

Local graphing has been available in a few different options over time. Hubigraphs was one option, but the developer quit supporting it for a few reasons. Then it got integrated in Webcore, There was also a option from @bptworld for a time.

The big concern with it is load on the hub to create graphs and manage the data.

1 Like

Most of the options to date were dependent on libraries that required Internet access to use, so the ability to download one library to the hub and make this work was attractive. webCoRE and Hubigraphs have local storage options integrated into them so not blazing a trail there but the file management utilities available on the hub now make it simpler to store and fetch the data, and by putting the number of iterations to retain in the user’s hands it allows them to control the impact to the hub.

So nothing completely new in concept, just a nice lightweight way of providing it. (In testing so far, it is less than half of my MakerAPI load for my Grafana implementation.) Browser does the heavy lifting for rendering, so minimal impact there too.

Edit: One other thing that may set this solution apart is that the visualization works through the cloud interface as well as locally.

6 Likes

v0.0.2 Changes:

  • Only show chart.js download button if not in File Manager
  • Fix download speed if chart.js is needed
  • Show available CSV files as a dropdown instead of requiring manual entry
  • Fix cloud rendering
1 Like

Much appreciated enhancements, thank you.

1 Like

I was interested in this, but I seem to be getting an error when I try to run the visualization:

Java.lang.NumberFormatException: For input string: "13:17:33" on line 191 (method pageRender)

The contents of the csv are as follows:

"device","timeStamp","avg-voltage","max-voltage","min-voltage","voltage"
"Bathroom Hot Water Heater zigbee","13:20:34","237.0","237","237.0","237"

I am in europe, so perhaps there is a difference in how the data is written?

The app is expecting the timestamp in the long format. Converting that back shouldn’t be too difficult, just something I hadn’t considered when writing this.

Edit: Actually thinking about it, that line is only formatting the time stamp into a readable format, so if it isn’t in the long format I could pass it through as is.

Edit 2: v0.0.3 is up

1 Like

I wonder if it would be possible to have more than one data file shown in the same chart (I guess that the attribute would need to be the same in each device/file). This would be useful where the device attributes are somehow related to (dependent on) each other or where we need to see the time relationship between event type attributes.
My particular use-case..... I have various problems with setting up vacation lights - so much so that I have started writing a simple app of my own. Most vacation light apps (including my own) are a pain to test (you have to test them over several evenings and live with the disruption they may cause to normal life). IMHO all vacation light apps should include a speeded up option and a graphical display of the results so that we can verify that everything is working in a satisfactory manner.
I have looked briefly at chart.js, could one do something like....
const mapDataPoint = function(xValue, yValue) {
return {
x: xValue,
y: yMap.indexOf(yValue)
};
};

myChart = new Chart(context,
{type: 'line',
data: datasets:
for each file
for each entry in file
data: [mapDataPoint(entry start, file index),
mapDataPoint(entry start + entry duration, file index),
]

Sorry for the above pseudo code - I was just trying to give some sort of idea of how it could be done. And reading back what I have written I'm not sure that I have expressed my self clearly (maybe that glass of wine didn't help - lol)

The issue I see is that the chances that both sets of data have exactly the same timestamps are so small as to be non-existant. Not an insurmountable problem but one that creates the need to extrapolate values when the timestamps are combined.

Why do the timestamps have to be the same? Surely the timestamps just serve to determine the location on the x-axis of the start and end of events (in my particular use-case) .

As far as I can tell chart.js only uses/allows one X-Axis so if they don’t agree you either make one set of data use the wrong X value, or you combine and extrapolate so that you don’t end up with values plunging to zero when the alternate data columns are read in.

I updated to the latest version. The error is gone, but the visualization is empty and no error message to share.

I realize this is beta, so just wanting to communicate what I am seeing (or not, as in this case).

1 Like

Probably your Device column. I'll need to ignore that for the visualization.

Edit: v0.0.4 is out to fix this.

So it's not as simple as simple as having one x axis whose range is updated whenever earlier or later data events/values are read in?

That's the effect I'll need to create, but the actual mechanics are a wee bit more complex.

v0.0.5 allows multiple CSV files as input. Couple of notes on this

  1. Timestamps MUST be in the default Long format (number of seconds/milliseconds since January 1, 1970).
  2. This is a heavier load on the hub than if you collect the data in the same file, but will allow to combine datasets from multiple devices.
5 Likes