The Hubitat and InfluxDB are all in the same LAN, yes?
What is the address of the Hubitat? InfluxDB is at address 192.168.27.113. What OS is InfluxDB running on? What firewall is involved?
everything is in same Lan. hub IP is 192.168.27.151 I am trying to run influx on windows 10, and the firewall involved is the one that comes with windows. i think is called bitdefender. my router is a tplink deco M9 mesh router.
Caveat: I am not a windows user... Perhaps a more Windows savvy user could chime in?
To the best of my knowledge, Bitdefender is an add-on (paid) firewall--perhaps included by the PC vendor as a pre-installed extra. I believe the built-in Windows firewall is called Windows Defender. Perhaps you are running both? Either way, my money is on the firewall being the issue.
Btw, depending upon how the host OS firewall is implemented, you may need to restart services (such as InfluxDB) after you add/edit/delete/disable firewall entries.
windows defender sounds familiar. I never really played with it before until now. I do not run any addon stuff on my computer, not even antivirus.
Windows Defender is part of Microsoft's Malware/Antivirus protection I believe. Go to the windows search bar and type in firewall. It should show you the OS firewall. For testing turn it completely off and then restart influxDB and see if that helps.. That very well could be the issue as there are a few different ways to punch holes in that firewall.
Just out of curiosity is this windows machine a always on computer? If it ever goes to sleep it may be a bad choice for this. You may want to consider the raspberry pi with InfluxDB. I have a Pi Image published on here that already includes INfluxdb and grafana to help simplify the setup. I would suggest though that you use a USB connected SSD instead of a Micro USB card. They will fail a lot sooner running a DB on them.
1 Like
It is working now. none of the firewall stuff i tried made any changes, but while looking at the influxdb loger hubitat logs there seemed to be some communication going on there because i could influxdb version in there. Turning on "post hub information" and "include hub name as a tag for events" fixed the issue in the influx db logger settings.
Thank a lot for your help and I really appreciate it. Now on to figuring out how to use grafana.
@mavrrick58 I am currently running this db on a nuc computer, but I also have a RPI4 running octoprint. Is octoprint something i could also add to you rpi image . That would make more sense than having two computers always on when i could use one for everything. thanks
I don't now what Octoprint needs. If i remember right the Image is just Raspberry Pi OS with the software loaded on it. If the requirement for Octoprint is just Pi OS then i don't see why you couldn't add it.
The image includes InfluxDB, Grafana, Portainor.io for Docker management, Docker, DuckDNS scripts, and some other stuff. It has a good amount of add on's already.
This didn't fix it.
You were previously having timeouts for the TCP connection to InfluxDB. Changing tags in the records being posted can have no effect on connect timeouts--the connection to InfluxDB has to established before any records can be presented. I expect that something that was changed with the firewall did in fact fix the connection issue.
I guess you are correct. I was not able to replicate the problem by switching those two settings. Whatever the issue was it had to be firewall related then, because it is the only thing i changed many times. I didnt touch any network settings.
Hi @dennypage,
I recently added code to subscribe to variables for my own use and thought it'd be nice to share it back to the community and created a pull request here: Add capability to subscribe to Hubitat variables by sdachen · Pull Request #60 · HubitatCommunity/InfluxDB-Logger · GitHub
Please let me know if you're interested in merging this in and I can make the changes accordingly.
Thanks!
I've got something I've been wanting to accomplish, but not sure the best way how.
I'd like to track how long my furnace fan runs. Mostly for changing filters based on runtime and not every XX months.
I'm thinking that data may already be getting pushed to influx if we're logging thermostats. I can see and graph when it's running, but not sure about getting a total time.
And I believe thermostat only shows running if it's heat or AC, but not just the fan circulating air.
I just had a interesting experience this morning. Come to find out around 3:30 this morning my two instances of InfluxDB Logger stopped working. Very strange both at the same time. I am in recovery now but was wondering if you have any insight into what may have happened.
To set the stage a little bit of context.
I have my main instance of InfluxDB that handles all of my stuff and about a month ago added a second that simply does devices that need the Advance All attribute option flipped. This is currently only used for the HubInfo Driver and I capture 4 things with it. It stays a little busy because 1 of those 4 things is captured every min. I figured this was a safe setup incase something in the Custom Attributes blew up one instance the other would continue to run.
Well this morning both instances were not writing to InfluxDB. Now my big miss is i had turned off logging so i don't have anything for why these quit writing data. At first I noticed it with only one instance. I checked my syslog export which kept writing fine, but it didn't seem to capture anything out of the ordinary. So with little options to diagnose the issue I went to turn on logging on the instance of InfluxDB Logger handeling the Hub Info driver. Though i didn't see anything to why data wasn't being written I immedately saw logs from that instance show that data was being written from backlog. This was around 8am. So the simple act of enabling logging kicked something so that it would start to load the data. I figured this out later and will explain below. This seemed to get things in a better state, but one value wasn't updating. That should have come from the Hub Info Driver. Come to find out one of it's polling jobs flat out failed and similarly once i enabled Logging on it that device it was restarted and it started reporting. This lead to when i found out the other instance failed. Simply put now that i have my performance data i start to know my hub is getting into a bad state and cpu is sky high. Again researching it come to find out that the other instance failed and now with a little bit of idea to look for i dug into that instances program data. It appears that the job to WritetoinfluxDB was sitting there with no scheduled time. This prevented the app from attempting additional writes to the DB and it just built up until the hub got into the ruff state it was in. Now thankfully your recovery logic worked great. I simply opened the app set the logging to a higher value for this instance and then saved it and the write to influxdb job was recreated and off the recovery went. It took about 21 min to recover a little over 31k records. I am glad it recovered so well, but that number is a little bit of a suprise since i reverted back to your code 100% and it is set to 10k. I remember us talking about this value and why that was a bad idea. It certainly is because posting the 250 records took nearly a min in many cases early in that recovery process.
I am now going to dig through other things in my hub to see if anything else had it's scheduling borked, but was curious if you had seen anything like this yourself.
Edit So the more i am looking over my hub the more this is clearly not a InfluxDB issue, but maybe there is room for some kind of validation to help it recover faster when the condition exists that it's write job gets messed up by a system issue.
The interaction of having multiple threads doing different things and can cause interaction issues. At work we do a lot of "API" first and tons and I mean tons are invested in logging, analysis, and automation to prevent these types of scenarios. And we just added machine learning for it to come up with patterns of interactions we didn't even think of.
That was some good research on your end. I haven't seen that happen yet on my two hubs and I also run InfluxDB logger and the hub information driver.
1 Like
@ronv42 After diging into it on my hub this was clearly not a INfluxDB Logger, Hub INfo Driver issue. I also have about 6 Govee devices (of my 20+ devices that also got there scheduled polling job messed up. Something clearly happened on the hub.
I also don't think it was memory related because before the event memory was in a pretty good state. After well not so much because have 31k records in a state value is a bit ruff on the hub.
There is also a noticeable gap in my syslog export data for about a 2 min interval right when it started. I have a feeling something on the backend crashed which caused this. That syslog export doesn't queue up like INfluxDB Logger it just writes the records as it hits the Logging websocket.
At this point my thought is something on the backend crashed while the jobs were running that prevented the job from getting updated to be completed and also prevented the job from getting it's data.It would take HE Staff to figure that out though unfortunately.
My thought though for INfluxDB Logger is that this may be a spot that could be improved upon. The recovery process would have worked fine and kept my hub in a good state if it had just realized at some interval after the fauilure to retry the scheduled job. Almost like saying If new event comes in during a failed state after a hour just unschedule and reschedule the job maybe.
Hi @dennypage thank you so much for all the help!
I am having problem with one of the devices that logs into influx. I configured second instance of the influx app and selected advanced attributes and selected Flume Water Meter devices to send some attributes to influxv2. it worked for a day then it stopped working completely.
Here is the error message i am getting:
Post of 1 events failed - elapsed time 0.052 seconds - Status: 422, Error: Unprocessable Entity, Headers: [X-Influxdb-Build:OSS, X-Platform-Error-Code:unprocessable entity, X-Influxdb-Version:v2.7.1, Content-Length:225, Date:Fri, 28 Jul 2023 12:07:09 GMT, Content-Type:application/json; charset=utf-8], Data: null
Here is the record that is being dropped:
Failed record was: usageLastHour,deviceName=Flume_Water_Meter,deviceId=279,hubName=Home value=19.5 1690500420813000000
it seems like influx does not like the format its receiving. I am using custom driver for Flume Water Meter. Any idea how to get it fixed? is it Flume Water Meter Driver or influx logger app?
Thank you! Appreciate it!
So you have successful records being processed for that device. What do they look like, do you have any examples.
yes still have database records. It looks OK to me; just dont get it it suddenly start rejecting.
Here are the samples from influx database;
| #datatype | string | long | dateTime:RFC3339 | dateTime:RFC3339 | dateTime:RFC3339 | double | string | string | string | string | string |
|---|---|---|---|---|---|---|---|---|---|---|---|
| result | table | _start | _stop | _time | _value | _field | _measurement | deviceId | deviceName | hubName | |
| 0 | 2023-06-28T14:00:29.140899576Z | 2023-07-28T14:00:29.140899576Z | 2023-07-25T14:00:00Z | 0 | value | usageLastHour | 279 | Flume_Water_Meter | Home | ||
| 0 | 2023-06-28T14:00:29.140899576Z | 2023-07-28T14:00:29.140899576Z | 2023-07-25T15:00:00Z | 0.412666667 | value | usageLastHour | 279 | Flume_Water_Meter | Home | ||
| 0 | 2023-06-28T14:00:29.140899576Z | 2023-07-28T14:00:29.140899576Z | 2023-07-25T16:00:00Z | 1.4 | value | usageLastHour | 279 | Flume_Water_Meter | Home | ||
| 0 | 2023-06-28T14:00:29.140899576Z | 2023-07-28T14:00:29.140899576Z | 2023-07-25T17:00:00Z | 0 | value | usageLastHour | 279 | Flume_Water_Meter | Home | ||
| 0 | 2023-06-28T14:00:29.140899576Z | 2023-07-28T14:00:29.140899576Z | 2023-07-25T18:00:00Z | 2.598571429 | value | usageLastHour | 279 | Flume_Water_Meter | Home | ||
| 0 | 2023-06-28T14:00:29.140899576Z | 2023-07-28T14:00:29.140899576Z | 2023-07-25T19:00:00Z | 0.097142857 | value | usageLastHour | 279 | Flume_Water_Meter | Home |
hope this helps. Data type showing as "double"
i created brand new bucket and the field got created as string this time around;
| #datatype | string | long | dateTime:RFC3339 | dateTime:RFC3339 | dateTime:RFC3339 | string | string | string | string | string | string |
|---|---|---|---|---|---|---|---|---|---|---|---|
| result | table | _start | _stop | _time | _value | _field | _measurement | deviceId | deviceName | hubName | |
| 0 | 2023-06-28T14:15:51.936582785Z | 2023-07-28T14:15:51.936582785Z | 2023-07-25T22:00:03.842Z | n/a | value | usageLastHour | 279 | Flume_Water_Meter | Home | ||
| 0 | 2023-06-28T14:15:51.936582785Z | 2023-07-28T14:15:51.936582785Z | 2023-07-27T16:00:07.491Z | n/a | value | usageLastHour | 279 | Flume_Water_Meter | Home | ||
| 0 | 2023-06-28T14:15:51.936582785Z | 2023-07-28T14:15:51.936582785Z | 2023-07-27T17:00:23.18Z | n/a | value | usageLastHour | 279 | Flume_Water_Meter | Home |
not sure how influxdb determines what the datatype is
Here is what the device is reporting:
Current States:
- commStatus : good
- flowDurationMin : 0
- flowStatus : stopped
- presence : present
- usageLastDay : 45.68
- usageLastHour : 0.0
- usageLastMinute : 0.0
- usageLastMonth : 2144.31
- usageLastWeek : 337.28
- water : dry
I suspect that INfluxDB is looking at the data and trying to make the best determination based on what is in it. This is part of the problem with community drivers that have custom attributes vs standardized attributes as defined by capailities.
This is also why I asked to see what the calls were that worked vs didn't work. There may be a change in parts of the data stream that make it incompatiable at times. Like as you pointed out in this case going from Double Number type value to a string value. This would take time to analyze what the driver is passing as events to see if the data type information is inconsistant.
This is likely a driver issue.
ohh I think i see whats going on now: if you look at the device events that is polling data every 5 mins and sometimes the attribute reports actual number and sometimes just says "N/A" instead of "0.0"
| usageLastDay | 48.01 | DEVICE | Flume_Water_Meter | * InfluxDB Logger_Flume (handleEvent) | 2023-07-28 10:35:38.711 EDT | |||
|---|---|---|---|---|---|---|---|---|
| usageLastHour | 2.33 | DEVICE | Flume_Water_Meter | 2023-07-28 10:35:38.710 EDT | ||||
| usageLastHour | 0.0 | DEVICE | Flume_Water_Meter | 2023-07-28 10:05:31.567 EDT | ||||
| usageLastHour | n/a | DEVICE | Flume_Water_Meter | 2023-07-28 10:00:30.382 EDT | ||||
| usageLastHour | 0.0 | DEVICE | Flume_Water_Meter | 2023-07-28 09:05:16.449 EDT | ||||
| usageLastHour | n/a | DEVICE | Flume_Water_Meter | 2023-07-28 09:00:15.208 EDT |
any idea how to address this?
