Too Many Temperature Sensors! How to ignore them, and thereby not clog event Database?

🛎️ Get Help
21

If one reads the various statements from Hubitat staff on multiple threads, power user functionality development isn't high on the priority list. So, log control/maintenance isn't likely to be high on the list as event logs are used mostly as a trouble shooting tool.

Having said that, they do listen to power user requests. And they've hired a few new developers that are making improvements to the platform and drivers. (And I agree that some of the stock drivers are really at minimum functional levels.) So users should log their requests.

There are external methods to track history that are more powerful and flexible (but not as convenient) than Hubigraph.

1 Like
22

I couldn't agree more. I try to take care not to truly argue about anything, since in addition to it being pointless as you’ve mentioned, those are the threads that invariably get moved to the debate chamber, timeout room, or closed entirely. And that doesn’t end up helping anyone.

I have my viewpoint, OP has his. But when some posts degenerate into name-calling, I like to acknowledge that tempers appear to be flaring and pause for a bit, because that rarely leads to anything productive.

One doesn’t need to have any experience in IT to recognize that.

Edit: but in fairness to the OP, he did warn us in the very first sentence:

:slightly_smiling_face:

3 Likes
23

No argument there! I have to do that often... Lol

2 Likes
24

To be honest I was sure this thread would end up in the timeout room at around 8:01am EDT today, coincident with the temporary ban on a certain participant in the discussion being lifted.

So we’re actually doing better today than I had hoped! :wink:

3 Likes
25

Now THAT's FUNNY!!!

peace... I can kludge something together.

26

SOLUTION

Shockingly, searching for some working source code that handles a similar "generic zigbee contact sensor", what do I find?

I find a built-in driver:

Generic Zigbee Contact Sensor (no temp)

So - don't want eleventy-seven temperature reports per hour from your zigbee devices?
Just SWITCH DRIVERS, it seems.

I am just stunned to have stumbled over this after being blessed with so many diverse opinions as to why I should just be happy with the useless temperature reports from windowframes at 3 residences.

4 Likes
27

Yup. Glad you found the driver - I didn't know that was there. :+1:

1 Like
28

This is totally inaccurate. After you get to 1000 events for an attribute, the number of events in the backup will be constant. It will be 1000. Because that is the number of events that the system keeps. Whether those 1000 are from 1000 days or 2 days, that is the max number of events that will be saves for each attribute.

Correction: Now it limits the database to 100 events as of 2.2.3.

29

Sorry if I was unclear - for 2 identical Iris V1 door sensors, one provisioned with either the "Generic Zigbee Contact Sensor" or the "Iris V1 Contact Sensor" driver, and the other provisioned with the "Generic Zigbee Contact Sensor (no temp)" the one that uses the "no temp" driver will not be getting the temp data.

My database will be that much smaller for the lack of the temp data, it would seem. Regardless of how many points are kept in the database before it starts removing the oldest reading when adding the newest, the database file will be smaller.

My hub will also be that much faster, as it is not having to do all this removal of the oldest data when writing the new. In ancient SQL terms, that saves a query to find the oldest record (the "where" part of a "DELETE WHERE"), the overhead of the DELETE itself, and the INSERT to. Multiply this by a few dozen very chatty sensors, each reporting the ramp up and ramp down of daily temperature between 70s at night and summertime high 80s during the day, degree by degree, and one is avoiding roughly 40 such operations per day per sensor. (Hey people complain about hubs bogging down, and it cannot hurt to eliminate some workload. There is no avoiding the "chatty" nature of the sensor, so we are doing what we can with what we have.

If the database does limit one to only 100 events per data point, that's not well thought-out all all, as this means that I cannot track much of anything at any decent resolution for a 24-hour period. 1440 mins in 24 hrs, so I can't even keep one data point every 10 mins and be able to see it the next day. Looks like that local file manager is gong to get a workout to save the data.

30

If it was 1000, and it was recently changed to 100, how do you know that's not well thought out?

I'm not disputing that there could be a downside to that change as you have suggested.

But perhaps it was very well thought-out nonetheless. Or perhaps they did it for no good reason at all, though that seems less likely.

2 Likes
31

In an attempt to be less judgemental, I used the phrase "not well thought-out" to describe BREAKING THE ENTIRE PURPOSE of deploying home automation sensors, which is to know what happened over the past few days. If I wanted to have to constantly monitor a place, I'd be a security guard, not a happy retired person Kayaking around and enjoying myself.

32

While I often find it helpful to be able to look back at sensor data over time, I would not describe that as:

It's clear that's important for you, since you've mentioned you have multiple houses in more than one country that you'd like to monitor remotely.

I won't pretend to know enough about this topic to suggest whether there are easy workarounds despite that change they apparently made to how database events are saved. You're clearly very experienced and bring a great deal of knowledge in this area to the discussion. But your individual needs and priorities are not necessarily shared by all other Hubitat users.

And an important limitation on all of us, no matter how knowledgeable or experienced, is that we are all users, none of us work for hubitat and couldn't possibly be fully informed as to how/why they made decision X or change Y. So why start from the assumption that it was a dumb idea, vs. considering the possibility that you don't personally know why they felt like they had to make such a change?

2 Likes
33

Here’s another related thread that may provide some context.

34

@marktheknife I don't know if you are well-read enough to be able to know who Dr. Pangloss was, but you seem to be his reincarnation.

Let me save you some time in the future:

a) If someone has a question, they do not need to told that the question is impertinent.

b) If someone has a problem they do not need to be told that their problem is trivial, or somehow not part of a "mainstream marketing plan" to which only fanboys are privy.

This means that you need not comment on the overwhelming majority of my posts, as I would not bother the group at this point with more than questions that I encounter as I try to climb up the long and twisty "learning curve" on this device.

This will save us both much time, and save you some aggravation, as each and every time you respond with a pat on the head, I will make a special effort to respond and point out the specific errors in your reasoning.

In this case, you are making an assumption about something you cannot possibly know anything about - the market for devices like Hubitat. There are about 8 million 2nd homes in the USA... clearly, these are owned by people with the money to afford nice things, so they all likely want to monitor them when away, and various "alarm companies" are a liability, as they cost you steep fines if they call the authorities too often.

Overseas, such services are rare, and infrastructure is far less reliable than here in the USofA.
Infrastructure is also far less reliable in the more pastoral locations where one tends to want a 2nd home. So, a product that can be self-reliant is a good thing.

So, if a neighbor kid becomes an entrepreneur, and starts using my deck electric outlets to charge up 100 electric mopeds to rent to tourists, I'd like to be able to see that unusual power use, so as to rewind the security camera footage and see who it might be, grab some video, and send it along to his parents if I can recognize the young businessman,if not, to the local constabulary.

1 Like
35

Tl;dr but we’ll just agree to disagree.

Your tone often comes off as dismissive of the opinions and efforts of others.

Take care.

Edit: and yes for the record I have seen performances of the operetta candide :roll_eyes:, but seriously man even if you're trying to have a technical conversation with people, see if you can avoid insulting their intelligence or needlessly calling attention to the supposed superiority of yours, my guess is you'll find the discussion becomes even more fruitful as a result.

1 Like
36

Gee, I dunno.... let's look at some actual metrics, shall we?... I haven't been here that long.. only 68 posts total to date but yet 34 hearts received from other forum members. That does not sound "dismissive" in the least.

What I am dismissive of is someone who nay-says, yet makes no tangible technical contribution to a discussion that is entirely technical! :wink:

37

What I've read is that the hub backups are now only keeping 100 entries per device attribute, but I believe the live database still keeps 1000 entries. Have you read otherwise?

2 Likes
38

As I understand it the nightly maintenance will cull to 1000 per attribute, like it always has. The backup itself only keeps 100 though.

Checking on my hubs, that appears to be correct as I have >100 events for many individual attributes.

EDIT: Sounds like I was wrong. Sorry. It does cull everything to 100 events at maintenance time - not just the backups.

Sorry for (unintentionally) spreading misinformation.

3 Likes
39

So the backup only saves the most recent 100 events to the backup file AND the device database is also purged?

This makes me think that I have "misunderestimated" estimated something, as the purge will drop the last hours in the day of anything reported or polled at an interval greater than 15 mins. So, weather, power, water, and motion are going to be truncated in the backups, unless there is some way to separately backup the WHOLE event database as a stand-alone file.

The problem with "motion" is that vegetation waving in the breeze, feral cats, birds visiting the feeders, and even the rippling of the pool surface can set of motion detectors. It is hard to get a "minimum motion" defined for the few that allow such settings.

Weather data I can get elsewhere, of course. Just a shame I can't monitor it with a purported "central home automation hub". Its no longer the core device, I guess a Raspberry PI or something with an attached NAS drive will have to become the true "core", and the Hubitat a mere peripheral.

Power, and water are obvious - less resolution means that I don't see a familiar "heartbeat" with the little spike at the start of every fridge cycle (the starting current of a motor is always higher than the running current) to show that it is the fridge, I just see a spike of so many watts. I end up with peaks and square waves.

With water... is the toilet leaking every time it flushes, or are the groundskeepers just doing a far better job of washing their hands after using the toilet in these days of pestilence? At 15-min intervals, I'll never know unless I get lucky, and the water leaking his a leak sensor before it does damage.

Almost not even worth the very high cost of the devices to monitor consumption if the data is just going to get dropped on the floor at any decent resolution.

So, I am not just flying blind during the maintenance window, I have amnesia for any event that would be hidden in the noise at a 15-minute interval, or I loose all data collected for some interval before the nightly 2am purge.

I'm going to have to think about this - when I was a postdoc TA, I had to teach the Nyquist–Shannon sampling theorem, so I kinda think I have a problem here. :wink:

1 Like
40

I believe that is correct.

1 Like