Database Size Growing - Correct Approach to Solving or Ignore it?!

So for a few months I have a familiar pattern: after a bit I start getting the "Alert Bubble of Doom" that tells me that my database is growing. I usually stare at it a while and sometimes it goes away (hourly cleanup?) and sometimes not. The dang ALERT is concerning though it seems like it shouldn't be. A couple of weeks ago I finally took the step of doing the Backup-Soft Reset-Restore boogie and all was well for awhile. So it's back. Anyway - sorry for the the long-winded preamble. I have read everything written on this subject, but I really don't know the right way to get to the identify the root problem. I would imagine that there's a device or app spewing out junk, but for the life of me I can't seem to find a definitive way to investigate it, calculate it and eliminate it. Can anyone point to a canonical guide to troubleshooting this or did they maybe set the threshold for the bubble of doom too low? Thanks!

Start with Logs > Device Stats and App Stats tabs.
Show settings on each and turn on the event/state columns at least (I just turn them all on).

You can the sort by those columns to see if any devices have high number of states/events saved.

You can also set the global Event/State history size lower at the top of the Devices tab.
If you do not care about the history a lot of people are setting these to very low values such as 11 or even 1.

If you dont want to change the global limits you can also go into the devices with higher metrics and reduce just those devices (this is what I have personally done).

Lastly, not sure if they added this to UI anywhere yet but you can also reduce the max days it keeps old data as well. Here are the endpoint links to set it:

Set Max Event and State Days to keep
YOUR.HUB.IP/hub/advanced/maxEventAgeDays/nnn
YOUR.HUB.IP/hub/advanced/maxDeviceStateAgeDays/nnn

Also you can view and set the alert limit:

View and update database size alert threshold
YOUR.HUB.IP/hub/advanced/databaseAlertThreshold
YOUR.HUB.IP/hub/advanced/databaseAlertThreshold/123
3 Likes

Should have provided that info. I have both set to 11.
image
And max set to 7:
image

Devices:

Apps:

Does anything scream out at you? Why? I mean if they'd say "hey, 23% of your db is made up of XX that would be great."

I appreciate the response though, I really do. Makes it look like Hub Information is an issue, though I added it to be able to view the growth of the database in the first place...

And while we're at it, with the settings at 11/11 why would we have counts over 11? It makes no sense. And I have logging turned off for Z-Wave Polling yet over 6000 events in the logs. It also makes no sense. These are some of the reasons why troubleshooting is perplexing!

I would do everything that @jtp10181 outlined above for setting states, max days, and so on. I have mine set pretty much identical to what he outlines above.

That is 11 per state. So lets say you have weather app. It has temperature, humidity, cloudy, precipitation, barometric pressure, today's high temp and today's low temp. That would be 77 states for just that one app. And if that weather app had a 5 day forecast, and you kept days or weeks worth of history, it would surely balloon out of control. So for things like this, and other very data intense apps, I crank everything down to a bare minimum.

I would say start with Echo Speaks. I would go in and see if you could reduce the states and events on the individual ES devices. I set mine to 2 for each device.

1 Like

Ok. I've cranked the echo speaks devices down to 1. Thanks. As I mentioned above, I already made the default 11/11 and max 7/7.

Thanks for the input.

Another consideration is the size of the attributes. Not specifically for your situation, but in case anyone stumbles on this later. For example, I had the Google Nest integration saving off images captured with camera events. Took less than 24 hours to start getting the DB size alerts.

1 Like

I didn't think apps could save to on-board storage?

If you have all reporting enabled Hub Information will produce a lot of data each time it runs. I personally use about half the attributes, polled at 10 minute intervals, and have the retention set at 1 & 1 (I do offload events for graphing CPU, Temperature, and Free Memory to my Synology - replaced a Pi 4 and a Pi 2B+).

The biggest tip seems to have been to reduce Echo Speaks devices from 11 to 1. My DB size has been varying between 40-60 since making that change.

It pulls the images as an attribute under the devices.

And it returns.
image
Geez. Really like Hubitat, but this "database is growing" thing is stressful.