Why does my backup file size keep growing?

My nightly backup file size is currently ~32MB. A month ago it was ~20MB. I haven't added any devices in the interim. And I've actually reduced the number of rules.

Yet, the size of this file keeps creeping up in size. Just in the last five days its crept up from ~30MB to ~32MB.

Why does this happen?

30%20AM

Could it be the watchdog your running? I remember you are running that...

1 Like

It seems to be normal (at least from my backups):

-rw-r--r-- 1 he-user he-user 7.1M Sep 19 02:00 2019-09-19~2.1.4.130.lzf
-rw-r--r-- 1 he-user he-user 8.3M Sep 20 02:00 2019-09-20~2.1.4.130.lzf
-rw-r--r-- 1 he-user he-user 11M Sep 22 02:00 2019-09-22~2.1.4.130.lzf
-rw-r--r-- 1 he-user he-user 11M Sep 23 02:00 2019-09-23~2.1.4.130.lzf
-rw-r--r-- 1 he-user he-user 11M Sep 24 02:00 2019-09-24~2.1.4.130.lzf
-rw-r--r-- 1 he-user he-user 12M Sep 25 02:00 2019-09-25~2.1.4.130.lzf
-rw-r--r-- 1 he-user he-user 12M Sep 26 02:00 2019-09-26~2.1.5.118.lzf
-rw-r--r-- 1 he-user he-user 14M Sep 28 02:01 2019-09-28~2.1.5.120.lzf
-rw-r--r-- 1 he-user he-user 16M Sep 29 02:01 2019-09-29~2.1.5.120.lzf
-rw-r--r-- 1 he-user he-user 15M Sep 30 02:01 2019-09-30~2.1.5.120.lzf
-rw-r--r-- 1 he-user he-user 15M Oct 1 02:01 2019-10-01~2.1.5.120.lzf
-rw-r--r-- 1 he-user he-user 16M Oct 3 02:01 2019-10-03~2.1.5.120.lzf
-rw-r--r-- 1 he-user he-user 17M Oct 4 02:01 2019-10-04~2.1.5.120.lzf
-rw-r--r-- 1 he-user he-user 18M Oct 5 02:01 2019-10-05~2.1.5.124.lzf
-rw-r--r-- 1 he-user he-user 18M Oct 6 02:01 2019-10-06~2.1.5.124.lzf
-rw-r--r-- 1 he-user he-user 19M Oct 7 02:01 2019-10-07~2.1.5.124.lzf
-rw-r--r-- 1 he-user he-user 19M Oct 9 02:01 2019-10-09~2.1.5.124.lzf

1 Like

the primary element driving database size is events, the more events, the bigger the database.

4 Likes

How long does the history last? At what point will it begin purging those events from the backup?

1 Like

events are pruned 2AM local time every day.
The system leaves 1K max of the most recent events in the system after the prune, where 1K is measured by device:attributeName pair.

4 Likes

Thanks Mike!

If you backup after the prune, shouldn't the size basically plateau at some point (within some noise band, as not all events are the same size), though?

And maybe it does - I don't watch my backups closely enough to know if they are continually growing or not. So I'm not saying there is a problem, I was just thinking out loud.

2 Likes

yes, all of my production hubs express a 3% max variance in size over the 5 backups the system has taken.

2 Likes

Reminiscing. Simpler times. So young and naive.

Annotation%202019-10-09%20105036

3 Likes

Are you data shaming your hub? It may have put on a few extra bytes, but haven’t all of ours? Sorry, couldn’t help myself :rofl:

6 Likes

The extra bites is my problem too :grin:

5 Likes

And the Pun of the Day award goes to.... :+1:

5 Likes

@mike.maxwell reviving this thread, because I just noticed my backup file size growing and started wondering. Is there a mechanism for flushing all log entries before backing up? Seems odd to back up (and potentially restore) historical events... under what scenarios would it be useful to restore event history?

This should no longer be an issue. What platform version are you running? Events were truncated in number as of 2.2.3.

historical events are used in some apps, numberOfButtons being one example, zone motion manager uses them to determine the last event times for motion detectors...

while in general the main purpose for event values is processing rules in real time, previous values do have purpose.

4 Likes

@aaiyar: 2.2.3.148... probably my logs are continuing to grow because the history is short (setup is new and only stabilized two weeks ago)

@mike.maxwell... thanks! I didn't realize zone motion manager was using logs for this. I guess I assumed some internal state variables were responsible and a system reboot would mean reset of these values. Good to know it's more intelligent and should restore to the correct logic after a reboot.