Database growing, size inconsistency, soft reset code 401, cookie clearing

🛎️ Get Help
1

This report has multiple questions, each numbered.
I could submit them as multiple reports if that is easier for you.

Platform Version: 2.3.2.141
Hardware Version: Rev C-7
Four devices: Aeotec aerQ temp sensor, Aeotec Energy Meter, Honeywell
T6 ZWave thermostat, virtual Thermostat Controller.

My Hubitat started telling me:

http://hubitat.local/alerts
"Your database is at 496m and growing. Try rebooting the hub first. If
that makes no difference, perform a soft reset as soon as possible."

The "soft reset" will erase the database. I don't want to do that.

I read that there is no way to ask Hubitat how much database space I am
using or have left. A reply by bravenel says I have to look at the size
of the backup and compare it with 8GB:

https://community.hubitat.com/t/hub-statistics/18426
bravenel: "As for disk usage, look at the size of your database backup. 
How much of 8GB does that represent?"

My daily database backup is only 1.6MB (http://hubitat.local/hub/backup),
and creating and downloading a copy gives the same size (1616048 bytes).
That size doesn't correspond with the alert size of "496m" at all, and
"496m" is not close to 8GB either, so I don't know why I see the alert.

The File Manager says (correctly) that I have no files and "Free Space:
1000 MB". I presume this file space is not shared with the database
space, so I can't use the free space here to know how much database
space I have left.

  1. If the database is "496m", then why does it download as 1.6MB?

  2. Why am I getting an alert at "496m" if I have 8GB of space?

  3. How do I find out how much database space I am using and have left?

As requested, I rebooted the hub. It came back up with:

Corrupt database found
Hub cannot continue. Please try the following actions:
Reboot the hub. Sometimes, a database file is locked but otherwise intact.
If that doesn't help, the hub needs a soft reset and a database restore. 
Click here to access diagnostics tool.
  1. Why did the hub corrupt its own database? I've had no power failures.

So I had to try a soft reset anyway. I started the diagnostics tool,
chose "Soft Reset", typed in "soft reset", pushed the button, and it
said "Error: Request failed with status code 401". HTTP code 401 is
"unauthorized". I tried "Reboot Hub", "Shutodwn Hub", "Enable Safe
Mode", and they all gave code 401. "Download backups" flashes a page
showing no backups then returns to the menu. "Login" changes to "Logout"
when I click it, and then "Logout" changes to "Login" when I click it,
but that's all; there is no login page.

  1. Why am I getting error code 401 on everything?

I power cycled the hub. Still corrupt.

I tried another soft reset. Still code 401.

I tried another reboot. Still corrupt.

I went to the diagnostics tool, cleared cookies in my browser, and tried
"Login" again. It asked for the MAC address. I entered the MAC address.
Now "Soft Reset" works. I restored from backup. Everything works.

  1. Why didn't "Login" work, i.e. why did I have to clear cookies to make
    the Login work?

This report has six questions. I can create separate reports for each, if
that is easier for you.

1 Like
2

Not true. Install the user-contributed Hubitat Information Driver.

2 Likes
3

Great questions! I'll start by clarifying a few assumptions before I respond to your 6 questions.

While that is true, it also allows you to restore a backup, which brings back everything you had at the time the backup was created.

As previously mentioned, you can extrapolate how much space is left, however, I would least be concerned about the space, and more so about the sudden spike in usage (more on that below).

Because the backup is compressed.

You are getting the alert because the database is growing unusually fast, and if it keeps growing at the same rate, your hub may run into problems. It is not looking at the available space, but rather at the increase rate.

There are ways to extrapolate, but size is rarely an issue, unless you are a power user with hundreds of devices connected, performing complex automation via custom apps and drivers.

The hub doesn't "corrupt its own database", the events that come from external devices, apps and integrations might lock its tables. When that happens, the only way to fix it is to perform a soft reset, then restore a backup. Once the access to the hub is regained, one might want to investigate what caused the database to become corrupted to ensure the same issue doesn't persist, otherwise the database will become corrupted again.

When the database becomes corrupted as result of sudden loss of power, the same external factors cause the database to become corrupted, but there is no longer a need to subsequently investigate, because the root cause is known (power outage).

The Soft Reset documentation explains that 401 error means that "for your privacy protection, the login is set to time out periodically. If you see Error 401 displayed, please Log Out of the Diagnostic Tool"

See above

See above

4 Likes
4

You restore from a backup. When you do a backup, the database is cleaned, so when you restore that database you restore is baby fresh clean. Nothing is lost.

5

Thanks for the quick reply. Some answers lead to more questions:

1. If the database is "496m", then why does it download as 1.6MB?
- Ans: Because the backup is compressed.

Is this really the answer? I do not believe that a 496MB Hubitat database
can be compressed to one third of a percent of its uncompressed size.

After reloading from a backup, when I use the "Hub Information" device I
just downloaded it says my database is only 41MB. Where does the figure
496MB come from? Looks like a bug to me.

2. Why am I getting an alert at "496m" if I have 8GB of space?
- Ans: You are getting the alert because the database is growing unusually fast

I would not have needed to ask this question if the warning had said
"Your database is at 496m and growing unusually quickly." Please upgrade
the alert to add your excellent clarification.

Also: The alert sentence "If that makes no difference, [then reset]"
is unhelpful, since the hub has no supported way for me to see the
information needed to detect a difference. I had to download the Hub
Information driver to even see the size of the database, and nothing tells
me the speed at which the database is growing, and without those numbers,
I can't detect any differences. The hub should have a supported way to
allow users to detect a difference, since the alert message demands that
I be able to do so.

3. How do I find out how much database space I am using and have left?
- Ans: (tells me I don't need to know this)

Really, I do want to know how much space I'm using and how much space
is available. The Hub Information device lets me know only the size
of the database, which is lovely but how big can it get without error?
Am I using 41MB out of a possible 50MB or 41MB out of a possible 8GB?

4. Why did the hub corrupt its own database?
- Ans: (some application did it)

Applications can corrupt the database, causing it to grow without bound
and requiring a manual reset and recovery? Oh my. Sounds fragile.

5. Why am I getting error code 401 on everything?
- Ans: (because I have to log in again - it's documented)

Yes, it's answered in the documentation that I only skimmed. Thanks.

6. Why didn't "Login" work, i.e. why did I have to clear cookies to
make the Login work?
- Ans: (no answer)

The answer to the cookie problem is that there is a bug where cookies
prevent the login window from appearing even after you log out.
The documentation avoids the bug by saying "close all your browser's
windows" and start over. A better fix would be to have the Logout
actually do a proper log out so that the subsequent Login would prompt
for the MAC address. I am not closing dozens of browser windows.

6

Thanks for your feedback. You make valid points. The "unusually quickly" is the key phrase. Everything else is driven by that statement. The sudden growth calls for immediate action and the solution you are looking for, is the alert to go away.

Just by looking at Logs you may be able to determine what's causing problems, maybe a device or an app is spinning events out of control. Occasionally, when growth happens "unusually quickly" might be because a table is locked, and a Soft Reset will unlock it. Sometimes a device spinning events out of control is expected (say you are updating the firmware on a device) and ignoring the alert is probably the best course of action, in that case.