Hub going unresponsive in middle of night--Errors with Dashboard in logs

This is the second night that my hub has gone unresponsive around 3:30 in the morning and then had logs with problems with the Dashboards app like this:

Unfortunately, it doesn't seem to specify a specific dashboard causing a problem. Any suggestions?

I suggest to download a backup, do a soft reset of the hub then restore the backup. Don't do a hard reset!

2 Likes

I downloaded and restored my oldest database. Since this has just been a problem in the last couple of days, I have hopefully gone back before the issue initiated.

I wish I understood what the logs were telling me so I could avoid causing it again.

Are you running any custom drivers/apps?

I have custom drivers & apps that others have created; I haven't gone down that wormhole yet for myself. Unfortunately, I can't remember what I have added in the last 4 days. This has inspired me to be more proactive backing up my DB and have implemented @dan.t's Node Red flow to backup my DB.

So 5he errors in the logs show DB corruption. Anyone ever figured out possible causes for the corruption?

The issue of crashing hubs does not appear to be getting much attention from the devs. The logs indicate that the query failed because no RAM was free to allocate. Whether this is a cause of the corruption, or a result of it is impossible to determine without full logs.

Reaching out to support is a good thing to do so the issue is documented.

I’ll second the suggestion of backing up and restoring the database. But I see that as a band-aid. Something is causing it and that’s what needs to be fixed.

For what its worth, this happens to me on occasion too.

I had a hub crash yesterday just clicking on the events tab to look at device history. That experience has pretty much convinced me that the hub crashes are in some way database related.

Yes. Hubitat staff is aware, and I’ve been told there’s a game plan.[quote="srwhite, post:7, topic:27853"]
That experience has pretty much convinced me that the hub crashes are in some way database related.
[/quote]

I believe this is correct. There’s an exception that can cause a DB connection to close.

It might be worth taking a look at my Rootin’ Tootin’ Self-Rebootin’ App thread. It should help keep your hubs up in case of a DB closure.

3 Likes

I definately will, however when my hubs crash there's no 500 error, they're dead with a red LED on the front. As I said, I simply clicked on the events tab on a device yesterday and killed the hub completely. Couldn't even access the :8081 tools page.

Is this a C-4 Hub? Red LED on those is a shutdown from overheating, I’d believe.

They are but I would be shocked if that were the case. The hub is in an unheated attic equipment room. The temps in the room were in the 40's yesterday. I moved it up there for that very reason. If it overheated something in software caused a race condition to occur.. Could be related to the DB issue..

1 Like

You've only got one hub, I presume?

A (bad) approach to this could be a separate hub with an Iris smart plug that monitors the power usage of the other hub --- if it drops to 1W or something, turn off power for a minute or two, then turn it back on...

Not that I recommend this :crazy_face: But it could be done...

1 Like