A Tale of Loss and Recovery

Preface: I am not looking for support, just sharing my tale in hopes some will find it useful / relatable. If you want to speculate as to what happened that's fine, I'm willing to blame my hasty actions in my diminished state of trying to get my house back in order.

I woke up today feeling quite ill, perhaps foreshadowing.
My wife got to moving much faster than me and began yelling from the other room that nothing was working.
She had tried Google Assistant and Homekit which are our primary interfaces.
I grumpily fumbled open Apple Home and watched it fail.
I stumbled to the Hubitat app and was greeted with something like Hub Unreachable...

I blinked a bit and then headed over to the Hub WebUI from my phone but just watched the connection spin.
In my weakened state I popped open the TP-Link KASA app and delivered a swift kick to the Hub's pants to reboot it. (Perhaps what caused the next issue).
Upon the hub's recovery, within the Hubitat app I was able to connect to the Local Dashboard App but there were no dashboard.
Back on the WebUI I gasped as all of my Devices and Apps were gone.
For some reason I thought it would be a good time to update the hub, as maybe there was a bug fix for this issue.

It updated, and alas, no Devices nor Apps.
Then I realized if I wanted to restore a backup the version would be wrong.
By this time I needed to get to work as a few people were depending on me and it would be too hard to orchestrate from home, I had finally showered and my wits were being restored.
I remembered much conversation about port 8081, so I navigated to HUBIP:8081 and saw a very easy to understand screen, so I was able to revert my firmware with ease.

Then at work when I had a moment to breath I noticed something odd about the backup sizes available to me. The last two were much smaller than the previous ones.
I restored the last large backup and voila, I was back in business.

I'm not sure what caused the lockup, or the wipe after the rough reboot, I understand the demons in the code exist, but I wanted to thank the Hubitat staff for creating such good recovery tools to get me out of this jam.
And hopefully this helps somebody some day.

6 Likes

Was this .120 that locked up or .119?

Have you considered assigning a button to do a post to [your hub IP]:8080/hub/reboot rather than the equivalent of a plug yank? I may not always or and takes longer, but if the database isn't corrupted, then it's actually faster in the long run.

I truly think that most of the time we are guilty of not being patient enough and yanking the power, thinking the hub is "locked up", when in fact it's just REALLY slow. I'm not exonerating myself of any wrong doing here. I'm just as guilty of losing patience with the hub.

1 Like

Just get several hubs and each hub has virtual buttons to reboot the other hubs :slight_smile:
And no, I am not trying to inflate HE sales.... maybe.....

3 Likes

.119

I thought that was closed up. I'll put it in Home Assistant maybe.

I think since I got the Hub Unavailable from the app and no response from the Web UI I considered it dead as a door-nail. It's reckless of me but I'm of the mindset that if it can't withstand a simulated power outage from me then it doesn't belong in my house, but I don't think much withstands that standard. Especially things running on operating systems....

My rule of thumb for me is, if I can ping it, I can "almost" always reboot it using the POST url. The hard part is that when the hub is in a state where the web ui is not loading, it might take a couple of tries and the patience of a Koala. I try the POST wait at least 3 minutes to see if the reboot process starts. If not, repeat the POST and wait. If nada...then we gotta pull the plug.

Unfortunately, these hangs don't happen at a time when our patience is at it's highest (6am and the wife is upset that nothing is working..happened to me twice already)...but if you can wait, it usually works.

2 Likes

Hub locking up and loosing all devices and apps is a recent thing. It was not like this before. Something changed recently ?

The largest thing I've been doing lately is the MQTT Alpha.
But usually the webUI gets slow if there's something funny going on, it's been pretty smooth.
The hub has been very responsive lately.

Granted there could have been some critical bug randomly thrown last night.

So this is possibly fixed in .120, there’s that.

Also, as you’ve seen already, shutdown and reboot url is now only available via Post

1 Like