Hubitat C7 hub keeps reverting firmware

I bought a Hubitat C7 back in Feb and got it setup and running, it shipped with firmware 2.2.5.131. I've since then upgraded the firmware several times since then to 2.2.6 and 2.2.7 but the firmware updates never stick. After a couple days it reverts back to 2.2.5.131 and has been doing this ever since I got it. I didn't realize this was happening till recently otherwise I would have reported it sooner. I updated to 2.2.7.126 and setup Devices and they showed in the app and used them until one day they vanished. This is when I finally noticed it had reverted to 2.2.5.131 and then updated to 2.2.7.128 and even rebooted the device a couple times and it stayed on that version. Then 5 days later it restarted and was back on 2.2.5.131 again... What the heck is going on?

Wow, that is weird!

Did you start a support ticket? That is the very first thing I would do. support@hubitat.com

It sounds like you are losing everything when this happens (devices, automations etc)? Did you try to do see if there were backups, and could you update firmware and restore a backup?

Is there anything unusual like power loss, lightening, or similar that could possibly interrupting power to this device?

I haven't yet, I'll start one now.

That's the strange thing, all of my automations and configuration have stayed in place which is why I didn't notice this was happening. The only thing I lost was the new Devices feature which was new to 2.2.7. I haven't noticed it rebooting but from the logs it's doing something. I have it running on a UPS to protect it from surge/power loss. But even if it was losing power it's odd it keeps reverting to the shipped firmware.

What do the main logs show, not the hub events you show above? Any errors? Any warnings?

If you have not already, I would try doing a fresh backup and save it to your desktop (you may need it at some point). Then do a soft reset, and restore this backup. I am not sure that will help anything, but a soft reset cannot hurt, and sometimes helps things especially if you have a database corruption of some type. Note: I would do this immediately after updating to 2.2.7xxx so you have the best chance of having a clean copy. https://docs.hubitat.com/index.php?title=How_to_backup_your_hub

The date on firmware and its start/stop scripts exactly matches the date of last restart, which makes me think it's been initialized from the diagnostics tool on port 8081 using "install platform 2.2.5.131" option. It doesn't have to be initiated from the tool either, just hitting http://your.hubs.ip.here:8081/api/switchTo225 will accomplish the same thing.
image
The feature is there to easily revert a hub to a "known good" firmware version without a need for cloud/internet access.

Sounds like a forgotten browser tab somewhere ..... then when you next restart that PC and it "restores" open browser tabs it hits that url.

1 Like

So there's a web link that can just be clicked to revert the firmware with no authentication or required approval...? That answers what's happening, my Nessus security scanner on the network which runs every Monday (and looking at these dates are Mondays) is crawling the device looking for vulnerabilities and is instead resetting the firmware on my device. That honestly sounds more like an issue than a feature. Especially if there's other known security issues in the firmware an attacker just has to hit that URL, revert a system, and then attack what was thought to be a patched device.

And que the age old counter argument that if someone is already on your internal network to issue a crafted or random URL to begin with, you are already F'ed from a security standpoint.

We use web crawlers at work for security, too, and find this type of thing on business grade/really expensive products designed for non-internet use all... the... time... So often that we regularly have to stick products behind a proxy and block those types of URLs to ensure they can't be misused.

All that said, I also think it would be better if it could prompt a confirmation. Not sure if that is technically doable or not.

3 Likes

If the attacker is in your network, you have much bigger problems then him resetting your hub

2 Likes

Someone getting into your network doesn't make you instantly pwned across the board if they're stopped at the device they got entry on. If someone is in my network I don't want something as simple as clicking a URL to be the way they get a permanent foothold inside my network. Just giving up on the idea of making something secure because there's no point is absolutely the wrong mindset for any product.

Ah, but when people speak in "absolutes" odds are they are "absolutely" wrong.

There is a whole spectrum of what "secure" means, and how concerning a given "insecurity" is. There is no one answer to "secure enough", and one person's "unacceptable" is another person's "who cares".

So what's my point? I don't know, I'll leave it to the reader to decide. :slight_smile:

All that said, I agree with you that having a URL that triggers a platform change with no confirmation is not ideal/prefered by me.

3 Likes

Thank you for your opinions on security.

On the topic of this issue I went into the Nessus scanner and updated the Web Application scanner -> Excluded pages (regex) from the default to include the api/switchTo to hopefully stop it from triggering it again. I left the version number off the tail JIC it changes.

/server_privileges\.php|logout
to
/server_privileges\.php|logout|api\/switchTo

I went looking at all the things there on that page to see what else I might need to exclude and realized there's a lot there but the only one triggering is the Restore Previous Version.

image

All of the other destructive ones like reboot, shutdown, soft reset, etc all require you to authenticate by putting in your hub MAC. The link to reverting firmware is the only one that doesn't.

image

1 Like

It's meant for when things go seriously sideways on the hub. I'm yet to see or imagine a situation when login becomes non-functional, but it might, at least theoretically.

2 Likes

It does prompt for a confirmation.

1 Like

Good deal. Then I guess there is no issue. :slight_smile:

I'm certainly not going to try the link Victor posted to prove you right or wrong. lol

4 Likes

On my C-5, it does not prompt for confirmation.

I entered the URL indicated by @gopher.ny
http://c-5-local-ip-address:8081/api/switchTo225

And without any prompting (or request for password), my hub rebooted. And is currently running platform version 2.2.5.131

P.S. Anyone from Hubitat, who wants to, is welcome to check my C-5's logs if it helps resolve this issue

4 Likes

Hitting the URL directly does not ask for a confirmation. It's a direct endpoint.
Going through diagnostics tool's full menu does.
Since it's a concern, I'll put it behind the same login as soft reset, etc. in the next diagnostics tool update.

8 Likes

Confirmation yes but not authentication. All of the other tools on that page require you to add your MAC to authenticate but the firmware rollback just asks yes/no.

image

image

image

Excellent, thank you!