Don't update to 2.2.9.128 - disables reverse proxy - Update - Fix is out in 2.2.9.130

wow this kinda sucks... downgrading I go

Oh oh, I don't use a reverse proxy but I do use OpenVPN to connect to clients. Most tunnel connectors are using 172.x.x.x IP addresses so it looks like this may be an issue if it true that only /24's are allowed.

1 Like

These should be defined as private IP ranges by default if I'm reading this correctly:

10.0. 0.0/8
172.16. 0.0/12
192.168. 0.0/16

2 Likes

I'm adding some back end logging to save IPs of refused requests. That should help with troubleshooting and eventual workaround or fix.

2 Likes

Thanks. Wont logging just tell you which IP addresses any admin's cell provider etc. was used when trying to access the hub? in other words enabling IP Whitelist all type functionality will still be required by some users.

an advanced user is fine enabling this via endpoints. a novice user wouldnt go that far, further many notice users will never update their hubitats so they are still exposed regardless.

thanks, looking forward to a fix or workaround as there are some lovely new features in 2.2.9.128 i am keen to use when i can get around this issue.

2.2.9.129 (just released) logs refused connections' IPs but there's no resolution yet. I'll need more time to experiment with NGINX reverse proxy this long weekend.

1 Like

Whatever proxy you use, can you try commenting out X-Forwarded-For header? As in do not have proxy create one. That made a difference between receiving real IP vs. proxy's IP on my (admittedly very basic) nginx setup.

1 Like

Interesting, my reverse proxy works just fine. But then again I'm only using it internally to get subdomain / ssl access. I specifically block external access.

This is what I was thinking. There either has to be a configuration issue on the reverse proxies that are being used... or the 2.2.9 update must be treating forwarded IPs as the source instead of the raw IP from the reverse proxy.

I am going to try later setting the passHostHeader to false on my traefik v2 reverse proxy, but it would be useful to know how 2.2.9 detects and blocks the proxy.

If the X-Forwarded-* headers are used as the detection method then they can’t be disabled in traefik.

Whilst it might be a bug, The feature appears to be acting as intended in disabling proxy. I am taking a little frustration that I can’t disable or override this feature.

Creating workarounds on reverse proxies probably isn’t the most optimal approach.

Figured where Jetty was processing the header (sneaky) and how to disable that processing. Next update will have the fix.

5 Likes

Fix is out in 2.2.9.130, please let me know if it solves the issue.

6 Likes

Absolutely awesome job.

I can confirm the fix on version 2.2.9.130 works fully on my Traefik V2 reverse proxy with no config changes and with out the need for whilelists.

Thanks so much!

4 Likes

Is this going to be the new norm, where all ip address not on the local network -- and only /24 or longer at that -- need to be whitelisted via the allowSubnets endpoint?

Or is there any plan to hook up an opt-out of this feature in Settings ?

We found enough hubs exposed to the world with no authentication or encryption to make it a permanent, no-disable feature. As above posts suggest, there are workarounds, and they tend to come with some built in security features - use them. Anything that provides SSL based transportation layer and authentication should be good enough.

4 Likes

So i just want to be clear, are we saying that Hubitat has hardened the web interface and believes the only missing piece was TLS?

To be a bit more specific, i deal with PCI Compliance and i have received requests from auditors to do various tasks to harden the servers by preventing certain injection, and other system identify characteristics from being allowed to prevent http application vulnerabilities? Are those steps being taken on the Hubitat UI to prevent that kind of activity.

Using something like a reverse proxy will enable certain security features, but doesn't naturally protect against web application vulnerabilities as i understand it.

We do not think that a hub should be directly accessible over the internet under any circumstances. It is not designed for that. And we do not think a reverse proxy is enough by itself, there should be additional security on top of it. Hub firmware changes discussed above merely make sure that a proxy can access the hub.

4 Likes

Thanks for the clarification. That is what I expected.

1 Like

hi @cjattard - Apologies for the necro post here, but I see you're a fellow Traefik user and I've not been able to get dashboards to work behind my proxy address (I just get the constantly spinning blue circle "loading dashboards.")

I'm currently on 2.3.1.129. My network setup is pretty straightforward. No VLANs in use, and I've enabled my main internal subnet using (/hub/allowSubnets?192.168.2.0). I'm only testing the subnet locally and haven't even yet tried to test it via my OpenVPN setup at home.

Everything else in Hubitat seems to work behind the proxy, just not the dashboards page. I've got a couple dozen services working properly with Traefik, so I know Traefik is working as well, just not for hubitat dashboards pages. Do your dashboards still work at https://hubitat.customdomain.com/dashboards? If so, any hints from your traefik config would be appreciated. Thanks!

To answer my own post for future Googlers, @cjattard was nice enough me to PM me his complete traefik config. That helped me confirm that the issue was not my traefik config, but the need to update the hub to serve all requests via https when behind a reverse proxy. If you don't click that option, the dashboard page still serves all assets via http and the browser will block it from loading for security reasons.

I'd not seen this option to force SSL before now. If others need a reference, it is discussed here. Thank you!

2 Likes