New C-8 crashes every few hours

I got a new C-8 a few weeks ago, to replace the aging (C-5?) I've had for a few years. It seems to have been a bad decision. My old Hubitat ran for ages without ever having a problem. I don't think my C-8 has ever made it a full day without crashing (ie. no sensors respond, and the web UI won't load) - it generally crashes within a couple of hours. I'm running 2.3.5.121 right now, but this has been an issue with every firmware I've run on this device.

After looking at the logs a little more, it appears the box is still somewhat "alive" after the crashes, in that I still see log messages from the time period where it was completely unresponsive to the outside world.

The only "suspect" things I run are:

My network is a Firewalla router with TpLink Omada switches. The IoT devices (including Hubitat, Google Home, and Hue) are all running in their own VLAN, with internet access and access to each other. I've seen previous threads about jumbo frames, but don't seem to be using them in my network.

I'm at a loss, and getting pretty frustrated with the new hardware.

Edit: Looking at the log messages a little bit more, there are some complaints that the Hue Bridge Integration app is failing to connect to the Hue Bridge. However, I have reauthorized a couple of times, and my Hubitat is capable of controlling Hue lights between crashes, so I'm not sure exactly what the failure means.

app:722023-04-01 08:12:54.746 PMdebugputHandler: status:408, error:No route to host (Host unreachable)
app:722023-04-01 08:11:37.877 PMdebugputHandler: status:408, error:Connect to 192.168.40.155:80 [/192.168.40.155] failed: connect timed out

Thanks,

Brian

Try replacing the power supply, and then the cable. Probably not it but good to rule out some issue there. If you're using POE remove that from the chain to simplify things.

Another won't hurt/might help action is to shut down the hub, pull power, wait a minute, then plug in again. Likely won't help w/your reported issues, but won't hurt.

The things you're running shouldn't be causing crashes. Sounds like you aren't seeing any runaway error messaging in the logs, but if you can do w/out Hue control for a while I'd disable that and see if anything changes for you.

Sounds like no networking changes/differences between the C8 and C5. Are you setting a fixed IP for the C8 on your Firewalla?

When the hub crashes can you ping it, or is it totally incommunicado?

Can you clarify what you mean by crashing. Does that mean the hub is rebooting, or you can just no longer control devices on zwaveor zigbee, ect.

1 Like

Out of curiosity are you using zigbee devices, z-wave devices, or both? I've had many instances where my z-wave devices become unresponsive - curious to know if that's what you're seeing or if your zigbee devices go south as well. In my case though I can still get to the UI and a simple reboot brings them back.

For the moment try turning off the integrations and see how things behave.... What does your z-wave table look like? When it crashes can you get to yourhubip:8081?

Some followup to address the various questions:

  • I haven't tried replacing the various parts yet, or the "unplug, wait a minute, plugin" idea. I'll do those if none of the other ideas work.
  • My hubitat has a fixed IP.
  • "Crash" means that no devices are reachable, I can't bring up the web interface, and the hubitat no longer responds to pings. I also can't get to port 8081. It does not seem to mean that the hubitat is doing absolutely nothing. After I reboot it, I see a bunch of activity from the period where it was "down." And, it looks like expected things - motion detected on sensors, contact sensors opening and closing, etc. I'm not quite sure what to make of that?
  • I'm using both zwave and zigbee devices.

I'll start by disabling the Leviton integration, and it's the one I added most recently (I used it on my old Hubitat, but only briefly before the upgrade). If I disable the Hue integration, I lose almost everything of interest, so I might not even notice whether it's still working or not!

Hmm, is there a way to disable the Leviton driver without deleting everything? Same question for the Google Home and Hue integration apps? My setup is basically Leviton and Hue lights, controlled by sensors connected to my Hubitat. If I have to delete things to disable them, that would end up wiping most of my configuration and rules out.

yes on the app page or driver page you have this.

then you can disable the app or device

image

That to me that possibly looks like the hub is changing IP addresses or a IP conflict. How do you have it set to a IP. Did you set a DHCP reservation on your router, or did you just set it in the Hubitat UI. If you just set it in the Hubitat is the IP addresses used part of your DHCP address pool.

Do you have jumbo frames enabled anywhere in the network?

1 Like

I'm reserving the IP address in my router. It won't let me reserve the same address twice. :slight_smile: I can't imagine why the hub would be trying to change addresses, but is there some log I could check that would verify that? I just checked through all the devices my router knows about, and there's no visible IP address conflict with my hubitat.

I've got the Google Home and Hue Bridge Integration apps disabled, and all of the Leviton devices disabled (I couldn't seem to disable the driver), and I still have to reboot every couple of hours.

As for jumbo frames, I've never enabled them anywhere, and I can't find any indication that they're on from looking through my switch and router configs. Is there a more scientific way I could look for them?

AFAIK they are not generally enabled by default, so unless you remembered turning them on, seems unlikely the would be enabled. You could Google Firewall & TPLink Omada switch jumbo frames settings to confirm if they exist and where they are just to be sure.

Since you've disabled the integrations you're just going to have to wait it out now and see if the hub hangs on, or goes south again. If it hangs on then you can try adding the Hue & Google integrations back in (one at a time) and wait again.

The shut down/pull power/reboot suggestion sounds like it can't do much, but it does allow your Z-Wave radio to "start fresh" in a way that a simple reboot doesn't, so it's generally suggested when there are issues, as it's a "can't hurt/could help" step that is easy to do.

Good luck!

What switch and router make/model do you have?

The hubitat is still "crashing" every couple of hours, even with the various integrations disabled. I'll see if I've got some replacement cords and power supplies laying around that I can swap in, and I'll do the "slow powercycle" while I'm at it.

My router is a Firewalla Gold Plus, and the switch the hubitat is connected to is a TP-Link Omada TL-SG3428X v1.0. There are a couple other TP-Link Omada switches in the network, for other purposes. I've been Googling about all the devices, and it sounds like I'd have had to put explicit effort into using jumbo frames, which I have not.

Do a sudo ip link set dev eth0 mtu 1500 (do this or all interfaces) just to be sure and make it persistent

This one will pass on jumbo frames so I would check every single piece of equipment for anything with an MTU over 1500...

I've tried everything in here, including the "long power cycle" and replacing the power supply. I've also done a network scan to see if there are any competing IP addresses, and found nothing. My hubitat still completely loses network access every couple of hours, requiring me to unplug it to recover. Funny enough, sometimes it's still sort of functioning - there is one lamp that's completely controlled by Hubitat (ie. no integrations), and it will sometimes cycle on and off on its schedule, even when everything else is unresponsive. But, sometimes it's busted, too. I have no idea what's wrong, but it's really frustrating to "upgrade" from the C-5 (or C-3?) I had before, which was working fine, to this.

@support_team @gopher.ny

Can you put a sniffer on the network? (Wire shark) and see what's coming through? Everything you describe screams jumbo frames. Someone else recently had a similar issue and insisted that jumbo frames weren't enabled. (They weren't on his unifi or his switch) but it turned out he had a device on the network that had jumbo frames turned on and it was broadcasting large packets (the switch was letting it through) and was hosing Hubitat. Turned it off and hasn't had a prob since.

2 Likes

Would jumbo frames have killed my old hubitat, too? It never had any problems, and all the equipment was the same. Do you have a pointer to a doc that would explain how to do this? I haven't made it a habit to sniff the traffic on my network!

It can affect any of them. You need to check your equipment to see if anything is broadcasting anything over 1500 mtu...