Hubitat Crashes - About Same time every night

192.168.1.122 is RainMachine. App 243 is RainMachine. It is currently offline. Investigating why now.

Looking at the logs for Konnected, it appears to be doing discovery all the time.

When do you have your backups schedule for on Hubitat? Is it scheduled for every day? Could try shifting that to another time as well if its set for 12AM.

Is there anything else that happens on your network at 12AM? Anything scheduled on the router? Does any other devices auto-reboot at that time which may send a bunch of discovery messages out on the LAN when they come back? Anything that might send out a bunch of broadcasts. I know the hub is sensitive to Jumbo frames (anything over 1500 MTU). Just thinking if some sort of jumbo frames broadcast went out at 12AM it could knock the hub out, or maybe any other high volume broadcast could overwhelm it.

1 Like

Hubitat backups are scheduled daily at 2:15 AM.

Hubitat is in a 100 Mbps hub so it does not see any jumbo frame traffic.

Rainmachine is now back on the network and I have setup an alert to my phone if it goes offline again.

I have one Ubuntu desktop in my environment that does a backup across the LAN to a NAS. That starts at 1:00 AM. Things are relatively idle on it at midnight,

If you have jumbo frames enabled anywhere I would suggest disabling it totally as a test, if it still crashes then ok but at least it would rule it out. Seems odd though that it is always at the same time.

@support_team is there anything the hub does at 12am that could maybe lock it up?

Can you PM me the hub id? I'll check out the engineering logs on the hub.

1 Like

I had another crash last night. Same things in the logs. The RainMachine integration is definitely causing issues. When I have it enabled in Hubitat, I am unable to get to its UI and eventually it falls off the network. I completely removed it from Hubitat YESTERDAY so it is not causing the crashes. I disabled jumbo frames on all my servers and desktops, router, and NAS this morning. Will report back on what happens tonight. I don't suspect this should make a difference since it is in a 100 Mbps hub and should never see any jumbo frame traffic, but this will tell us definitively one way or the other.

1 Like

Can you PM me the hub id? I'll check out the engineering logs on the hub.

@brad @jtp10181 @gopher.ny Appreciate your help and patience...

Sorry for the delay, I was traveling. I disabled jumbo frames on all devices on the network and the issue did not happen yesterday. I investigated the hub I pulled from inventory and discovered it was not a 10/100 Mpbs hub. Must have fat fingered the model number when I looked it up. Was an ID10T error. I swapped it out for another hub that is 10/100 Mpbs, re-enabled jumbo frames on all devices on the network, and did not have a crash this morning.

I have noticed during this troubleshooting that the TCP/IP stack on the Hubitat hub does not seem to be as resilient as all other devices on my network. If I reboot my router to enable/disable jumbo frames, the hub falls off the network and does not come back. It has to be power cycled. When I reboot my Linux servers (one at a time) running my internal DNS, the hub falls off the network and does not come back. It has to be power cycled. Hubitat is the only device on my network out of 50-ish that exhibits this behavior. Not sure if it does not have the same retires, error handling, etc. that all other devices have or if there is something specific to how my Hubitat is configured. None of the other devices require a power cycle to get them back on the network after these type of events.

I also did some testing from the Network Test page. Pings to 8.8.8.8 and www.google.com fail, but traceroute to both work fine and return proper results. This test works from all other devices on my network. I would know immediately if it didn't. The wife and kids provide real-time monitoring of Google services :smile:. Not sure if there is an issue with the Ping Network Test or if this is indicative of an issue with my Hubitat config.

The ping test on the hub always fails for 8.8.8.8 for some reason, 1.1.1.1 however will work.

1 Like

I tagged @gopher.ny about that...

I've never had an issue with HE dropping off and requiring a reboot after a network device has been rebooted. Not that I do it all that often, but when I update the firmware on my router or switches never seen that behavior.

Ping to 8.8.8.8 fails for me too, though tracert works. Weird but not specific to your install.

This topic was automatically closed 365 days after the last reply. New replies are no longer allowed.