C-4 / C-5 Hubs crash on Loss of Internet

🛎️ Get Help
1

I searched the forum, but didn't find a close enough match to this issue based on what I read.

C-4 / C-5 Hubs crash on 'Loss of Internet'

Two hubs:
C-4 running 2.1.1.122 configured w/ DHCP MAC reservation
C-5 running 2.1.3.128 configured w/ DHCP MAC reservation

   *Both hubs are wired to the same switch

I use a Fing box (hardware / cloud service) to monitor my home's Internet health while away from home. Fing box records indicate there were five Internet outages over the last 30 days. (Thank you Metronet) Upon waking up or returning home from work, after all five of these occasions, I have found one or both of my hubs hung and not responsive to port 80 calls. Both hubs would accept port 8081 connections, which allowed me to safely reboot them.

In an effort to try and collect more data pertaining to the source of this problem, I have
begun the process of moving all Ethernet connected devices and services (local & Internet based) over to the C-5 and configure the C-4 for local Z-wave & Zigbee devices only. I am 95% done with this migration. I am hoping this migration may provide some additional details about the source of this problem.

For anyone familiar with the OSI model:

Is it possible I have a problem with upper OSI layer apps (native and or user) locking up if the Internet were to be lost during a communication cycle? I suspect that if I were to physically disconnect the Ethernet cable from the back of the Hub (layer 1 disruption), wait a few minutes and plug it back in that I wouldn't encounter a problem. Idk. I will try that tonight.

Anyone else observed this problem?

I do run the following apps:

  • Amazon Echo Skill
  • Several Cobra Apps
  • Hue Bridge
  • IFTTT
  • MyQ Lite ( Garage door opener)
  • Sonos Integration
  • Total Comfort API (Honey Wifi Thermostat)
  • Sleep IQ Manager (Sleep IQ Bed sensors)

Basic OSI information outlined below

By definition on a layered model as OSI or TCP/IP each layer works independent and not-aware of the lower layers.

When you remove the cable, it's a physical disruption ( layer 1 ), so almost inmediately ethernet ( layer 2 ) detects a loss of signal (if you're on Windows you will see the very dreaded pop-up informing network disconnected )

IP ( layer 3 ) and TCP ( layer 4 ) won't notice it, so they will try to keep on working.

TCP won't break a established TCP connection during a period of time because when TCP sends data, it expects an ACK in reply and if it doesn't arrive within a period of time, it re-transmits the data.

TCP will re-transmit the data, passing it to IP, who will pass it to Ethernet, who is unable to send it and simply discard it.

TCP will be waiting again and repeating this process until a timeout happens that let it declares that the connection is over. TCP resets the segment sequence number, discard the information that was trying to send and let free the buffer and memory resources that were allocated for that connection.

Plug the cable in before it happens and everything will keep going. This is what makes TCP reliable and at the same time vulnerable to DDos attacks.

If the OS has more than one interface (for example, ethernet and wi-fi), it is possible that when the ethernet goes down, it will try through wifi. It depends how the routing is configured, but in general terms " TCP won't be aware of that ".

2

Hubitat doesn't need the Internet. Yes, it will use it if available, but it can get along without. NTP and DNS would be examples of OS layer traffic to the Internet. Your Hub 'phoning home' to see if any new platform updates are available, is fractionally more visible, but won't break the hub if it fails to find NTP, DNS or AWS.

I seriously doubt Cobra's Apps are using Internet, except for their once a week 'sniff' of the version to see if any updates are needed.
Several Cobra Apps

All of these are non-Hubitat apps that use Internet. A loss of internet could at least be a potential problem.

IFTTT
MyQ Lite ( Garage door opener)
Total Comfort API (Honey Wifi Thermostat)
Sleep IQ Manager (Sleep IQ Bed sensors)

I would suggest rebooting whatever device (modem? router?) that connects you to Metronet. See if that causes your problem. (Obviously do it at a convenient time :slight_smile: ) Ideally you need to find a means to repeat the problem faster than once a week.

If you can repeat the problem, disabling the Apps in the list. Cause the problem.. if it fails, then you know it's not those apps and you can enable them again. If the problem didn't reoccur, enable half the apps and try again. Rinse, repeat til you've identified which one (or more likely two) apps are crashing when they lose access to the Internet. Contact the developer of the app for advice.

All the OSI layer info is good, but only applies to those portions of the Hub that USE the Internet. One of the apps is (apparently) going into an infinite loop on a loss of internet... at least that's my hypothesis. :smiley:

2 Likes
3

Is it possible that your loss of Internet is really an indicator of some power fluctuation? I have occasionally seen lights flicker and some devices fall over but others carry on as nothing happened. A server may restart but a desktop machine remains active. On those occasions, I have not had stoves or microwaves with power failure notifications.

1 Like
4

Both Hubs / Router / All Switches / Wifi APs are on battery backup and monitored. (all networking equipment)

5

I totally agree. I will start that process. Hoping someone else has started this already to cut down on what could be substantial troubleshooting.

All the OSI layer info is good, but only applies to those portions of the Hub that USE the Internet. One of the apps is (apparently) going into an infinite loop on a loss of internet... at least that's my hypothesis. :smiley: <== My guess as well