Hub froze

I've never experienced anything similar to this, but this morning my hub "froze". Until now the hub was responsive, had no stability issues, but it seems something started to go wrong.

The hub wasn't actually frozen, it responded to ping requests, but its web server wasn't responding (timeout), and the automations were not working. After a reboot (power cycle) everything was working again.

At 6PM (when I got home) I noticed that the hub was extremely slow. It took 10-20 seconds to get any kind of response from its web interface... so I rebooted again. And it was working fine.

And now (11 PM)... it started to slow down again. Everything takes "ages" to complete (so far it was instantenous). Here is the log from a button activated scene (three, actually):

It seems there is a ~4 seconds delay somewhere, and it can be experienced everywhere, even on the web interface.

Nothing has changed since yesterday... I don't use TTS, I have no Google Home, no integration with other hubs... All I have is a (hardware) dashboard running 24/7, a telnet socket to my NUT server, and 2 speakers (the driver uses HTTP to communicate with them), but nothing else special. I have a simple Zigbee-only network and I use built-in apps (+ influxdb logger).

@mike.maxwell do you have any idea where that 4 seconds may come from? My suspect was DNS, but a local-only hub doesn't use my DNS server to execute automations... So I have no ideas where to start right now,,,

fw 2.1.0.123

This app is on the bad list. I had frequent lockups and I believe this app contributed heavily to my instability. Everyone's experience is different but I would recommend disabling (you don't need to remove it) and see how your hub responds after a reboot.

Also be careful with any 3rd party driver's/apps that communicate on the LAN. Nothing confirmed, but there have been rumblings of issues with LAN devices especially when they do not respond in a timely fashion.

1 Like

As @stephack says, http CAN be a problem depending on the method used by the driver. Sync or Async. Async doesn't "wait" while the other end responds. Sync, which is easier to write, waits for the other end to respond. Any slowdown in it's response is directly added to the driver's response.

On the other hand, influxDB would be the very first thing I'd disable.

3 Likes

Thank you guys, I removed influxdblogger, rebooted, and now everything seems to be just fine... but it's normal, a reboot always solved the issues for a few hours.

Next one will be the speakers's driver in case anything goes wrong.

I have no idea why it worked for me for months, but... whatever, we'll see.

Actually everyone was suggesting you disable not remove. And yes, reboot can temporarily clear up issues. So just disable next time you have an issue. That will make troubleshooting a lot simpler.

I assume sync and async is really at the app side of things, not the driver? For instance I have a few drivers that collect data via HTTP like this:

sendHubCommand(new hubitat.device.HubAction(
	method: "GET",
	path: "/query/device-info",
	headers: [ HOST: "${deviceIp}:8060" ]
))

Or is there a way to do those async, too? Sorry if it is a stupid question - LAN apps/drivers are not my specialty...

Edit: I guess if this is correct, then my example would already be async... (?).

A driver example might look like:

	def params = [ uri: "https://api.apixu.com/v1/forecast.json?key=$apixuKey&q=$zipCode&days=3" ]
	try {
	    httpGet(params)		{ resp ->
	        if (resp?.data)     obs << resp.data;
	        else                log.error "http call for ApiXU weather api did not return data: $resp";
	    }
	} catch (e) { log.error "http call failed for ApiXU weather api: $e" }
	//if (debugOutput) log.debug "$obs"

for Sync(hronous)

Async would snap that in half and look similar to:

private getXUdata()   {
	def obs = [:]
	def requestParams = [ uri: "https://api.apixu.com/v1/forecast.json?key=$apixuKey&q=$zipCode&days=3" ]
	asynchttpGet("asyncHTTPHandler", requestParams)
}

def asyncHTTPHandler(resp, data) {
	if(resp.getStatus() == 200 || resp.getStatus() == 207) {
		//log.debug "$resp.data"   
		return parseJson(resp.data)
} else ...

Yes, but you can also use hubaction commands for that instead of httpget or asynchttpget... Thus my confusion.

I'll go read some more on the difference between the two ways of doing it. I've been flipping through the ST docs on it, but I haven't seen anything conclusive in the sync vs async behavior of hubaction yet.

Sorry, misunderstood the question. :frowning:

Nah, I didn't express my question right - typical for me. Thanks for the concise example though!

I have a project that will need a new lan driver that I've been avoiding, as I don't know what I don't know on those. Guess I just need to dig in. :smile:

EDIT: The Hubitat guys were nice enough to confirm that Hubaction is async (which it looked like it was based on how it works, but it is always nice to confirm). So that's good news for me since a few custom drivers I'm using did it that way.

1 Like

Well, slowdown is back. More cleanup... :frowning:

Er... how can I disable a certain device or app? It must be well hidden somewhere :slight_smile:

It is well hidden :smiley:

On the Apps page, in the upper right corner is a well hidden (dim grey) X. Click it and it turns bright red and an extra column appears left of every App. Click any box there to disable that app.

43%20PM

1 Like

Ahhh... no comment, thank you :slight_smile:

This works on the devices page as well :wink:

I had my hub lock up this afternoon. Had to unplug it to get it back to work.

The only thing in the logs was that it could not reach my Sonos devices. I'll try disabling that functionality.

Meanwhile if someone has suggestions or knows of a solution?

Are you running Chromecast (Beta)?
That caused me hub lock up issues.
It was awhile ago now so it may not be an issue anymore but thought I would mention it.
I'm running the Sonos integration and it is OK for me.
Any other custom device handlers or apps running?
Support have advised that you can try disabling a few custom apps/drivers at a time and see if things improve to try and troubleshoot what can be causing your issues.

I have disabled a few custom apps and drivers. For example, Inreplaced Life360 with States for the default Life360 connector.

I've written a driver for a Sony tv that allows me to start tv apps such as Plex, youtube, amazon etc to be started from Hubitat. Don't feel that would be an issue.

Verify the unit is adequately ventilated / cooled. Multiple people have had hub lockups due to heat when installing it in a media cabinet, etc, on top of hot equipment.

After that, start disabling user code and apps. You might not THINK your self created device is the issue, but that doesn't mean it is not....

This.

My media cabinet is vented with three fans, but my hub still locked up. It is now mounted vertically behind the media cabinet using thin command strips. If I had thought through it better, I would have mounted it face down.

It froze again this morning. Limited it to only original drivers and it is located on top of a cabinet in the open.

I am running hubconnect to smartthings. Could that be an issue?