Hub Load Severe

Was a long shot, but sometimes we get lucky. Given the symptoms I'd look at any app or device that does http calls as the wait for a response can sometimes accumulate and cause unexpected results. Same drill disable the device or app until you work through all of them and hope you find the answer early.

Do you happen to know if disabling the App will result in the CPU to go down? I took an extreme approach and disabled all apps but CPU is still high.

Disabling the app should stop it from doing any processing. Have you gone through the devices also? (Disabling a device will keep the driver from reacting to or initiating devices actions, but doesn’t keep the device from generatiing zigbee/zwave traffic.)

Do you have any errors occuring in Live Logging.

I also don't think it is a problem, but what if you stop InfluxDB Logger for a while.

6.6% seems high for it. Especially after Denny updated the code to alleviate some processes that were causing that meteic to be falsely elevated.

Thanks for the input. I've tried to iron out the errors from the rule machines. I have a few left from LG (installed for ~2 years) and Rachio (just installed 3 days ago). I will disable them and see what happens.

Re InfluxDB Logger, I've been logging a lot of stuff and I think the load is expected to be high because I'm logging a lot of stuff. I read through Denny's code and it's very high quality with good abstractions & error logging. The errors seem to be handled correctly with the messages logged. FWIW, I installed InfluxDB logger very lately to debug Hub Load Severe so it's not that likely it's the root cause. I also created a PR to log variables here Add capability to subscribe to Hubitat variables by sdachen · Pull Request #60 · HubitatCommunity/InfluxDB-Logger · GitHub... which I think wasn't the root cause either, because this was done after I had issues.

I'll try disabling everything with error messages and see what happens. Now I just have Ecobee turned off since it had one strike record of consuming 60+% of all my resources.

The reason I suggested it is becuase though you are right that how busy InfluxDB Logger is is based on how much you are logging my prod hub uses less then 1% as shown in the screen shot below. I use soft polling and have over 120 devices it tracks so not exactly a small setup. Many of the devices have multiple attributes tracked. So I would say 6% could be elevated from where it should be.

I agree Denny has done a great job with that code. You may still want to take a close look though at how you have it setup to ensure it isn't running with values to tight could based on what you said add to a already loaded hub.

What do you have Ecobee poll interval set to. It seems counter productive, but it should be set to 1 min to reduce load. For some reason setting it much above that has been found to cause issues.

Thanks for all the help...

After testing for a week, I found out it's likely Ecobee Suite Manager that is causing the issue. Since disabling it. my system hasn't been getting into Hub Load Sever anymore...

There are a few odd things I found though...

  1. Disabling it directly doesn't drop the CPU load, a reboot is needed. This is different from what I'm learning from @thebearmay in this thread. Does anyone know how it works under the hood?

  2. The load from Ecobee Suite Manager on the Apps page isn't high even if the hub was at 100% load for a few days.

Part of the load calculation is the number of threads waiting on a response. Http calls can stack up while waiting and may take some time to either receive an answer or time out

2 Likes

Sharing a update back here.

I finally got rid of the problem. It turned out ecobee suite was causing the issue (for unknown reasons).

I implemented the a new App and the corresponding Devices myself and now the hub has been running without issues and without the need to reboot for more than a week!

Guess it was indeed some http request but not a local one.

Thanks for all the help!

5 Likes

Hey what did you do to fix yours? Reinstall the app? I'm getting the same errors I think. App 6 is ecobee

2023-07-26 02:34:42.025 AMdebugDiscovering Konnected devices on the network via SSDP
app:62023-07-26 02:34:37.201 AMwarnApp 6 generates excessive hub load
app:62023-07-26 02:34:37.199 AMwarnApp 6 generates excessive hub load
app:62023-07-26 02:34:37.188 AMwarnApp 6 generates excessive hub load
app:62023-07-26 02:34:37.185 AMwarnApp 6 generates excessive hub load
app:62023-07-26 02:34:36.778 AMwarnApp 6 generates excessive hub load
app:62023-07-26 02:34:36.775 AMwarnApp 6 generates excessive hub load
app:62023-07-26 02:34:36.773 AMwarnApp 6 generates excessive hub load
app:62023-07-26 02:34:36.756 AMwarnApp 6 generates excessive hub load
app:62023-07-26 02:34:36.754 AMwarnApp 6 generates excessive hub load
app:62023-07-26 02:34:36.752 AMwarnApp 6 generates excessive hub load
app:62023-07-26 02:34:36.739 AMwarnApp 6 generates excessive hub load
app:62023-07-26 02:34:36.736 AMwarnApp 6 generates excessive hub load
app:62023-07-26 02:34:36.729 AMwarnApp 6 generates excessive hub load
app:62023-07-26 02:34:36.726 AMwarnApp 6 generates excessive hub load
app:62023-07-26 02:34:36.723 AMwarnApp 6 generates excessive hub load
app:62023-07-26 02:34:36.720 AMwarnApp 6 generates excessive hub load
app:62023-07-26 02:34:36.717 AMwarnApp 6 generates excessive hub load
app:62023-07-26 02:34:36.713 AMwarnApp 6 generates excessive hub load
app:62023-07-26 02:34:36.709 AMwarnApp 6 generates excessive hub load
app:62023-07-26 02:34:36.705 AMwarnApp 6 generates excessive hub load
app:62023-07-26 02:34:36.702 AMwarnApp 6 generates excessive hub load
app:62023-07-26 02:34:36.697 AMwarnApp 6 generates excessive hub load
app:62023-07-26 02:34:36.692 AMwarnApp 6 generates excessive hub load
app:62023-07-26 02:34:36.688 AMwarnApp 6 generates excessive hub load
app:62023-07-26 02:34:36.684 AMwarnApp 6 generates excessive hub load
app:62023-07-26 02:34:36.670 AMwarnApp 6 generates excessive hub load
app:62023-07-26 02:34:36.667 AMwarnApp 6 generates excessive hub load
app:62023-07-26 02:34:36.652 AMwarnApp 6 generates excessive hub load
app:62023-07-26 02:34:36.648 AMwarnApp 6 generates excessive hub load
app:62023-07-26 02:34:36.645 AMwarnApp 6 generates excessive hub load
app:62023-07-26 02:34:36.642 AMwarnApp 6 generates excessive hub load
app:62023-07-26 02:34:36.638 AMwarnApp 6 generates excessive hub load
app:62023-07-26 02:34:36.634 AMwarnApp 6 generates excessive hub load
app:62023-07-26 02:34:36.630 AMwarnApp 6 generates excessive hub load
app:62023-07-26 02:34:36.626 AMwarnApp 6 generates excessive hub load
app:62023-07-26 02:34:36.623 AMwarnApp 6 generates excessive hub load
app:62023-07-26 02:34:36.619 AMwarnApp 6 generates excessive hub load
app:62023-07-26 02:34:36.615 AMwarnApp 6 generates excessive hub load
app:62023-07-26 02:34:36.610 AMwarnApp 6 generates excessive hub load
app:62023-07-26 02:34:36.605 AMwarnApp 6 generates excessive hub load

When you say it is ecobee do you mean it is the community app Ecobee suite. If so whay is your polling interval.

Make sure it is 1 minute to reduce its chance of having issues.

I would also suggest doing a backup, soft reset, and restore from that backup. I had one occasion were something got corrupted and caused some issues with performance.

1 Like

How did you get this log? This seems super helpful! I had to find out which app was causing the issue one by one.

I solved the problem by disabling everything and uninstalling the app and implement what I needed for myself.

Simply turn on the debug on the device and it spits it out in the normal logs

2 Likes

The excessive load logs should come out automatically, if the app / device breaks the threshold you set.

1 Like

I rebooted the hubitat and installed from the package manager the ecobee suite update but that just added some apps code but nothing new on the app page. I believe its set by default to every 5 minutes as thats what it states on the app page and I dont think I changed it.

I have the ecobee in my apps that says (I've had this installed for probably a couple years without any issues)

  • Ecobee Integration
  • Copyright© 2022 Hubitat Elevation
  • Version: 1.0.2

I got the log by
1 . Open the hubitat main page
2. Click developer tools / logs
3. Click Past logs
4. Scroll down to the error code
5. On the left of the date and error code it will say app# or dev#
6. Roll the mouse over the app#/dev# and it will say Show Logs for the app XXXXXXXX and it lists the app throwing the error.

Now i do see i'm also getting amazon echo codes every 30 minutes almost exactly

2023-07-26 10:15:20.548 AMerrorEcho (v4.2.2.0) | getPlaylists Response Exception | Status: (404) | Msg: status code: 404, reason phrase: Not Found
dev:3212023-07-26 09:45:20.639 AMerrorEcho (v4.2.2.0) | getPlaylists Response Exception | Status: (404) | Msg: status code: 404, reason phrase: Not Found
dev:3212023-07-26 09:15:20.626 AMerrorEcho (v4.2.2.0) | getPlaylists Response Exception | Status: (404) | Msg: status code: 404, reason phrase: Not Found
dev:3212023-07-26 08:45:20.513 AMerrorEcho (v4.2.2.0) | getPlaylists Response Exception | Status: (404) | Msg: status code: 404, reason phrase: Not Found
1 Like

It was unfortunately the community version of the app. I've tried many things including many great ideas suggested by the maintainer and amazing ideas from the forum but unfortunate the problem keeps coming back :grimacing:.

I modified the community app, deleted what I didn't needed, used asyncHttp instead of blocking http calls to get the data and controls I needed.

1 Like

404 looks like the http request status code... it looks like some network issue and I've learnt on this forum that network calls can cause issues. I'd try disabling everything that is suspicious and see if it solves the problem. In my case, I found the ecobee community app was causing the issue and then I spent sometime to tackle it.

1 Like

Based on that information it looks like the built in Ecobee stuff instead of Ecobee suites. Hubitat Staff would need to look at that. You may want to create your own thread if that is the case.

Can you open up your Ecobee thermostat device and see what driver it is using?

There is no doubt that Ecobee Suite can be a bit of a hog, but i kind of think that is a result of the massive amount that it does. I don't use much of it's abilities at all. I do know it is best practice to run the polling every minute. Longer polling can lead to more work to be done at a give time. Think of it like smoothing the spikes. If you have 20 things to do every 10 min at one time it can be allot more impactful then 1-2 things every min.

2 Likes

This topic was automatically closed 365 days after the last reply. New replies are no longer allowed.