Hub process slowdown after several days

I have noticed that after a reboot of the hubs, everything is snappy and works very well, but after 3 to 4 days, all processes seem to slow down. a button push to turn on a light that would be less than a second, becomes 2 to 3 seconds. I have tried disabling multiple apps / drivers, but nothing seems to help. I am using HubConnect and 3 HE hubs, Hubduino, and Alexa skill. Is it possible that complex rules involving motion and cancel on truth change could be bogging the hub down after several days? I have multiple rooms on "automatic" lighting rules, with cancel on truth change for motion to keep the lights on. I don't recall having this issue until I started automating the lighting around the house.

:popcorn: :yum:

I have a theory, that I will be testing as I experience the same issues at times and have currently implemented rule machine rule to auto reboot hubs twice per day, which has helped, but it's still a ways out until I have ALOT of time to complete this time consuming test.

Currently right now my setup I have

My Server Hub - ALL wifi/lan/ cloud devices, almost every built in app, along with a few custom apps running, as well as where my Dashboards are located. With Both Zigbee/Zwave radios turned off.

My Remote Hub - All Zigbee/Zwave devices, Groups/Scenes, Simple Lighting, Motion Lighting, LCM and Rule Machine with most rules/logic pertaining to ONLY zigbee/zwave devices ran on this hub. Nothing cloud related.

I'm going to move my "Server" to what is now my device hub, also add to the remote hub all the "built in" apps, and lan devices. Leaving behind all wifi/cloud devices and custom apps on their own hub which will now be the "remote" hub.

My suspicion here is that because of the wifi/cloud device web traffic, they are causing the cache to fill up. causing the slowdowns. This traffic volume currently according to my router is

Server Hub

Versus the Device Hub

The goal is after this move the wifi/cloud devices will be segregated to themselves, leaving everything on the "server" as local ONLY, which I'm hoping will reduce the hub web traffic and less amounts of cache.

2 Likes

I have something similar now... my "server hub" has no internet based anything, it is connected to all normal zigbee devices and 3 GE Z-wave motion switches (closer to this hub and operate much faster than making the jump from another hub), and acts as the HubConnect server, connecting the other hubs and homebridge, as well as my Hubduino devices. This main hub runs all of the rules also. I purposely used a slave hub to interact with anything http related, such as Alexa, to help with the possibility of delayed responses and such from the web. the third hub is for everything else Z-wave (not a lot) and zigbee lamps. I do have 2 Hue bridges connected to my main server hub, as well as a Lutron pro hub. When it has been freshly rebooted, it screams, but give it 4-7 days and it is slow (smartthings slow)....

2 Likes

Do you typically notice the slowdowns on the server hub with "all normal zigbee devices and 3 GE Z-wave motion switches"

Or the hub with "everything else"

Or both? Do you notice any difference between speed between devices on either hub?

The slowdown is on all of them, as far as i can tell. The zigbee lamp hub has some buttons connected to it to control lighting, so it is completely separate of the main hub at that point, and it is slow too. Ultimatley every zigbee or z-wave device is slow, although logs show the trigger event, but 2 seconds or so later it executes the desired action. The reporting to homebridge seems unaffected, and as soon as Hubitat executes an action, such as turning a light on, arming alarm, etc, Homekit shows the change right away.

Interesting, that is strange.

I've been chasing this since early on. Initially it was zwave issues with devices not get deleted from the stick properly. After that...I don't know. I've disabled apps/custom drivers and then finally bought hub #2. completely thinking it would be the RF HUB (my zwave/zigbee hub) that would cause the most issues. Nope...it was the hub with all my RM and Motion Lighting rules. To really, really diagnose it...you'd have to completely setup a duplicate test environment. Not something I have time to do. So like @waynespringer79 I reboot nightly. It has helped my reliability tremendously. Is that the right way? Nope. But it works. I have recently deleted a bunch of things I didn't really "NEED" and by need I mean things that other household members would notice. Automated lights are really the things they notice. So I have not disabled all my Motion Lighting rules. I suspect some of these are the issues as I've had issues with them and the ML app getting updated. It's better for sure...but I suspect a few lingering problems.

Definietly RM4.0 can cause issues if your rules aren't rock solid. I know that as a fact as I had one that ran multiple times with nested IFs. Wasn't using the private boolean trick to stop it from running every time a sensor was tripped. That really slowed the hub down quick. I've kind of gone back to RM3 rules becuase they are easier for me to write and know they will only trigger on the initial instance rather than using the "changes" trick. Or when I do write Rule 4.0 rules I have multiple rules using virtual switches as boolean variables. (Yes I know there are global and I use a few of those also)

I too would like to pitch in here and say I'm having similar issues. I live in a pretty small 700sq ft apartment with a mix of Zigbee/z-wave devices and a bunch of pocket socket repeaters.

I initially pinned down my slowdown issues to cloud based smartapps - Ecobee, Echo speaks, etc. so I moved all of these to ST via hubconnect. The only custom applications I have running on my hub are:

Combined Presence
Hubconnect
LG TV Smart discovery
Welcome Home

I've tested these separately and couldn't pin down any issues with any of these apps. I do have a few rule machine rules, but nothing too complex.

Anyway, around every 2-3 days, like clockwork, my automations and webUI start slowing down. My usually blazing fast Zigbee based motion detectors have a 2-3 second delay between detecting motion and running actions. What's funny is that it's not just the actions that are delayed, sometimes, even zigbee events themselves are delayed when i look at live logging. I supposed the hub CPU being tied up slows down the zigbee radio?

A reboot is the only thing that fixes these issues, everything is quick and snappy right after. I've been going through the forums, and it seems like quite a few people are either rebooting their hubs on a regular basis, or depending on multiple hubs to get around this.

I'm not sure what else there is that I can do, seriously just considering deleting every single automation and smart app from Hubitat, setting up MQTT and moving all logic processing to Home Assistant.

There needs to be a better way to pinpoint performance issues other than manually disabling apps and spending hours if not days of trial and error only to arrive at the wrong conclusion.. I get that "htop" for hubitat won't do much, but we're all pretty much enthusiasts here and the lack of any real tools to troubleshoot performance issues is really quite disappointing.

6 Likes

I don't know what's going on, but I have been experiencing really bad slowdowns all of a sudden as well. I rebooted my hub for the second time today to try to fix slow automation, and it didn't help. A motion lighting automation that was near instantaneous this morning is now taking 30 seconds or so to fire. I am trying to look at the logs, but the web interface just spins on loading. None of my Lutron integration is working all of a sudden too.

So that clearly tells you it's NOT a mesh issue or radio issue. If it was a reboot would not help. Also some of us with multiple hubs...are still rebooting hubs nightly. I see it as a double edge sword. If we the consumer could see usage information support would have 5x the tickets.

Do you have any custom drivers? I've seem to found those can really cause issues.

1 Like

In the past when I've had slowdown problems it was almost always LAN based user drivers causing issues (especially when the driver couldn't connect to the end device it was trying to poll). Only one time has an app ever done it to me - and it was the native Chromecast integration ironically enough.

1 Like

I rebooted my main hub last night, the hub with most of the zigbee devices, 3 z-wave, and almost all of the rules. Everything is snappy again. The only wifi devices on that hub are @ogiewon 's Hubduino code, which has been very solid. I have tried disabling the code, but it didn't change anything, so I don't think that is it. It almost seems that something in the system is running out of resources, or not handling them right. Maybe the reboot flushes a cache that isn't being cleared on its own. I also wonder if it is days of logs being held in memory and causing the slow down...

Would you mind elaborating on this "trick"? I'm new to HE and RM and interested in improving my rules.

No trick really... You just use a variable (private boolean or any other local variable of your own creation) and use that as a condition for the action running.

That way the rule will run exactly one time until the variable is set back to true. It is handy to use if the trigger is very chatty, or the device erroneously produces duplicate events back to the hub.

Something like:

IF (private boolean = true AND door=open) THEN
set private boolean = false
Turn on lights
set private boolean = true after delay of 5 minutes
END IF

1 Like

I've been experiencing these slowdowns for months now. Every 5-7 days I need to reboot to get my motion related lights back on track. After about 5 days I notice all of my RM4.0 rules using motion become a few seconds slower than normal, also creating/editing rules is really sluggish. Everything else about the hub is still snappy including all my rules using the motion lighting app. Weird how this slowdown is only affecting RM for me. These slowdowns are starting to become a common theme around the forums with the common denominator being LAN Integrations it seems. Theres definitely more to this whole thing then were being told I feel. I guess I'll just keep rebooting, I'm not willing to loose my lan Integrations.

What @JasonJoel said. Here is an example with local variable

That keeps the top part of the if from triggering every time the motion sensor is triggered. Otherwise it would execute the top part of that rule over and over again. Now, all it's doing is cancelled the delayed turn off every time it's triggered...UNTIL the lights go off and the variable is set back to false.

1 Like

Heres my chart of hub device page loading speed. I dont experience slowdowns. Its a C-4 hub. Its pretty flat over time, the small steps correspond to updates. Im guessing folks with this slow-down problem would see more of a sawtooth shape, ramping up and then back to the typical <1s after being restarted. Would be interesting to see.

Another troubleshooting step that might be interesting is to run constant ping to the hub and watch the ping time over time as the hub slows down. This would show if there is any LAN issue, and if the hub cpu is getting slow at even servicing the network stack to reply to pings. Or it may show constant sub-ms ping times even through the "slowdowns", and that may be instructive data too.

image

1 Like

Same here.

I've rebooted twice in the last 2 weeks, and I'll need to do that again tonight.

The hub response gets distinctly slower every few days -- for example, accessing the hub from a local machine and clicking on any item in the interface (ie., listing dashboards, listing apps, opening a particular dashboard) takes 30~90 seconds, and hub control of lights/response to motion or other events is basically non-existent.

I can't be certain of what's on my hub, as it's inaccessible right now, but it's a C5 hub, with only 3 (4?) RM4 rules, 2 HSM rules. Lutron integration with 4 buttons programmed on 2 Picos -- less than 50 devices in total. I think there's only one 3rd-party app (voice messaging to Logitech Music Server daemon).