Zero custom drivers other than from HubConnect on the main and second hubs, the web hub also has hubduino drivers as well as HubConnect drivers.
have you tried rebooting on a set frequency. It seems to be the one way to prevent the hub from slowing down. I reboot both of mine nightly.
This is just silly.
It would be so beneficial if we had some type of access to the processor usage and memory usage. It would help us pin point the problems.
MAYBE it would. What if both of those showed OK, and it was a database latency issue and IOPS limit, or a Java heap problem. None of those would show up on CPU/memory.
It would be nice to see CPU/memory to rule them out of the equation, though.
This has been helpful to prove that it is slowing down over time. But obviously can't tell you what is causing it.
Bruce has made it super clear that this is never going to happen. The first of the following two posts is directly on point to why, he says, this is not useful.
Good points. Does anyone know WHAT exactly causes the hub slowdown when they do slowdown? Aka is it CPU spiked or memory overloaded? IF not, what specifically is causing the actual sluggishness of the hub.
If you want some additional history, I recommend you take some time and read this earlier thread. Everything folks in this thread are inquiring about has been asked previously. I am just trying to share some information, nothing more.
I have not seen anyone, including hubitat staff, that "know" anything. And let me be super clear too...I am not saying that I agree or disagree with Bruce. (Quite frankly I have no knowledge either way and it seems everyone makes good arguments in both directions.) I was merely pointing out that he has made his views perfectly clear. That's all.
I was having slow downs the system was taking upwards of 30 seconds to load a page my rules was delayed and taking a long time to fire. In my case though my issues started after several power outages in the same day. I did a soft reset and my hub has been good since. I think there may be multiple causes for the issues and in my case I feel the DB got corrupted causing my latency.
Not based on facts of course just my experience.
I had the same issue very recently. I safely powered down my hub as I needed to re-route the power cable. When I powered it back up, it was crawling at a snail's pace. The web UI took forever to load each page, automations were happening minutes later than they should, if at all.
Luckily a restore to a previous backup got things working again. Then I thought I should upgrade to the most recent firmware, as I had been putting it off, and thought it might help clean the hub up. After the update the hub was back to the snail's pace again. I had to restore a previous backup again and then it was fine.
This makes me nervous as what started out as a hobby, home automation, we are now fairly dependent on the hub to turn on/off some lights with Pico remotes, fireplace is controlled solely by a MimoLite through the hub, home alarm, alarm clocks (via the hub through lights and google homes), etc.
The simple fact that these slowdowns are resolved by a soft reset should be huge clue as the underlying cause. What is different about a machine running slow before reset, and one running fast after, with effectively the same data on board after restoring. Shouldn't they effectively be in the same "state" before and after?
Have the HE team actually picked up a case where a customer has this issue and remoted in to get a look at what's goin on internally? Surely they could quickly see if its was thrashing or swapping and what was causing it.
I know, I know, if it was easy, they would have done it...
Every day, all day. I think that's probably Support's Job Description right there.
Either they are not seeing a single smoking gun, or they are hiding the info. Right?? LOL
I said this in the other thread, but I guess it's worth repeating...
Symptom = slowdown --> does not mean a single problem.
GUI issues, where pages load slowly has been mostly tracked down to the NetGear auto negotiate problem. It's also mostly C-5 hub vintage only. So if you're seeing a slow GUI, and your hub is a C-5 and it's wired to a NetGear hub/switch, get a support call started
Reboot alone gets the hub working again.. but eventually it slows again. I posit that this is mostly DB Lock issues and that the reboot clears all the DB table locks and it's good to go.
Soft Reset + Backup restore seems to get many hubs working again. The process of doing that: getting a 'right now' backup and then doing a soft reset (functionally discarding the internal DB) and then restoring the backup takes just a couple of minutes. What occurs though is very sweet.. the DB backup process "exports" the DB but leaves behind any 'orphaned elements'.. same with the restore, it only imports complete DB records. Success with this means the DB is getting 'corrupted' -- although even that symptom is fuzzy and subject to multiple meanings -- and mostly what's known is that it IS corrupted but not how. It could have happened a day ago, or a minute ago and it's just sitting there waiting to 'stick a leg out and trip us'
These last two are recurring issues and the underlying causation is not known. I don't know how to inject an event and have that cause a Lock (and it's truthfully: the lack of an unlock that causes the problem ) I can write infinite loops, I can inject absurd amounts of events, both of which will be a resource problem for the Hub. but the moment I stop those, the hub quickly recovers and gets back to being snappy.
I'll add this.. I don't think Hubitat Staff are afraid of a bug We've seen Bruce say: "well that's a bug, thanks, fixed in next..." and "It's a lot of code, there will be bugs.." If there's a way to get this to stop, I'm pretty sure Hubitat isn't holding it back for some future release
I am not seeing the issue.. and in many ways I wish I was.. so I could contribute towards finding the underlying cause.
I wasn't seeing the issue for the longest time, until yesterday. I am ready and willing to contribute to finding the cause, but how? Do they actually remote in and look at processes, memory, etc?
If soft reset has a way of cleaning out the db from corruptions, orphans etc, surely they could write a housekeeping process to do that periodically on the live db without needing to soft reset your own hub?
Just out of interest, Do you have a longer data set ?
The 7 day graph seems to be showing the time increase as being approx 1 sec.
Edit: whoops just realized the chart above is month/day. That’s back to front for us down here. LOL
What exactly is this a graph of. Response time to what?
Yes, they remote in and look around.. mostly logs. I think it's just more day-to-day experience at where and what clues look like than a better view, more tools. Grep may be the tool they use