@frits It is quite possible that they are. How would i narrow that down? I have about 50 z-wave devices. It would be a real chore to read each log just to find that.
To answer your question, the delays aren't exactly constant. early in the ramp up, sometimes they are over the threshold, and others they are below. but it seems like as the Creeping Crud expands, that fewer and fewer times it is below the threshold. I am still watching logs from my hourly test runs. I will report back as my observations roll out.
That is a very interesting point,. Is there anyone following this thread and experiencing this problem that is running exclusively Native Drivers and Apps? IF so can you point that out? IF that situation exists, then that would be pretty strong evidence of a hubitat issue.
Well I only have about 50 devices so my logs aren't that crowded. I just looked at the past logs as soon as something wasn't reacting in time. For me it was the Hue dimmer (via HE) which I pressed and the light (via Hue bridge) that didn't react until about 12 seconds later. And the only thing I could see is that it always happens right after i get an sensor update from my Z-wave (aeon multisensor 6, running csteele driver) though the difference in times between button press and turning the light on are within milliseconds. So the hub sends the light on command right away, though the light reacts about 12 seconds later. I've tried giving the command by hand via the Philips Hue mobile app and it reacts right away every time. So it doesn't seem to be a networking issue (though never 100% sure of that).
And this is the ONLY time you have slowdowns? The hub speeds right back up after that? The problems myself and others are describing are systemic. Once the hub slows down, there is no recovery from it. It just gets worse and worse. So, I highly doubt it is tied to a single cause but something more impactful that is causing the issues described on this thread.
I only have the Hub Watchdog app and Tile Master for that one tile, Combined Presence, NOAA weather, and HubConnect for 3rd party apps. I have the TP-link (davegut) driver only (no app) for 3 bulbs with assigned IP addresses, GE/Jasco double-tap driver (Botched1) on 5 dimmers/switches, Virtual Motion with Switch, and Virtual Presence with Switch (Ogiewon) and that's it. Everything else is stock. I'm using the Amazon Echo Skill, Button Controller 3.0, Groups and Scenes, Hubitat Dashboard, HSM, HSL, Hue Bridge Integration, Mode Manager, Motion Lighting, Notifications, Rule Machine, and Zone Motion Controllers. I can't believe it is one of the drivers or apps I'm currently using causing the issues. My main slowdown starts with zigbee devices. I only have 16 or so z-wave and they are all switches except for 4 First Alert zcombo detectors.
EDIT: I don't know if z-wave is affected because mostly my zwave switches are manually operated, and automations involve zigbee sensors.
If it is systematic it's highly doubtful indeed. Maybe it's an idea to summarize the findings of all in this thread to the first post? That way someone like me (who is apparently experiencing something else) can read up on it to see if it's the same). It could also help the HE team a little in spending time reading up on this.
I feel like we need to clearly define the different kinds of "slowdowns" people are facing so that staff can have a better look and clear understanding of what's going on. Going off of my personal experience and following this thread pretty closely - symptoms in no particular order of severity:
WebUI slowdown - Clicking on any of the links on Hubitat's WebUI has a pretty significant (4-5 second) delay. If this is unrelated to 3., I've found this to be an issue with your network environment.
Z-wave/Zigbee device control slowdown - seems to be rare. This means if you directly control a Zigbee/Z-wave device from Hubitat's WebUI - there is some latency/delay. Example, turn on your Zigbee pocket socket or Z-wave switch. Does it change the instant you press the on button from the WebUI? IMO, this points to mesh issues.
Automated device control slow down - This means programmatic control (i.e via Rule Machine, Motion Lighting, etc.) of devices show significant lag. By significant, I mean around >1500ms, but this depends on your tolerance to latency. This issue usually shows up over time.
Network integrations slow down - This mostly has to do with latency around Hubitat controlling devices that reside on third party hubs. For example, Hue lights, Lutron Pro hub etc.
For me personally, the biggest issue is 3, after about a week or so of uptime. The funny thing about 3. is that the symptoms 1,2 and 4 also apply. But if you take a closer look, it's just the hub running automations really slowly.
To me that sounds like a network issue. The Hue app is immediately responsive because it has already secured the connection before you can do anything. I was having a problem almost identical to yours. Hue button controller on HE controlling Hue lights on Hue bridge. I didn't notice the problem until I added a zigbee light to turn on with the same button press. If I put the zigbee light command after the Hue command there was a 4 second delay in the zigbee light turning on or off after the Hue lights. If I added the Hue command after the zigbee light command then there was about a second or 2 of delay between the zigbee light turning on/off and the Hue lights turning on/off. I didn't realize it was so bad (Hue response time) because it's my wife's office lighting, and she always uses Alexa to control the lights which takes the HE out of the equation, and the other Hue lights are activated by motion, or by switches attached to the Hue bridge.
We'll see. I'm changing my Mikrotik router with a Ubiquiti Edgerouter Pro 8 next week. Maybe that will help? Don't know. I don't have any other networking issues though. The other thing is, when I configure the Z-wave devices to never report I never have any delays (but this could be coincidental).
The issue really is that 3 can look like 1, 2 or 4. For example, last night when i was experiencing a huge latency (10 seconds), at first I thought it was a rule problem or something I had screwed up. Anecdotal analysis can be very misleading, which is why I've tried to not speculate on a root cause.
I really think not a single environment can be compared to another one. I have three production hubs
Coordinator - C4 - no stick inserted
ZWave Hub - C4 - USB stick radio with Zigbee disabled
Zigbee Hub - C5 - Internal radios, Zwave disabled
For the sake of it, I am only going to compare my ZWave and Zigbee hubs as these only have 1 custom app installed (Hub Watchdog)
Zwave Hub
Zigbee Hub
As you can see, the Zwave Hub (C4 HW) is significantly "faster". Now, this can have several reasons:
I use a lot of Iris V1 contact sensors on my Zigbee hub that are known to be chatty
There is a difference between C4 and C5, while it was made clear by the team there shouldn't
Something else that we are all hinting for
Do I see slowdowns on these two hubs? Yes, but only on the Zigbee C5 hub. The Zwave C4 hub has always been rock solid. I just installed HubWatchdog on my Dev hub which is also a C5, I removed all custom apps and this hub does not have a single real device attached. Early indication is that the times for this hub is in between the two other production hubs but time will tell and I will report back.
What that could also mean is that the performance of the hub is very much on which real devices you have attached to your hub, that is why I say that you can't compare one user to another.
There is one thing I would be very very curious about but we can't get to that data. The platform relies on an underlying database. I would love to see some statistics in regard to database read/write performance to see if there is a trend over time. Reason is simple, a lot users that have a problem report that a soft reset with restore makes the hub feel "faster" again which could point to something in that area.
I don't even own a printer
I only have one desktop, 5 Raspberry pies, HE, Philips Hue and QNAP nas wired through a D-Link managed switch.
And 2 android phones, Samsung soundbar and two laptops via Unifi AC AP Pro.
No funny things like smart TV's or Wifi microwave ovens
Why do you have your devices split up like that? So, 4 of your 6 radios don't do anything ever? That doesn't sound like it's spreading the workload out but in fact stacking it up.
So, why not spread your zigbee devices out to all 3 hubs? Maybe the radio is the bottleneck causing slowdowns.
I split it by function. It was just a design choice to build the most robust mesh by radio type.
The Coordinator has all kinds of custom code and ethernet devices attached, so I did not want to mix in radio devices.
I did not want to have 3 zwave meshs and 3 zigbee meshs as this would harm the performance more than stacking them up that way.
In this setup, the slowdowns are rare but do occur. With rare I mean something like once every month or so. My coordinator is much busier and there it happens more often but I also have way more custom apps installed
I am a dreamer... LOL
