I figured out my issue I think. One night I noticed in the log that Alexa App was going NUTS (Builtin not the TTS). There were tons of throttling mesages from Amazon and my hub voice announcements were getting slow. I think I had Alexa Echo App not Alexa Echo Skill. I had always wanted to upgrade, but hadn't ever seen the option to install Alexa Echo Skill. Well I deleted the Alexa Echo App, and when I went to reinstall I saw the Alexa Echo Skill and instead installed it. I also pruned the list of allowed devices down to 20ish. Ever since then I've been a OK. My Alexa view of my house via the app is also more responsive.
That's interesting information.
I have disabled the Chrome cast integration (under guidance from support) and my hub has been ok ever since. Still early days but fingers crossed.
1 Like
Just an anecdotal piece of info.... The other day when my hub slowed down to the point automation stopped, the only error in the logs was from the chromecast integration. There was just a single error in the log, so I didn't think much of it, but it was there...
Not saying that is what caused my hub to bog down, but based on the discussion above I thought I would mention it.
I have been thinking of moving all "LAN" style connections to my 2nd hub anyway, maybe I'll move Chromecast 1st when I have time next week.
I will say that I'm going to laugh my **** off if a native Hubitat created piece of code ends up being the issue - and not all the "evil user code".
Yes, I know that's immature and petty - what can I say? I'm flawed.
7 Likes
When my production hub was experiencing lockups about every 2 weeks, I went through the list and disabled/removed what I considered 'suspect applications'. Two of those were the Hubitat's Chromecast Integration (Beta) and the amazing Echo Speaks by @tonesto7 .
It is hard to say if either one of them was the cause, however my hub issues seem to be reduced. I am now running both of these integrations on my development up as I wanted to see if I could 'move the problem' as part of my troubleshooting. The development hub has not locked up ever. Of course, I really don't use it for much other than development, and it gets routine firmware updates/restarts as it is part of the beta program. So if the problem is a gradual resource exhaustion issue, it might take months on my development hub versus weeks on my production hub.
Recently, I enabled a feature on my Asus Router that collects WAN usage data by device. My production HE hub did not show anything unusual (i.e. very little data per day going to the WAN.) My development hub, on the other hand, showed a significant amount of data usage every day going to Amazon. I suspected Echo Speaks as the large data consumer and so I disabled it yesterday. You can see in the chart below the dramatic change in traffic as a result. This hub does not even utilize Echo Speaks (i.e. no TTS whatsoever) so I am not sure why there is so much traffic. Maybe it is trying to auto-discover new devices as they added to your Amazon account? Over the course of a full 24 hours, Echo Speaks on this one hub consumes ~500MB of total bandwidth for my 4 echo devices. Not a massive amount of data, but for folks with a monthly bandwidth cap it is something to be aware of. This could also consume a cellular backup WAN solution's data cap fairly quickly.
I am not pointing the finger at either the Chromecast Integration or Echo Speaks. I just wanted to add some more data to the discussion.
4 Likes
Not really fair Jason. They do label Chromecast Integration as Beta, and @chuck.schwer has said it probably should actually have the label “Alpha”. Really none of these guys have used evil to describe user created code, and have often gone out of their way to help with issues in user created code. They’ve also never said any of their code is perfect. They’re not putting themselves on a pedestal, so let’s not accuse them of looking down at anyone from a pedestal.
@ogiewon Thank you for the insight to your tests. I’ve been wondering about the Chromecast Integration Beta, but have not tried disabling it. I will do that now to see if things improve over the next week.
I must say I have suspected echo speaks of causing slow downs as well. I however like it so much I have been hoping it wasn't so. I plan on switching my chromecast stuff back to cast web api and likely going back to my telnet driver for Alexa tts and routine triggering.
My anecdote - I had some unexplained lock-ups. I had the original Hubitat Nest app. It was throwing some errors. I deleted it and installed the community app. No lock-ups since. It's hard to say if that was the issue as there have been regular reboots due to platform upgrades. I also have Echo Speaks and Chromecast installed.
My hub became extremely slow last night and motion lighting seems to have been the culprit. I hit a button to turn off the lights and they went into some kind of on/off loop for about 5 minutes until I disabled the motion lighting app which immediately fixed the lights. The web interface regained its responsiveness, but automations still had some lag, so I ended up rebooting it. It only made it a day and a half since the last lockup. The thing is, everything works great, (when it’s working) but I always see more delay with motion lighting on Hubitat than I did on SmartThings. I don’t understand why it’s not faster. Same motion sensors and lights.
EDIT: You know... You're right, probably not "fair". I thought it was "funny" though. 
1 Like
We appreciate the passion everyone has for trying to solve sources of Hub Lockups.
What we know is this:
- If we can reproduce it, we can probably fix it.
- We need to isolate apps and drivers users are using to determine where or what is the cause
- Bad code is bad code... No software is perfect, but we continually improve the hub core code to make sure we fix what we know are issues and what we can reproduce or observe as issues.
We have identified one significant bug in how network connections, under specific circumstances, caused the hub to stop working. We could not have found this cause without the help of the community. This fix will be available in the next release.
We continue to try to find root causes of lockups, freezes, slow downs, etc in our never ending quest to build the best hub out there.
It is basic troubleshooting to baseline and isolate potential sources of problems and determine what the cause is, what the steps to reproduce are and ultimately what the fix might entail and then make sure that fix doesn't break anything else.
When we ask you to disable (not remove) user drivers and apps, it's simply to get a baseline of the hubs operation. If the problem still exists, then we know it's not related to user drivers and / or apps.
We added the option to disable devices and apps very easily in the Web Interface to make this process much easier.
One simple process of troubleshooting is then cutting things in halves. Enable half of your disabled apps and /or devices and see if the problem returns. Continue cutting that half in half until you isolate and identify the potential root cause.
With lockups that are potentially due to time (typically memory leaks or resource issues, random events like 3rd party endpoints that don't respond, etc.) it's important to try to track down those causes. Some problems like a random lockups could simply be due to a power surge, or a rouge device flooding the network, etc. These are the hardest to track down, let alone reproduce or eventually fix.
When we have enough information to identify a cause, be able to reproduce it, we can then fix it. We have fixed many causes over the last 14 months and will continue to knock them out as we find them.
We are a dedicated team that actually uses Hubitat in our own homes, we use the hub's code editor to write our apps and drivers just like any community developer does. There isn't any magic answer to what users or user installed code can do to create issues, random lockups, unhandled exceptions, excessive logging or even loops that can drag the hub down.
We can not solve these issues without your help. Anything you can do to isolate, identify and reproduce the issues, helps us out tremendously.
Thanks for your patience and willingness to help out!
24 Likes
Thanks for spending the time on this it's appreciated.
The higher consumption for Echo Speaks makes sense as each device makes calls to amazon for device status and other data.
I really hope that they can track down the network connection bug because I feel very strongly that the random failures from Amazon requests are causing issues on the hub which are causing the slows.
1 Like
I’m not offended or otherwise wounded in any way. But, Hubitat team finds that code they labelled beta has a problem. Where is the punch line? 
Eh, funny is subjective.
2 Likes
This has definitely been an interesting read and fortunately something I have not experienced. I thought I would provide some data points about my setup in case it helps anyone or the HE staff in troubleshooting. I wouldn't say I have an overly complex/automated setup and most things happen with button presses or physical switches, doors opening/closing, etc. All of the following runs on a single hub.
235 Total Devices:
- 90 Zwave (55 Plus, 35 non-plus)
- 51 Zigbee
- 24 Lutron
- 70 Virtual and Other Devices
Since LAN/Web Services integrations were identified as possible culprits, here are mine:
- Lutron Telnet
- Denon AVR
- Onkyo AVR
- Network UPS - telnet to my NAS for UPS status every 10 seconds
- Ecobee Integration - 2 thermostats
- Lennox iComfort - 1 thermostat
- Alexa TTS
- MyQ Garage Door
- Twilio SMS - only a few a day
- NodeRed via web socket for Grafana logging and calendar integration
- HubConnect with SmartThings for 2 locks - I could never get my Schlage locks to pair with HE so left them on ST and use HubConnect to integrate.
Custom Code:
- 16 custom apps
- 17 custom drivers
2 Likes
I just upgraded to 2.1 and disabled chromecast and now my responsiveness is TOTALLY better. Kudos to the hubitat folks!
2 Likes
I removed Chromecast integration yesterday too...
I want to use it, but it was pretty discretionary so I thought I would just dump for now until the dust settles.
Hmmm, my hub seems to be auto rebooting now... I’ve seen it reboot on its own at least 3 times now...
Make a backup, do a soft reset(not hard reset or reboot), restore the database.
Thank you. Should have read that.
Last event I had was 630am - left the house at 8am and typically would of got a push that I left the geo-fence. Any way to stop these hub lockups ?
[app:322](http://192.168.1.140/logs/past#app322)2019-08-03 12:19:22.533 pm [error](http://192.168.1.140/installedapp/configure/322)java.sql.SQLException: Connections could not be acquired from the underlying database! (updateMembers)
[dev:262](http://192.168.1.140/logs/past#dev262)2019-08-03 12:19:22.495 pm [error](http://192.168.1.140/device/edit/262)java.sql.SQLException: Connections could not be acquired from the underlying database! on line 333 (generatePresenceEvent)
[dev:262](http://192.168.1.140/logs/past#dev262)2019-08-03 12:19:22.470 pm [error](http://192.168.1.140/device/edit/262)java.sql.SQLException: An SQLException was provoked by the following failure: com.mchange.v2.resourcepool.ResourcePoolException: A ResourcePool cannot acquire a new resource -- the factory or source appears to be down. on line 110 (extraInfo)
[dev:261](http://192.168.1.140/logs/past#dev261)2019-08-03 12:19:22.346 pm [error](http://192.168.1.140/device/edit/261)java.sql.SQLException: An SQLException was provoked by the following failure: com.mchange.v2.resourcepool.ResourcePoolException: A ResourcePool cannot acquire a new resource -- the factory or source appears to be down. on line 370 (generatePresenceEvent)
