Hub process slowdown after several days

That's an interesting thought that I think I'll take a look at. I also have multiple motion sensors in my bedroom/master bath that I'd like to explore that with.

It has been more reliable for me without hammering the logs with the motion sensors. I started using it because the lights kept turning off when I was working in the garage.

That's one of my uses. Lights stay on whether i'm in the shower. on the toilet, or at the sink. Single triggering device, single simple rule. Win!

2 Likes

Did not know that it existed. Ya learn something every day! Thx @Ken_Fraleigh and @bjcowles.

Now... once more unto the breach...

1 Like

Forgot about this. Perfect for some of my needs.
Nice catch. :smile:

You know this is my thoughts on it. I like hubitat owned it a few months now. To always blame it on 3rd party plugins I think is looking at it the wrong way. We all buy these hubs to run our automatons the way we want too. If the hub turns to crap for any reason it's not meeting the consumers needs. I really loved hubitat when I first got it then came the slow automatons the slow response time of the UI 30 - 60 seconds per click. I have my hub rebooting every few days which has helped alleviate some of these issues and caused other issues with my mesh so as far as I'm concerned it's not meeting my needs. I had better luck using ST cloud. I contacted support they helped make changes and blamed it on the 3rd party apps/drivers and closed the ticket. If Hubitat knows we are suffering why is no one officially addressing this issue? Local processing does no good if it takes longer than cloud processing due to these slow downs. I'm honestly ready to go back to ST I never had to contact there tech support for issues. Don't get me wrong they had outages but I was aware of them and aware they was working on them. It seems the only answer we get here is contact support and after a reboot it all runs good for a few days and the ticket is closed. I don't know about you but I'm not about to waste my time in this loop each time. Someone needs to officially address there is an issue if they have not already.

5 Likes

We have and continue to address "this issue". The issue is that people install bad custom apps and drivers on their hubs, and then expect us to fix that problem for them.

It's not possible for us to address the problems in user installed custom code. We have provided you the tools to diagnose which of these is the source of your problem. What you do about it is up to you. We've provided the power in this hub to do anything you want with it. That doesn't mean that the choices you make are good ones and will have good results. Most users, including those with very large systems, have no "hub slowdowns" at all. What's the difference between those with and those without? The difference is in the choice of custom things that you've put on your hub. Do you know those apps are solid? If you don't, then why would you expect us to protect you from yourself? We aren't the custom app police here.

We aren't going to devote support resources to people who won't help themselves by making good choices for what they do with their hub. We bend over backwards to help people who want the help, and who follow our advice about how to resolve their problem. But many who complain simply refuse to listen to our advice, and then go on to complain more. There's no helping some people.

Our advice is always the same: Selectively disable custom apps to see if the problems are resolved without that app running. By this means, it is possible to find the culprit, or culprits, responsible for your hub slowing down. If a person were to disable ALL custom apps and drivers, and still have a demonstrable issue, of course we would dig into it. However, in our experience, all of these hub slowdowns are coming from custom apps and drivers that haven't been designed properly. Look at @csteele's post above, where he discusses this in depth.

I should add that it is possible to design rules that get into infinite loop types of problems, and these too could cause a hub slowdown.

5 Likes

It is what it is, I guess.

That is exactly why I don't think Hubitat will ever get market share outside of the enthusiast community.

That is not a stance that will ever allow for wider adoption with consumers. Telling a customer - "well it works fine as long as you don't install anything on it" simply isn't going to work for an average consumer. Yes, I'm being a little facetious - that isn't exactly what Hubitat has said, but it is how it feels on the customer's side.

For the record, the last app I took the time to prove was slowing down my hub was Hubitat's Chromecast integration. Oh yeah, that's BETA though - so also not Hubitat's problem either I guess.

Anyway. I understand the company's stance, but I don't agree with it. I know - tough luck, buttercup.

But that is exactly why I won't recommend Hubitat to casual users over SmartThings, and why I'm migrating logic off of Hubitat as fast as possible and turning it into a zwave/zigbee/eventsocket/MakerAPI connector hub and nothing else as much as I can.

5 Likes

Don't project your use of the hub onto to "average consumers". The 'average consumer' does not install custom apps. To take your point to its logical conclusion, to support the broader audience of home automation newcomers, we wouldn't allow any custom code at all. No MakerAPI, none of the things that allow you to hang yourself. Just easy to use automations in an environment with no coding and no custom apps or drivers.

Quite wrong in my opinion. I think if you guys collected telemetry on your users, you would know that's wrong, too. Casual users are the ones that install ALL/ANY APPS. They are the users that hit install on every box that pops up without regard of loading or impact.

All they know is - I have hardware widget X and want it in Hubitat. They search, find a driver or app (regardless of source) and install it with little further thought.

And then you would have probably 5 customers and be defunct. Your choice I guess there....

2 Likes

The inference is that we would have to police all of these apps and drivers. And only allow 'approved' ones to be installed. At that point, what's the difference between that and a closed platform?

Which do you want, openness with the opportunity to screw yourself, or closed and protected?

1 Like

Incidentally, from what we can tell, a fraction of 1% of our customers have the problems discussed in this topic. So it does not appear to be something that affects even our "average" consumer.

2 Likes

@bravenel - I understand where you are coming from it is hard to support 3rd party apps and integrations. I know you can't afford to keep throwing man hours at each issue on a daily basis. I think there is more of an underlying issue though and just feels like Hubitat's stance is well you did it to yourself so your options are 1. Disable the custom apps that runs your home and loose all the functionality - That lead you to Hubitat in the first place! 2. Suffer the consequences of having the lag through all these customization's are causing and keep your automations.

I like the hubitat community and I love the ability to customize the hub as I see based on the hard work your team and the community developers has put in to it.

I think the best thing to do is team up and tackle these things as they become issues. I know with ST I had twice as many custom apps and drivers and never seen the hub slow down. I'm no developer nor do I claim to be an expert on the issues at hand but I do have 13 years in the IT field and I would not get far if I took the same stance.

You did it to yourself now suffer or change it's your choice?

I don't think I would get very far like that I'm not sure I would keep my job very long. Anyway my point is I love hubitat and the community but we need help whether it be through better documentation or better team work to drill down the exact cause.

You say you provided us the tools to diagnose is there documentation on this somewhere I'm missing?

I have several RM4 rules but nothing overly complicated as I'm not really understanding RM4 yet to be overly complicated in my rule designs.

Maybe that is the issue. I watch your Youtube post all the time and that helps some but really it just seems to hit on what is coming and not addressing ongoing issues. Let's work together and make Hubitat the best of the best is the way I feel I would hope the Hubitat team would feel the same way!

Don't get me wrong I see you guys active in here all the time and I'm sure it takes up a lot of your free time and we really appreciate the teams hard work and dedication but we never seem to make any real progress and some of the responses come off as if it is Hubitat against the users honestly.

3 Likes

My issue with this is the basic logic behind this so called tool to diagnose issues. Yes, in theory, if you disable all custom apps and if everything is running good, Hubitat can shift the blame on custom apps, and that's the end of that support ticket.

But let's take a closer look into this - I've already touched on this before. When I first got started with Hubitat, I had close to 13 custom smartapps running. A mix of LAN based and Cloud. These weren't some super advanced smartapps with niche uses cases.. One was a way to integrate Echo dots (super cheap devices) into Hubitat to enable TTS capabilities. Not everyone has an expensive Sonos system. An average consumer might search on your forums, see that it's possible to make their cheap Alexas have TTS capability and blindly install the smart app.

Two days later, their hub is slowing down and the consumer has no idea why. NOW, if that's the only smart app this person installed, it's all good - super easy to diagnose. What if they installed multiple smartapps in the same short period?

You could disable one by one. Or all at once. Disabling 12 smartapps one by one and waiting however long to see if your issues pop up is just.. What about those niche cases where it's a combination of two specific smartapps that causes the issue? I'm sorry but, is this really the best tool you guys can offer?

I'm going to have to disagree here. Think of the iPhone - everyone loves installing apps. Whether it's to integrate their brand new shiny LG TV into their hub, or get their Alexas to talk or it's to get insights on power usage in their home - people like to get more out of their purchase. I'm not some home automation guru, I would say I'm a moderately savvy consumer with lots of different kinds of smart home equipment.

To assume that the majority of your consumers are just going to solely rely on Rule Machine and default integration is hubris..

Imagine if installing an app on your iPhone slowed the phone down to the point that it was useless or crashed it overnight? Well, you don't have to imagine - this was basically Android until Kitkat/Lollipop.

How do you plan on working around this? RM is an integral part of Hubitat. Let's take your assumption that the average consumer is just going to use RM and stick to the default integrations. The average consumer isn't going to spend hours on the forums trying to learn best practices for rules design. How is the consumer even going to know that it's their poorly designed rule that's causing everything on their hub to slow down to a crawl?

Most importantly - WHY is a poorly thought out rule bringing down the entire hub to the point that everything from the WebIU to automations are slowed down?

I, like probably most others here came from Smartthings. Smartthings got around poorly designed code by offloading it all to the cloud. The people installing smartapps there weren't all IT gurus and sysadmins, a lot were just regular people that just picked up the hub from BestBuy.

Please note that I actually really do love Hubitat as a platform, and there's really nothing else like it out there. I just wish there was a way for people like me that are moderately technically inclined to get the best out of it without having to rely on external systems.

2 Likes

One thing I think is causing part of my problem. In RM I had a couple rules that could get triggered several times quickly. And with delays or waits in them it was causing multiple timers to be going at the same time.

Support recommended breaking it down in multiple rules, one for each device so to speak. I have been doing that and it seems to have made a difference. Still evaluating, but looking good so far.

I don't have any idea what you think might be an "underlying issue". It's called bad code. It comes in all stripes and flavors, but mostly it has to do with how apps connect to cloud or LAN resources. For example, using synchronous HTTP instead of asynch. @csteele's post discusses this in some detail.

Just as was and still is the case with PCs, there is all sorts of bad code out there. How do you as a consumer of a PC deal with this? You learn to only install code from trusted sources, not just anyone you can find with Google. Even then, sometimes bad code still causes horrible problems with PCs, allowing hacks and viruses to infect the PC. Are these problems something that Microsoft can deal with? On one level, yes, they constantly strive to improve the platform, but on the larger scale, no, they can't police what people put on their PCs. They can throw up a warning, are you sure you want to install this? But then, when you do, it's on you, not them. How is that any different that what is happening with Hubitat?

We encourage you and all users to critique the code you are using, to report publicly that some app is causing problems. Usually, the app author will address these problems, but not always.

To selectively disable an app, there is a grayed out X in the upper right hand corner of the Apps page. Click on it and a list of check boxes will appear just left of the apps. A similar tool exists on the Devices page.

Also, apply basic common sense: If your hub was running fine, and suddenly starts slowing down, what did you most recently install? Perhaps start with that app or driver. Check things out after you install them.

And it turned out that this app had serious problems, and brought hubs to their knees. We looked at the code, and it was a hot mess.

This isn't a wise thing to do. What can I say? If you install 5 bad apps in quick succession, and things break, what should you do?

At this time, yes it is. We don't have a magic wand here to wave over an app and know that it's good or bad.

The vast majority of our customers have not installed many custom apps. It's not hubris, it's our attempt to provide a complete set of apps to cover most automation scenarios. The very nature of a customizable hub means that there are infinite number of things that one might dream up to do with it, and obviously we can't implement an infinite number of apps. But, the 90/10 rule applies, and the vast majority of our customers don't need exotic apps.

This is the risk of providing a tool with power. We can't teach everyone how to write good rules. The community is constantly coming to help people figure things out. We actually don't expect the "average" consumer to use Rule Machine --> it is intended to provide power for more advanced users. Most basic automations can be accomplished with other built-in apps: Simple Lighting, Motion Lighting, Notifier, etc. Certainly, we can and will do a better job providing guidance and help for beginners.

But, when you are past being a beginner, and you take off the training wheels, is it realistic to expect that someone can't write a bad rule or write bad Groovy code? You can write an infinite loop in any programming tool, and in every programming environment that is a bad thing. I could bring SmartThings to its knees, at least my connection to it, with an infinite loop. Sure, in a multi-user cloud system an individual user process can be isolated and the rest protected from it. But Hubitat is a single user system and its single user can make mistakes that bring the hub to its knees. How can you protect against that?

OK, we've supported hundreds of devices and brought forward dozens of apps in the 20 months since Hubitat launched. We don't plan to stop. There will always be more devices, more integrations, etc. that can be supported. We've been running hard at it, and arguably have been more responsive and more productive than any of our competitors. That's the best we can do, and we aren't ashamed of the state of affairs.

4 Likes

Yup. I agree - I've been given suggestions by Hubitat staff both in the community forum and by email from support@hubitat.com

I think the slowdown in my hub can be attributed, in part, to badly written rules. I turned on logging for each rule, ran it, and changed it until there were no errors while it ran. I also removed a bunch of rules and virtual devices that I wasn't using.

At ~48 hours now, my hub has not had any slow downs.

But there is one rule that is still throwing errors - I don't know why that is, and I am hoping you or someone else can tell me what I've done wrong so I can fix it.

Thanks!

1 Like

I am fully aware that my rules might be the issue, but I have a difficult time finding examples to match what I would like to accomplish. It would be a really nice option to have a wiki or list or resource of example rules and how to structure them, that are "good" rules, for users to create rules. I think RM4 is great, but it is easy to create a problem rule, and not know it. Maybe even a best practices for RM. Here is an example of a motion lighting rule I have made (about 9 similar running on the hub) -

1 Like

That looks like a good rule. Does it give you any problems?

Correct - it definitely wasn't a wise thing to do in retrospect. I actually installed two of the worst offenders - Echo Speaks AND the influxDB integration at once. Things went from "oh yeah this is cool" to "what the hell" real quick and I wrote an irate E-mail to Bobby. But my point was.. it does happen. If it happened to me, it could happen to someone else with similar needs. I have limited time, so I like to get things done all at once. It's poor practice ofcourse, but it is what it is. People like me are then more likely to raise a stink about it on the forums or Amazon reviews and blame the hub, which sways the opinion of others into thinking Hubitat sucks.

I completely understand your outlook on this whole issue - you guys built a solid platform and are trying your best to achieve a delicate balance between satisfying the enthusiasts (usually the loud ones) to the average Joe (the guys that bring in the money).

My only real point is that if you guys intend on keeping the moderately savvy/enthusiasts on your platform in the long run - there needs to be a fundamental shift in your thinking on how to cater to us.

Might I suggest reconsidering a "Pro" hub with a beefier CPU/Memory etc? I did a disassembly of the C5 hub, and it looks like the SoC is an Amlogic chipset meant for use on Smart devices. Not really sure where the performance stands vs the S905 that was used on the C4, but I'm sure it's not all that better than a Raspberry Pi 3.

Personally, I would LOVE to see Hubitat available as a software package like homeseer where the Pro crowd could just install on their own hardware with near unlimited resources. Now whether that's a sound business decision or something that aligns with your future plans for the platform.. I don't know.

edit: I suppose this hinges on the assumption that beefier hardware would solve slowdown issues by brute forcing against poorly designed code, which I'm not entirely sure of.