2.3.9.147 - C7 - Virtual switches not responding to automations

Ok, I set a longer delay, and it did show up in scheduled jobs:

No change in behavior. But wow, do I have a rather long list of scheduled jobs. Could that indicative of the issue?

Resulting logs from the test:

The logs are from the same version of the rule, which is now updated to 60 seconds instead of 10. in my "apps" page, Basic rules are listed as "Basic Rule-1.0"

I went to "Past Logs" filtered on errors, and it's showing only errors from the Ecobee app, and one from the Ecobee device itself. No others I'm afraid:

Something is not right with your rule, the act should look like mine, "Wait XX seconds"
Maybe delete it and make a new one?

image

I deleted and created a new Basic rule. Used a different device. Logs look more like yours now. Still no auto-off. I guess the issue isn't that devices aren't responding to rules. The rules are turning them on successfully. They just won't turn them back off:

Yes, looks like the delayed action on the rule never even tries to fire, the schedule job is getting killed for some reason.

Go to Logs > Device stats
Click the display settings and enable all the columns.
Should be sorted by % of busy be default (highest at top).

Post a screenshot of the first page including the stats above the table.
Do the same for the App stats tab

Usually if delayed actions never fire it is from other apps/devices consuming the hub resources.

Device Stats:

App Stats:

Looks like you rebooted and then got the app stats? Will need to see another one after the hub has been running for a bit.

When you reload the scheduled jobs tab in the logs page, does it show any jobs scheduled in the past ?

I have about 60 entries in the scheduled jobs list. All except 9 of them show a date in the past in the "Next Run" column.

@hubitrep good catch this seems to be a recent issue that has popped up for some people. The jobs are getting hung up and not executing for some reason.

I wonder would would happen if you just delete all the old ones? Nothing bad could happen, but they might just come back again and get stuck. Worth a try.

I think @mavrrick58 was having this issue also, not sure if there was an actual resolution to it.

1 Like

It was slightly different as I had the problem as part of a migration from my C8 to a C8 pro.

The solution in my case, ended up being using a backup from the C8 prior to migration after the migration was done.

Did this just happen out of the blue with no DB restore being done?

EDIT : To add more info after reading the whole thread.

To look for the issue I was experiencing you may want to look at the logs and then the tab for "location events" or "Hub Events". Just after a restart there was a error that appeared for the scheduler.

Not necessarily so recent, I remember bringing it up to @bobbyD and friends when it happened to me some months ago. But obviously rare.

IIRC a soft reset and restore from an older backup did the trick

I believe i tried this and it didn't seem to fix it in my case.

I believe so. The logs show delayed-off commands working one minute, then not the next. I had been on firmware 2.3.9.141 for about two weeks at the time this happened.

Thanks a ton jtp10181 for all the guidance and time helping out on this.

Ok I got a lead on a possible fix for this. First look in hub events for an error as shown here:
None of the basic rules apps working - they use to, but not anymore - #12 by bahree

Then look in past logs filed under "sys:" with "Hub" as the device for an error which should reveal the app / device ID number.

Then you can use an endpoint to remove the broken jobs
Apps: None of the basic rules apps working - they use to, but not anymore - #16 by bahree
Devices: None of the basic rules apps working - they use to, but not anymore - #17 by bobbyD

This is the endpoint to use and replace ### with the app/device number from the error.
http://your-hub-ip/hub/advanced/deleteAppJobs/###
http://your-hub-IP/hub/advanced/deleteDeviceJobs/###

Update: Found where they saw the app ID, it was in the normal logs, not sure what your error will look like though: None of the basic rules apps working - they use to, but not anymore - #9 by bahree

3 Likes

I don't see any scheduler errors in hub events. There are also no errors whatsoever in past logs, other than the Ecobee ones I mentioned before.

Thanks again for all your work over the past few days. I learned a ton about my hub!

The soft reset and restore got me back up and running, I'll be keeping an eye on that scheduled job list and the post-reboot hub event logs from here on out.

FYI, that should be the same thing as doing a database rebuild (but with more steps) which I think you said up above that you had already tried.

Yep, several times actually. The difference (I think) is that I restored a version from a previous backup taken before the issue occurred.

3 Likes

Yup, that would do it, using an older backup.