I can't be 100% certain that nightlyCleanupJob is responsible but this always starts somewhere after 2:00 AM and continues until a bit after 3:00 AM and nightlyCleanupJob is set to run at 2:15 AM so it is the smoking gun.
Every night for most of an hour my hub isn't dealing with ZWave devices in any sort of reasonable time period. It can sometimes take 2 minutes for it to actually trigger a zwave device from the time it was switched on the dashboard (or directly from the device page). The worst part is it stores up all the requests and triggers them later. (I finally had time tonight to confirm that Zigbee devices weren't affected.)
- This is actually a safety concern: There are multiple devices controlled with zwave momentary switches like the main access gate, garage doors, etc. A momentary switch is always activated to ON so multiple queued key presses is like pressing the gate/garage button at random. I've had the gate try to close on us as we go through it. I've come home to find the gate partially open. Plus the access lights are on zwave switches and backing into a long driveway under a huge oak tree at night is like backing into a blackhole.
This may have been happening for a long time but in the last year I've been having to go pickup my wife in the early morning hours and it is very often during this time period.
My zwave mesh is setup so there are no devices that require multiple hops to be reached. Any devices that would need to go through a repeater are on a different hub. The momentary switches for the gate and garages are 12 feet from the hub with nothing to interfere with radio between them.
What can be done to troubleshoot or chase down what is causing this? I'm surprised I couldn't find a way to list what commands are waiting in queue. I was also very surprised that I couldn't find any posts referencing "nightlyCleanupJob" except a single one that included it in a list of every scheduled job in the hub.
This has to be fixed or I'll have to scrap this system.
C8-Pro - platform version 2.4.4.156 (problem has existed through multiple version upgrades)
Let's go ahead and get this said so the rest of the thread can focus on what you believe to be an issue. If you think
then
is precisely what you should do immediately. Hubitat is a tool. The consequences of how you use it is on you. On its own, it cannot create a hazardous situation until you put it to use.
That said, you have a couple of clues in your post that indicate your situation is a little complex. You have multiple hubs? You have "setup" your Zwave mesh in a particular way? Tell us more. Also, screenshots of logs demonstrating the issue would be helpful.
Perhaps this is somehow totally coincidental, but looking at my *nightlyCleanupJob" schedule, it's set for 0314, which is when I've scheduled my (daily) Local Backup time.
I can't speak to any performance weirdness around that time since we're always asleep and nothing at all is happening in the house.
I have a C8 pro (2.4.4.151) and a rule to reboot whenever the hub logs indicate "z-wave" crashed, it has fired ~3 times in the last month. My Z-wave device table is clean of ghosts. Z-wave on my old C7 was much more reliable.(same devices) . My routine for sunset/sunrise misses random device on/off commands once or twice per week. Whateve is causing it will likely get fixed in due time.
Going to try a scheduled reboot Q2days and seei if t helps.
I have to say I think it is apparent that ANY system that effects things in the real world should have a methodology that ensures actions will happen in something close to a "now" time frame.
Either actions happen in somewhat of a reasonable time frame (a few seconds but never more than 5 seconds) or the requested action should be discarded and logged as failed (with perhaps an option to perform an "else" action on failure).
Storing a trigger (or multiple triggers) for well over a minute when the rest of the system is completely responsive (dashboards, zigbee, ...) is not the actions of a product ready to control anything in the real world. Actually it is just asking for a legal challenge.
So even though the response was rather snide and dismissive I am scrapping the system for Home Assistant. It may require more work but I won't be having a gate close on the side of a vehicle as it is going through and causing a few hundred in paint damage.
A system that acts in the real world must be predictable to some level and that is not a problem that I've ever seen in Wink, Vera, or heard of in Home Assistant.
Obviously the "nightlyCleanupJob" needs to be troubleshot as a lightly loaded system like mine having this kind of issue shows that there is a clear logic problem. If the hub can send zigbee commands during the cleanup then there is no reason it shouldn't transmit z-wave except there is a programming issue.
My suggestion is for programming to set the job to run during the day and work on it until it stops even the possibility of queuing z-wave tasks for ridiculous times. Ever. If it is reliable enough to run during a time you expect the system to be used then it is ready for production.
And as to how to setup z-wave so there are no hops: Put your hub is a very central location and look at the Settings>Z-Wave Details and adjust the location of the hub and devices that have optional locations till the routes have no hops for anything you care about. In my case I put an insulated box (cheap foam cooler) dead center in the attic, hooked up two ducts connected to grills in two different rooms with a muffin fan for cooling. The obviously snide question about this did nothing constructive as I already knew I have no z-wave communication issue beyond the fact the hub isn't sending the commands when the "nightlyCleanupJob" is running.
(BTW - I highly recommend mounting security systems, NVRs, DVRs in the same fashion as if a burglar/intruder can't find it they can't remove evidence of their actions.)
Once again . . .
Your comments are not helpful or constructive. Why are you here?
For security (which is the ultimate in safety) there are numerous reasons that you'd want to have a gate that will try to close no matter what is in it.
Part of the safety of the design is having the triggering of the mechanism so it goes when triggered.
I made the obvious mistake of assuming a commercially available system would operate in either of the categories of "it activates when triggered" or rarely "it doesn't activate when triggered". Either of those is safe because you look at it and deal with the condition.
The completely ridiculous option of "I'll save up all the commands for an indeterminate period of time and then do something that no one would ever want" is on no one's request list. The logic of how to avoid this in a system that has a real time clock chip for reference is trivially simple. It is sloppy code design and dangerous.
So . . .
The system has a logic problem with how it is receiving, queuing, and then finally transmitting z-wave commands. I have no belief that anyone is trying to understand the issue or going to resolve it. Because "it only happens during one hour of the day when most people are asleep" it appears there isn't interest in addressing it.
(I've seen this with Hubitat before when I found a completely repeatable migration issue and no one wanted to look at the step by step screenshots of what was happening. One such situation is a "frustrating occurrence". The second consecutive documented and repeatable issue that is not addressed is a pattern and I'm not ignoring it.)
That sounds extremely dangerous, I hope a child or animal doesn't get caught in this deadly contraption.
Hubitat does not claim to be a security system, and in fact they have often said that it was not to be used for mission-critical or lifesaving items. If you want a monitored professional system, that is more like what you should be purchasing. There is no way a $150 hub can be the same as a full monitored system.
I would also ask, how often are you closing this gate at exactly 3am when you are now aware that the hub is not responsive at that time? Why not just close the gate before the cleanup? Are people just trying to ram their way into your yard at exactly 3am regularly? That sounds like you have deeper issues rather than a hub.
The latest 2.5.0 have some improvements in the cleanup sessions and other improvements for delays. Could these possibly be the issue you are having? Have you tried the latest release?
I went back and read that. I don't see screenshots, unless you deleted them? Not sure why you would delete them if they were there?
Maybe staff just can't duplicate what you experienced, that is what I gleaned from that thread anyway. Two different staff members replied to you there, and other experienced community members made some suggestions to try as well.