Oh boy, that integration IME is highly problematic, it leaks memory and chews CPU like nothing else I’ve seen!
On the current firmware (.148) I’m not seeing the performance fall off a Cliff below 220 MB free anymore. As a result I’ve dropped my reboot threshold to 150 MB free, and I’ll see how that goes.
1 Like
What performance were you seeing when the memory got low? I've been noticing laggy and missed automations etc as memory drops around 200 or so.
A week ago I went and cut down the event history in all my devices and free memory after a reboot is up over 400. 4 days later I'm still in the 300s.
Not sure if the history helped that or this last update.
1 Like
Honestly I've dropped to 85k and didn't see any real issues
2 Likes
The same as you, lag and automations falling to run on time.
1 Like
Well, I almost made it a full week without any issues. But woke up this morning and my Z-wave network failed again. Odd thing was there were no problems with free memory before it crashed. A reboot fixed it but definitely something still not quite right. HE was rock solid before.
1 Like
Do you know how much memory was left before it crashed?
After seeing the Z-wave down it was the first thing I checked. Odd thing was the memory was over 400 so I don’t think this time it was a memory issue.
Memory on my C-5 gradually sunk to just over 200 over the last three days. At that point it started getting laggy, rules not running or not running correctly, etc. I rebooted, and memory has been over 500 since. We'll see what tomorrow brings. I do think that something is going on, though. I have a rule that reboots on the 1st of each month, just because it's a good idea to reboot things now and then. But other than that and upgrades I never rebooted and everything was solid.
2 Likes
Same, just had a week of uptime and it's frozen again now.
You are better off having it reboot when it actually needs it.
2 Likes
I have rules that do that also.
1 Like
Are you only looking at Memory, or do you also look at CPU and response time to determine if the hub needs to be rebooted. I use a dashboard like this in Grafana to give me all of those metrics. The hub is just responding to what you are asking of it. If you are running low on memory, I would try to see what is causing the memory to be consumed. Look at apps and rules that may be hard on memory. As @dJOS has said it is better to have it reboot when it is needed. Rebooting just because memory gets low just mean the hub has to spend time adding data back into memory for processes that are running. The hub will do cleanup for items that are not in use. As time passes the memory that is consumed will stabilize and the line will be come flatter and decrease slower. this is a process call garbage collection.
Something else I found is that memory is a metric that updates much more frequently then CPU load. In theory you can collect it every few seconds, I have changed mine to collect data every min via Node-Red instead of 5 min.
1 Like
Where are you getting the response time data?
It has been a long while since I setup this method to capture this stuff. It was before @thebearmay made his hub info driver. It is a flow in Node-Red and upon inspection what it looks like the flow is doing to get the response time is doing a call to (http://hubip/driver/list/data) and measuring the response time.
The flow started as a shared flow titled (Hubitat Performance). I added a few things to expand it before the hub info driver was created. It looks like it does use the Hub info driver to get the CPU Percent used and the DB size.
Incase you are interested the flow is below, but there is some setup involved and this outputs to InfluxDB to store the data.
[{"id":"e990e72.c9c5718","type":"subflow","name":"Check Temp","info":"","category":"","in":[{"x":50,"y":30,"wires":[{"id":"d926818d.0004f"}]}],"out":[{"x":820,"y":280,"wires":[{"id":"55d580a1.46db","port":0}]}],"env":[],"color":"#DDAA99"},{"id":"dcbd5b90.410ac8","type":"http request","z":"e990e72.c9c5718","name":"Get Temp","method":"GET","ret":"txt","paytoqs":"ignore","url":"","tls":"","persist":false,"proxy":"","authType":"","x":340,"y":100,"wires":[["ca7caa8d.555ee8","87915bca.fa7cf8"]]},{"id":"d926818d.0004f","type":"function","z":"e990e72.c9c5718","name":"Build Tempurature URL","func":"msg.url = \"http://\" + flow.get(\"$parent.hubIP\") + \"/hub/advanced/internalTempCelsius\"\n\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","x":150,"y":100,"wires":[["dcbd5b90.410ac8"]]},{"id":"55d580a1.46db","type":"change","z":"e990e72.c9c5718","name":"Add tempValue","rules":[{"t":"set","p":"tempValue","pt":"msg","to":"payload","tot":"msg"}],"action":"","property":"","from":"","to":"","reg":false,"x":800,"y":100,"wires":[[]]},{"id":"ca7caa8d.555ee8","type":"switch","z":"e990e72.c9c5718","name":"Did we get a value > 0?","property":"payload","propertyType":"msg","rules":[{"t":"gt","v":"0","vt":"num"}],"checkall":"true","repair":false,"outputs":1,"x":570,"y":100,"wires":[["55d580a1.46db"]]},{"id":"87915bca.fa7cf8","type":"debug","z":"e990e72.c9c5718","name":"URL output","active":true,"tosidebar":true,"console":true,"tostatus":true,"complete":"payload","targetType":"msg","statusVal":"payload","statusType":"auto","x":520,"y":180,"wires":[]},{"id":"43ac6c1.6e31c94","type":"subflow","name":"Check Mem","info":"","category":"","in":[{"x":50,"y":30,"wires":[{"id":"aae357be.dd5bc8"}]}],"out":[{"x":820,"y":280,"wires":[{"id":"6aeb7ef4.8dfae","port":0}]}],"env":[],"color":"#DDAA99"},{"id":"68199469.2e8c1c","type":"http request","z":"43ac6c1.6e31c94","name":"Get freeOSMemory","method":"GET","ret":"txt","paytoqs":"ignore","url":"","tls":"","persist":false,"proxy":"","authType":"","x":370,"y":100,"wires":[["6888c7ab.da6ba8"]]},{"id":"aae357be.dd5bc8","type":"function","z":"43ac6c1.6e31c94","name":"Build Free Memory URL","func":"msg.url = \"http://\" + flow.get(\"$parent.hubIP\") + \"/hub/advanced/freeOSMemory\"\n\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":150,"y":100,"wires":[["68199469.2e8c1c"]]},{"id":"6aeb7ef4.8dfae","type":"change","z":"43ac6c1.6e31c94","name":"Add freeMemValue","rules":[{"t":"set","p":"freeMemValue","pt":"msg","to":"payload","tot":"msg"}],"action":"","property":"","from":"","to":"","reg":false,"x":810,"y":100,"wires":[[]]},{"id":"6888c7ab.da6ba8","type":"switch","z":"43ac6c1.6e31c94","name":"Did we get a value > 0?","property":"payload","propertyType":"msg","rules":[{"t":"gt","v":"0","vt":"num"}],"checkall":"true","repair":false,"outputs":1,"x":590,"y":100,"wires":[["6aeb7ef4.8dfae"]]},{"id":"9d92739f.d0d64","type":"subflow","name":"Check Time","info":"","category":"","in":[{"x":60,"y":100,"wires":[{"id":"e7bb410b.23be4"}]}],"out":[{"x":1050,"y":100,"wires":[{"id":"1b1bcd68.5341b3","port":0}]}],"env":[]},{"id":"e7bb410b.23be4","type":"time-range-switch","z":"9d92739f.d0d64","name":"Am I Awake?","lat":"35.66797","lon":"-78.62354","startTime":"07:30","endTime":"22:30","startOffset":0,"endOffset":0,"x":180,"y":100,"wires":[["6a1a35f2.e1ef9c"],["c2b68233.30716"]]},{"id":"74587240.2245ec","type":"time-range-switch","z":"9d92739f.d0d64","name":"Maintenance Window?","lat":"35.66797","lon":"-78.62354","startTime":"02:00","endTime":"03:00","startOffset":0,"endOffset":0,"x":550,"y":100,"wires":[["22e93162.f1ae3e"],["b1d549cf.166e38"]]},{"id":"22e93162.f1ae3e","type":"change","z":"9d92739f.d0d64","name":"Yes","rules":[{"t":"set","p":"inMaintenance","pt":"msg","to":"true","tot":"bool"}],"action":"","property":"","from":"","to":"","reg":false,"x":750,"y":50,"wires":[["1b1bcd68.5341b3"]]},{"id":"b1d549cf.166e38","type":"change","z":"9d92739f.d0d64","name":"No","rules":[{"t":"set","p":"inMaintenance","pt":"msg","to":"false","tot":"bool"}],"action":"","property":"","from":"","to":"","reg":false,"x":750,"y":150,"wires":[["1b1bcd68.5341b3"]]},{"id":"6a1a35f2.e1ef9c","type":"change","z":"9d92739f.d0d64","name":"Yes","rules":[{"t":"set","p":"awake","pt":"msg","to":"true","tot":"bool"}],"action":"","property":"","from":"","to":"","reg":false,"x":350,"y":50,"wires":[["74587240.2245ec"]]},{"id":"c2b68233.30716","type":"change","z":"9d92739f.d0d64","name":"No","rules":[{"t":"set","p":"awake","pt":"msg","to":"false","tot":"bool"}],"action":"","property":"","from":"","to":"","reg":false,"x":350,"y":150,"wires":[["74587240.2245ec"]]},{"id":"1b1bcd68.5341b3","type":"change","z":"9d92739f.d0d64","name":"Adjust Fields","rules":[{"t":"delete","p":"headers","pt":"msg"},{"t":"delete","p":"starttime","pt":"msg"},{"t":"delete","p":"endtime","pt":"msg"},{"t":"delete","p":"redirectList","pt":"msg"},{"t":"delete","p":"responseCookies","pt":"msg"},{"t":"delete","p":"topic","pt":"msg"},{"t":"delete","p":"payload","pt":"msg"},{"t":"set","p":"reboot","pt":"msg","to":"false","tot":"bool"},{"t":"delete","p":"url","pt":"msg"},{"t":"delete","p":"responseUrl","pt":"msg"}],"action":"","property":"","from":"","to":"","reg":false,"x":930,"y":100,"wires":[[]]},{"id":"588f2940.1ba848","type":"subflow","name":"Check Page","info":"","category":"","in":[{"x":40,"y":80,"wires":[{"id":"e1a197b2.ce6208"}]}],"out":[{"x":1150,"y":80,"wires":[{"id":"9156ff4a.f5a61","port":0}]}],"env":[]},{"id":"ed4d8669.d3fe48","type":"http request","z":"588f2940.1ba848","name":"Get App List","method":"GET","ret":"bin","paytoqs":"ignore","url":"","tls":"","persist":false,"proxy":"","authType":"","credentials":{},"x":500,"y":80,"wires":[["d5b40c47.eec4a"]]},{"id":"d5b40c47.eec4a","type":"change","z":"588f2940.1ba848","name":"Add endtime","rules":[{"t":"set","p":"endtime","pt":"msg","to":"","tot":"date"}],"action":"","property":"","from":"","to":"","reg":false,"x":670,"y":80,"wires":[["82fc59e4.931628"]]},{"id":"e1a197b2.ce6208","type":"change","z":"588f2940.1ba848","name":"Add starttime & name","rules":[{"t":"set","p":"starttime","pt":"msg","to":"","tot":"date"},{"t":"set","p":"displayName","pt":"msg","to":"Driver List Data","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":190,"y":80,"wires":[["a81ec577.c0b588"]]},{"id":"9156ff4a.f5a61","type":"function","z":"588f2940.1ba848","name":"Do Math","func":"msg.responseTime = msg.endtime - msg.starttime;\nmsg.responseTime = msg.responseTime / 1000;\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","x":1040,"y":80,"wires":[[]]},{"id":"82fc59e4.931628","type":"change","z":"588f2940.1ba848","name":"This is not a test.","rules":[{"t":"set","p":"test","pt":"msg","to":"false","tot":"bool"}],"action":"","property":"","from":"","to":"","reg":false,"x":860,"y":80,"wires":[["9156ff4a.f5a61"]]},{"id":"a81ec577.c0b588","type":"function","z":"588f2940.1ba848","name":"Build URL","func":"msg.url = \"http://\" + flow.get(\"$parent.hubIP\") + \"/driver/list/data\"\n\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","x":360,"y":80,"wires":[["ed4d8669.d3fe48"]]},{"id":"33ae8b6d.804ef4","type":"comment","z":"588f2940.1ba848","name":"URL's","info":"was /installedapp/list\n\nnow /driver/list/data\n\nAlso update displayName in the change node","x":140,"y":160,"wires":[]},{"id":"dbd54f40a5843152","type":"tab","label":"Hubitat Performance (LAN) C-7","disabled":false,"info":""},{"id":"b1f9ef73439ddf0e","type":"inject","z":"dbd54f40a5843152","name":"5 mins.","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"300","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":100,"y":120,"wires":[["26766660db0ddb14"]]},{"id":"7de67d917450c090","type":"http request","z":"dbd54f40a5843152","name":"Reboot Hub","method":"POST","ret":"obj","paytoqs":"ignore","url":"http://192.168.86.35/hub/reboot","tls":"","persist":false,"proxy":"","authType":"","x":1330,"y":590,"wires":[[]]},{"id":"8fbdef7804250603","type":"pushover api","z":"dbd54f40a5843152","keys":"b3ffec8b.aa42d","title":"Hubitat Watchdog","name":"Alert","x":1210,"y":350,"wires":[]},{"id":"a5878149b5905157","type":"delay","z":"dbd54f40a5843152","name":"YES: 1x/Hour","pauseType":"rate","timeout":"5","timeoutUnits":"seconds","rate":"1","nbRateUnits":"1","rateUnits":"hour","randomFirst":"1","randomLast":"5","randomUnits":"seconds","drop":true,"outputs":1,"x":920,"y":310,"wires":[["84177b2eca1afc27"]]},{"id":"f17e0521a9205be0","type":"delay","z":"dbd54f40a5843152","name":"NO: Only Once","pauseType":"rate","timeout":"5","timeoutUnits":"seconds","rate":"1","nbRateUnits":"9","rateUnits":"hour","randomFirst":"1","randomLast":"5","randomUnits":"seconds","drop":true,"outputs":1,"x":920,"y":350,"wires":[["84177b2eca1afc27"]]},{"id":"e3431516fc30113d","type":"config","z":"dbd54f40a5843152","name":"Configuration","properties":[{"p":"criticalsForReboot","pt":"flow","to":"2","tot":"num"},{"p":"criticalsForRebootDuringMaintenance","pt":"flow","to":"3","tot":"num"},{"p":"hubIP","pt":"flow","to":"192.168.86.35","tot":"str"},{"p":"thresholdWarning","pt":"flow","to":"5","tot":"num"},{"p":"thresholdCritical","pt":"flow","to":"15","tot":"num"},{"p":"hubName","pt":"flow","to":"Hubitat","tot":"str"},{"p":"deviceId","pt":"flow","to":"1","tot":"str"},{"p":"supportsMemCheck","pt":"flow","to":"1","tot":"num"},{"p":"memCritical","pt":"flow","to":"75000","tot":"num"}],"active":true,"x":140,"y":40,"wires":[]},{"id":"ef79f90eab2ce497","type":"change","z":"dbd54f40a5843152","name":"Log Auto Reboot","rules":[{"t":"set","p":"payload","pt":"msg","to":"$.payload & \" CRITICAL! Rebooting Hub!\"","tot":"jsonata"},{"t":"set","p":"rebootType","pt":"msg","to":"automatic","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":740,"y":530,"wires":[["2692fe914c8774c9"]]},{"id":"6a1e92fb8324da8c","type":"function","z":"dbd54f40a5843152","name":"Crit","func":"flow.set(\"criticals\", flow.get(\"criticals\") + 1);\nmsg.severity = \"crit\";\nmsg.payload = \"Response time above CRITICAL threshold for Hub: \" + flow.get(\"hubName\");\n\nif (msg.inMaintenance) {\n msg.critsNeeded = flow.get(\"criticalsForRebootDuringMaintenance\");\n} else {\n msg.critsNeeded = flow.get(\"criticalsForReboot\");\n}\n\nif (flow.get(\"criticals\") >= msg.critsNeeded) {\n msg.reboot = true;\n flow.set(\"lastReboot\", Date.now());\n return msg;\n} else {\n return null;\n}","outputs":1,"noerr":0,"initialize":"","finalize":"","x":520,"y":370,"wires":[["ef79f90eab2ce497","127af81ced072146","bcd6df5dbe64eb02"]]},{"id":"4a5ce03dd38e4f0d","type":"switch","z":"dbd54f40a5843152","name":"Severity","property":"responseTime","propertyType":"msg","rules":[{"t":"lt","v":"thresholdWarning","vt":"flow"},{"t":"lt","v":"thresholdCritical","vt":"flow"},{"t":"gte","v":"thresholdCritical","vt":"flow"}],"checkall":"false","repair":false,"outputs":3,"x":380,"y":330,"wires":[["3d673a876386ca15"],["f14936919003e4db"],["6a1e92fb8324da8c"]]},{"id":"d5d0f53c0ac8b79a","type":"inject","z":"dbd54f40a5843152","name":"Manually Reboot Now","props":[{"p":"payload","v":"","vt":"date"},{"p":"topic","v":"","vt":"string"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":160,"y":590,"wires":[["df558db61db5dc63"]]},{"id":"26766660db0ddb14","type":"function","z":"dbd54f40a5843152","name":"Initialize","func":"if (typeof flow.get(\"criticals\") == 'undefined') {\n flow.set(\"criticals\", 0);\n}\nif (typeof flow.get(\"warnings\") == 'undefined') {\n flow.set(\"warnings\", 0);\n}\nif (typeof flow.get(\"clears\") == 'undefined') {\n flow.set(\"clears\", 0);\n}\nflow.set(\"lastRun\", new Date());\n//flow.set(\"criticalsForReboot\", 2);\n//flow.set(\"criticalsForRebootDuringMaintenance\", 3);\n\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","x":120,"y":180,"wires":[["3cd3f8618bb41f44"]]},{"id":"f14936919003e4db","type":"function","z":"dbd54f40a5843152","name":"Warn","func":"flow.set(\"warnings\", flow.get(\"warnings\") + 1);\nmsg.severity = \"warn\";\nmsg.payload = \"Response time above warning threshold for Hub: \" + flow.get(\"hubName\");\n\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","x":520,"y":330,"wires":[["20acd13f8f109755","bcd6df5dbe64eb02"]]},{"id":"df558db61db5dc63","type":"change","z":"dbd54f40a5843152","name":"Log Manual Reboot","rules":[{"t":"set","p":"payload","pt":"msg","to":"MANUALLY REBOOTING HUB","tot":"str"},{"t":"set","p":"rebootType","pt":"msg","to":"manual","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":740,"y":590,"wires":[["2692fe914c8774c9"]]},{"id":"1a564c951d89168f","type":"inject","z":"dbd54f40a5843152","name":"OK","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":110,"y":440,"wires":[["6688eb35926d5333"]]},{"id":"6be4020ba948d9e7","type":"inject","z":"dbd54f40a5843152","name":"Warn","repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":110,"y":470,"wires":[["b5068fc136def639"]]},{"id":"be611c757c9b982c","type":"inject","z":"dbd54f40a5843152","name":"Critical","repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":110,"y":500,"wires":[["76747c3895a25edc"]]},{"id":"6688eb35926d5333","type":"change","z":"dbd54f40a5843152","name":"","rules":[{"t":"set","p":"responseTime","pt":"msg","to":"$flowContext('thresholdWarning') / 2","tot":"jsonata"}],"action":"","property":"","from":"","to":"","reg":false,"x":185,"y":440,"wires":[["dfbf240ade6b5ea6"]],"l":false},{"id":"b5068fc136def639","type":"change","z":"dbd54f40a5843152","name":"","rules":[{"t":"set","p":"responseTime","pt":"msg","to":"($flowContext('thresholdWarning') + $flowContext('thresholdCritical')) / 2","tot":"jsonata"}],"action":"","property":"","from":"","to":"","reg":false,"x":185,"y":470,"wires":[["dfbf240ade6b5ea6"]],"l":false},{"id":"76747c3895a25edc","type":"change","z":"dbd54f40a5843152","name":"","rules":[{"t":"set","p":"responseTime","pt":"msg","to":"$flowContext('thresholdCritical')","tot":"jsonata"}],"action":"","property":"","from":"","to":"","reg":false,"x":185,"y":500,"wires":[["dfbf240ade6b5ea6"]],"l":false},{"id":"3cd3f8618bb41f44","type":"subflow:588f2940.1ba848","z":"dbd54f40a5843152","name":"Check Page","env":[],"x":110,"y":240,"wires":[["b13d75f23918bb47"]]},{"id":"b13d75f23918bb47","type":"subflow:9d92739f.d0d64","z":"dbd54f40a5843152","name":"Check Time","env":[],"x":290,"y":240,"wires":[["4a5ce03dd38e4f0d","0ef54cefe8fd5f21"]]},{"id":"d39ca6ece1e6edd8","type":"comment","z":"dbd54f40a5843152","name":"Test Injects","info":"","x":100,"y":410,"wires":[]},{"id":"3d673a876386ca15","type":"function","z":"dbd54f40a5843152","name":"Info","func":"flow.set(\"warnings\", 0);\nflow.set(\"criticals\", 0);\nmsg.severity = \"info\";\nmsg.payload = \"All Clear\";\n\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","x":520,"y":290,"wires":[["bcd6df5dbe64eb02"]]},{"id":"0e945c3b949c9635","type":"inject","z":"dbd54f40a5843152","name":"Simulate Hub Reboot","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":450,"y":650,"wires":[["43be41a91018b9c0"]]},{"id":"dfbf240ade6b5ea6","type":"change","z":"dbd54f40a5843152","name":"Init","rules":[{"t":"set","p":"test","pt":"msg","to":"true","tot":"bool"},{"t":"set","p":"reboot","pt":"msg","to":"false","tot":"bool"},{"t":"set","p":"inMaintenance","pt":"msg","to":"false","tot":"bool"},{"t":"set","p":"awake","pt":"msg","to":"true","tot":"bool"}],"action":"","property":"","from":"","to":"","reg":false,"x":235,"y":470,"wires":[["4a5ce03dd38e4f0d"]],"l":false},{"id":"20acd13f8f109755","type":"switch","z":"dbd54f40a5843152","name":"Awake?","property":"awake","propertyType":"msg","rules":[{"t":"true"},{"t":"false"}],"checkall":"true","repair":false,"outputs":2,"x":710,"y":330,"wires":[["a5878149b5905157"],["f17e0521a9205be0"]]},{"id":"32da5d723fb5cfbe","type":"delay","z":"dbd54f40a5843152","name":"YES: 1x/Hour","pauseType":"rate","timeout":"5","timeoutUnits":"seconds","rate":"1","nbRateUnits":"1","rateUnits":"hour","randomFirst":"1","randomLast":"5","randomUnits":"seconds","drop":true,"outputs":1,"x":920,"y":410,"wires":[["84177b2eca1afc27"]]},{"id":"f97f8f8e801e583f","type":"delay","z":"dbd54f40a5843152","name":"NO: Only Once","pauseType":"rate","timeout":"5","timeoutUnits":"seconds","rate":"1","nbRateUnits":"9","rateUnits":"hour","randomFirst":"1","randomLast":"5","randomUnits":"seconds","drop":true,"outputs":1,"x":920,"y":450,"wires":[["84177b2eca1afc27"]]},{"id":"43be41a91018b9c0","type":"change","z":"dbd54f40a5843152","name":"Log Reboot Sim.","rules":[{"t":"set","p":"payload","pt":"msg","to":"CRITICAL! Simulating Hub Rebooting!","tot":"str"},{"t":"set","p":"test","pt":"msg","to":"true","tot":"bool"}],"action":"","property":"","from":"","to":"","reg":false,"x":740,"y":650,"wires":[["2692fe914c8774c9"]]},{"id":"84177b2eca1afc27","type":"no-op","z":"dbd54f40a5843152","name":"","x":1095,"y":380,"wires":[["8fbdef7804250603","5ca6e2ca49253e8f"]],"l":false},{"id":"2692fe914c8774c9","type":"no-op","z":"dbd54f40a5843152","name":"","x":895,"y":590,"wires":[["53e5cc2d7cf9786f","8c7373be4d331ae7","5ca6e2ca49253e8f"]],"l":false},{"id":"7ee409309f7d633d","type":"comment","z":"dbd54f40a5843152","name":"Read for config info. ","info":"1) Edit items in flow configuration item\n - if hub version is >= 2.2.3, set supportsMemCheck =1 \n - set hubName to be whatever you call your hub (main, LAN, etc)\n - deviceId will be whatever device you have (1 would be 1, if you have 2, make 1 be 1, and another be 2). I only need this because I have 2 devices and need to keep the flows and responses separate. \n \n2) Edit the \"reboot hub\" node with the right IP\n\n3) Configure all other config nodes - influx db, pushover","x":610,"y":40,"wires":[]},{"id":"54af75f3a7498b3d","type":"delay","z":"dbd54f40a5843152","name":"1/hr","pauseType":"rate","timeout":"5","timeoutUnits":"seconds","rate":"1","nbRateUnits":"1","rateUnits":"hour","randomFirst":"1","randomLast":"5","randomUnits":"seconds","drop":true,"outputs":1,"x":1170,"y":590,"wires":[["7de67d917450c090","59c6731b8761914c","919d1d396c217742"]]},{"id":"59c6731b8761914c","type":"function","z":"dbd54f40a5843152","d":true,"name":"INSERT","func":"var query = \"INSERT INTO reboottimes(displayName, `type`) VALUES(\";\nquery += \"'Hub Reboot',\";\nquery += \"'\" + msg.rebootType + \"'\";\nquery += \")\";\n\nmsg.topic = query;\n\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","x":1320,"y":630,"wires":[[]]},{"id":"127af81ced072146","type":"switch","z":"dbd54f40a5843152","name":"Awake?","property":"awake","propertyType":"msg","rules":[{"t":"true"},{"t":"false"}],"checkall":"true","repair":false,"outputs":2,"x":710,"y":430,"wires":[["32da5d723fb5cfbe"],["f97f8f8e801e583f"]]},{"id":"919d1d396c217742","type":"pushover api","z":"dbd54f40a5843152","keys":"b3ffec8b.aa42d","title":"Hubitat Watchdog","name":"Alert","x":1310,"y":550,"wires":[]},{"id":"53e5cc2d7cf9786f","type":"function","z":"dbd54f40a5843152","name":"Not Testing?","func":"if (msg.test) {\n return null;\n}\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","x":1020,"y":590,"wires":[["54af75f3a7498b3d"]]},{"id":"8c7373be4d331ae7","type":"function","z":"dbd54f40a5843152","name":"Testing?","func":"if (msg.test) {\n return msg;\n}\n\nreturn null;","outputs":1,"noerr":0,"initialize":"","finalize":"","x":1010,"y":550,"wires":[["919d1d396c217742"]]},{"id":"bc6eeaf000099ede","type":"function","z":"dbd54f40a5843152","name":"Set Influx Data","func":"var v = {\n 'value' : Number(msg.value),\n 'isBinary' : false\n};\n\nmsg.payload = [\n {\n measurement: msg.name,\n fields: {\n value: v.value,\n },\n tags:{\n hub: flow.get(\"hubName\"),\n deviceId: flow.get(\"deviceId\"),\n displayName: msg.displayName,\n unit: msg.unit\n },\n timestamp: new Date()\n }\n]\n\n\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","x":700,"y":200,"wires":[["530be295ab61fedf"]]},{"id":"530be295ab61fedf","type":"influxdb batch","z":"dbd54f40a5843152","influxdb":"e0131e84.c60ad","precision":"","retentionPolicy":"","name":"","database":"","retentionPolicyV18Flux":"","org":"Home","bucket":"SmartThings","x":940,"y":200,"wires":[]},{"id":"0ef54cefe8fd5f21","type":"change","z":"dbd54f40a5843152","name":"Format","rules":[{"t":"set","p":"name","pt":"msg","to":"responseTime","tot":"str"},{"t":"set","p":"value","pt":"msg","to":"responseTime","tot":"msg"},{"t":"set","p":"deviceId","pt":"msg","to":"1","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":540,"y":200,"wires":[["bc6eeaf000099ede"]]},{"id":"42e6a4b8aea3a52f","type":"config","z":"dbd54f40a5843152","name":"Initial State - Do Not Edit","properties":[{"p":"clears","pt":"flow","to":"0","tot":"num"},{"p":"warnings","pt":"flow","to":"0","tot":"num"},{"p":"criticals","pt":"flow","to":"0","tot":"num"},{"p":"lastReboot","pt":"flow","to":"","tot":"date"}],"active":true,"x":380,"y":40,"wires":[]},{"id":"5ca6e2ca49253e8f","type":"debug","z":"dbd54f40a5843152","name":"Hubitat Performace (LAN)","active":true,"tosidebar":true,"console":true,"tostatus":true,"complete":"true","targetType":"full","statusVal":"payload","statusType":"auto","x":1200,"y":480,"wires":[]},{"id":"bcd6df5dbe64eb02","type":"debug","z":"dbd54f40a5843152","name":"Hubitat Performance (LAN)","active":true,"tosidebar":true,"console":false,"tostatus":true,"complete":"payload","targetType":"msg","statusVal":"payload","statusType":"auto","x":760,"y":240,"wires":[]},{"id":"8c0b57d4f40eac39","type":"inject","z":"dbd54f40a5843152","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":130,"y":320,"wires":[[]]},{"id":"3dcb4cd037b0e4d0","type":"influxdb batch","z":"dbd54f40a5843152","influxdb":"e0131e84.c60ad","precision":"","retentionPolicy":"","name":"","database":"","retentionPolicyV18Flux":"","org":"Home","bucket":"SmartThings","x":1340,"y":200,"wires":[]},{"id":"a1c5ce4b5c7ba254","type":"debug","z":"dbd54f40a5843152","name":"Mem Check (LAN)","active":true,"tosidebar":true,"console":true,"tostatus":true,"complete":"payload","targetType":"msg","statusVal":"payload","statusType":"auto","x":1300,"y":140,"wires":[]},{"id":"e679c4663215203b","type":"switch","z":"dbd54f40a5843152","name":"Supports Mem Check? (>2.2.3)","property":"supportsMemCheck","propertyType":"flow","rules":[{"t":"gte","v":"1","vt":"num"}],"checkall":"false","repair":false,"outputs":1,"x":330,"y":100,"wires":[["0a14573168922b19"]]},{"id":"0a14573168922b19","type":"delay","z":"dbd54f40a5843152","name":"5s delay ","pauseType":"delay","timeout":"5","timeoutUnits":"seconds","rate":"1","nbRateUnits":"1","rateUnits":"second","randomFirst":"1","randomLast":"5","randomUnits":"seconds","drop":false,"outputs":1,"x":380,"y":160,"wires":[["ea5c488e0cbad7d6"]]},{"id":"ea5c488e0cbad7d6","type":"subflow:43ac6c1.6e31c94","z":"dbd54f40a5843152","name":"","env":[],"x":530,"y":140,"wires":[["b9c6f40f67526e5e"]]},{"id":"b9c6f40f67526e5e","type":"subflow:9d92739f.d0d64","z":"dbd54f40a5843152","name":"Check Time","env":[],"x":710,"y":140,"wires":[["062303bedcf547e5","1fa627b6cd45e28f"]]},{"id":"062303bedcf547e5","type":"change","z":"dbd54f40a5843152","name":"Format","rules":[{"t":"set","p":"name","pt":"msg","to":"freeMem","tot":"str"},{"t":"set","p":"value","pt":"msg","to":"freeMemValue","tot":"msg"},{"t":"set","p":"deviceId","pt":"msg","to":"1","tot":"str"},{"t":"set","p":"displayName","pt":"msg","to":"Free Memory","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":880,"y":140,"wires":[["1d3e1a0b043d4a82"]]},{"id":"1d3e1a0b043d4a82","type":"function","z":"dbd54f40a5843152","name":"Set Influx Data","func":"var v = {\n 'value' : Number(msg.value),\n 'isBinary' : false\n};\n\nmsg.payload = [\n {\n measurement: msg.name,\n fields: {\n value: v.value,\n },\n tags:{\n hub: flow.get(\"hubName\"),\n deviceId: flow.get(\"deviceId\"),\n displayName: msg.displayName,\n unit: msg.unit\n },\n timestamp: new Date()\n }\n]\n\n\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","x":1060,"y":140,"wires":[["a1c5ce4b5c7ba254","3dcb4cd037b0e4d0"]]},{"id":"1fa627b6cd45e28f","type":"switch","z":"dbd54f40a5843152","name":"Memory Critical?","property":"freeMemValue","propertyType":"msg","rules":[{"t":"lt","v":"memCritical","vt":"flow"}],"checkall":"false","repair":false,"outputs":1,"x":410,"y":420,"wires":[["92760509d8ad4638"]]},{"id":"92760509d8ad4638","type":"function","z":"dbd54f40a5843152","name":"Crit","func":"flow.set(\"criticals\", flow.get(\"criticals\") + 1);\nmsg.severity = \"crit\";\nmsg.payload = \"Memory below CRITICAL threshold for Hub: \" + flow.get(\"hubName\") + \" (\" + msg.freeMemValue + \" < \" + flow.get(\"memCritical\") + \")\";\n\nif (msg.inMaintenance) {\n msg.critsNeeded = flow.get(\"criticalsForRebootDuringMaintenance\");\n} else {\n msg.critsNeeded = flow.get(\"criticalsForReboot\");\n}\n\nif (flow.get(\"criticals\") >= msg.critsNeeded) {\n msg.reboot = true;\n flow.set(\"lastReboot\", Date.now());\n return msg;\n} else {\n return null;\n}","outputs":1,"noerr":0,"initialize":"","finalize":"","x":470,"y":480,"wires":[["127af81ced072146","ef79f90eab2ce497"]]},{"id":"59b551dc713e731f","type":"subflow:e990e72.c9c5718","z":"dbd54f40a5843152","name":"","env":[],"x":610,"y":80,"wires":[["cb8dd3ed512bc1df"]]},{"id":"cb8dd3ed512bc1df","type":"subflow:9d92739f.d0d64","z":"dbd54f40a5843152","name":"","env":[],"x":800,"y":80,"wires":[["3b495db680a3f4d8"]]},{"id":"3b495db680a3f4d8","type":"change","z":"dbd54f40a5843152","name":"Format Temp data","rules":[{"t":"set","p":"name","pt":"msg","to":"hubTemp","tot":"str"},{"t":"set","p":"value","pt":"msg","to":"tempValue","tot":"msg"},{"t":"set","p":"deviceId","pt":"msg","to":"1","tot":"str"},{"t":"set","p":"displayName","pt":"msg","to":"Hub Temperature","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":990,"y":80,"wires":[["dede233176fccf1d"]]},{"id":"dede233176fccf1d","type":"function","z":"dbd54f40a5843152","name":"set Influx Data","func":"var v = {\n 'value' : Number(msg.value)*1.8+32,\n 'isBinary' : false\n};\n\nmsg.payload = [\n {\n measurement: msg.name,\n fields: {\n value: v.value,\n },\n tags:{\n hub: flow.get(\"hubName\"),\n deviceId: flow.get(\"deviceId\"),\n displayName: msg.displayName,\n unit: msg.unit\n },\n timestamp: new Date()\n }\n]\n\n\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","x":1220,"y":80,"wires":[["73258af0038842fa","74171ec20a00219f"]]},{"id":"73258af0038842fa","type":"debug","z":"dbd54f40a5843152","name":"Temp Check (LAN)","active":true,"tosidebar":true,"console":true,"tostatus":true,"complete":"payload","targetType":"msg","statusVal":"payload","statusType":"auto","x":1470,"y":60,"wires":[]},{"id":"74171ec20a00219f","type":"influxdb batch","z":"dbd54f40a5843152","influxdb":"e0131e84.c60ad","precision":"","retentionPolicy":"","name":"","database":"","retentionPolicyV18Flux":"","org":"Home","bucket":"SmartThings","x":1610,"y":120,"wires":[]},{"id":"ab926970d72046e2","type":"inject","z":"dbd54f40a5843152","name":"1 mins.","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"60","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":100,"y":80,"wires":[["59b551dc713e731f","e679c4663215203b"]]},{"id":"4ef0de2cdd08781d","type":"hubitat device","z":"dbd54f40a5843152","deviceLabel":"hub inf0","name":"","server":"37a6fe31.441bc2","deviceId":"449","attribute":"cpuPct","sendEvent":true,"x":320,"y":780,"wires":[["5ab8e13615e86e41"]]},{"id":"b96006e1a68a9a64","type":"hubitat device","z":"dbd54f40a5843152","deviceLabel":"hub inf0","name":"","server":"37a6fe31.441bc2","deviceId":"449","attribute":"cpu5Min","sendEvent":true,"x":320,"y":860,"wires":[[]]},{"id":"4091821282657fdb","type":"inject","z":"dbd54f40a5843152","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"300","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":100,"y":820,"wires":[["4ef0de2cdd08781d","b96006e1a68a9a64","e225fb11cd27a958"]]},{"id":"5ab8e13615e86e41","type":"change","z":"dbd54f40a5843152","name":"Format Temp data","rules":[{"t":"set","p":"name","pt":"msg","to":"hubcpuPct","tot":"str"},{"t":"set","p":"value","pt":"msg","to":"payload.value","tot":"msg"},{"t":"set","p":"deviceId","pt":"msg","to":"1","tot":"str"},{"t":"set","p":"displayName","pt":"msg","to":"Hub CPU Percent","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":510,"y":780,"wires":[["64bf40707d923bbb"]]},{"id":"64bf40707d923bbb","type":"function","z":"dbd54f40a5843152","name":"set Influx Data","func":"var v = {\n 'value' : Number(msg.value),\n 'isBinary' : false\n};\n\nmsg.payload = [\n {\n measurement: msg.name,\n fields: {\n value: v.value,\n },\n tags:{\n hub: flow.get(\"hubName\"),\n deviceId: flow.get(\"deviceId\"),\n displayName: msg.displayName,\n unit: msg.unit\n },\n timestamp: new Date()\n }\n]\n\n\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":720,"y":780,"wires":[["887a1b52ef2a6ecc","8de8e4d0bc79b0c4"]]},{"id":"887a1b52ef2a6ecc","type":"debug","z":"dbd54f40a5843152","name":"CPU Usage (Hub Info Driver)","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","statusVal":"","statusType":"auto","x":1040,"y":740,"wires":[]},{"id":"8de8e4d0bc79b0c4","type":"influxdb batch","z":"dbd54f40a5843152","influxdb":"e0131e84.c60ad","precision":"","retentionPolicy":"","name":"","database":"","retentionPolicyV18Flux":"","org":"Home","bucket":"SmartThings","x":1040,"y":780,"wires":[]},{"id":"e225fb11cd27a958","type":"hubitat device","z":"dbd54f40a5843152","deviceLabel":"hub inf0","name":"Hub Info DB Size","server":"37a6fe31.441bc2","deviceId":"449","attribute":"dbSize","sendEvent":true,"x":360,"y":940,"wires":[["789e1f12824dbe3b"]]},{"id":"789e1f12824dbe3b","type":"change","z":"dbd54f40a5843152","name":"Format Temp data","rules":[{"t":"set","p":"name","pt":"msg","to":"hubDbSize","tot":"str"},{"t":"set","p":"value","pt":"msg","to":"payload.value","tot":"msg"},{"t":"set","p":"deviceId","pt":"msg","to":"1","tot":"str"},{"t":"set","p":"displayName","pt":"msg","to":"Database Size","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":570,"y":940,"wires":[["64bf40707d923bbb"]]},{"id":"b3ffec8b.aa42d","type":"pushover-keys","name":"Craig Pushover"},{"id":"e0131e84.c60ad","type":"influxdb","hostname":"192.168.86.10","port":"8086","protocol":"http","database":"SmartThings","name":"InfluxDB SmartThings","usetls":false,"tls":"","influxdbVersion":"2.0","url":"http://192.168.86.10:8086","rejectUnauthorized":true},{"id":"37a6fe31.441bc2","type":"hubitat config","name":"Main Hub","usetls":false,"host":"192.168.86.35","port":"80","appId":"563","nodeRedServer":"http://192.168.86.10:1880","webhookPath":"/hubitat/webhook","autoRefresh":true,"useWebsocket":false,"colorEnabled":false,"color":"#ace043"}]
Or search for bundle manager and download hub watchdog and you setup various devices and how often they should toggle on/off the device to measure response time. I set one up for zwave one for zigbee and one virtual. Of course you will have to sacrifice one of each type of physical switch to do this.
1 Like
I recommend you work out at what point in the free memory range your hub Starts to fail and configure it to reboot when that point is reached.
Eg say you notice it starting to fail at 190 MB free, set it to restart when there is less than 200 MB free at say 4am.
This approach works really well for me and many others.
3 Likes
@dJOS What page are people seeing their remaining memory on? I can't see find this in logs.
Also how do I build this in rule machine, I use WebCore not Rule Machine so i'm not aware of how you get the hub memory details to appear as they don't show in my default install.
I'm a little biased but my favorite:
8 Likes
Thanks - how do we then add options to reboot based on memory levels to (say WebCore?)
