New device db consistency/ghost checker

kahn-hubitat · March 22, 2025, 3:59pm

Initial release a device/app i just created because occasionaly i have found erroneous entries in my overall device database that were not in the zwave radio database.

Only remedy was exclusion and re-inclusion.

This Device (not app so it can have attributes to trigger rules off of) loads the entire device, zwave table, and zigbee tables/databases when the compare command is executed.

It then checks for zwave devices in the device list that are NOT in the zwave radio table, and also zigbee devices in the device list that are NOT in the zigbee radio table.

It also checks for entries in the zwave radio table that are NOT in the device list (commonly known as ghosts).

It reports on all these in attributes and in the logs.

There is also an option to schedule runs in number of days (its cpu intensive so not recommending running all the time), and a specific time.

There are also timing attributes so you can see how long it takes to run, and a results attribute suitable for use in a dashboard.

Be carefull turning on full debugging if you have large device/zwave/zigbee database in can fill up the log rather quickly.

Not get in git hub.. will do that in a day or day..

here is example runs and rules i have set up to alert myself..

github.com

lgkahn/hubitat/blob/master/dbconsistencychecker.groovy


/* lgkahn kahn@lgk.com
* device to enumerate through all devices in the system and check that there are no zwave/zigbee devices that are not in their corresponding zwave/zigbee tables.
* also checks in the other direction to make sure there are no zwave devices in the zwave table that are not in the device list (ghosts).
* v 1.1
 
*/

preferences()
{  
    input "debug", "bool", title: "Enable debugging?", required: true, defaultValue: false
    input "descLog", "bool", title: "Enable description Text logging", required: true, defaultValue: true
    input "runSchedule", "enum", title: "Schedule to run every X days?",options: ["Disabled", "1", "2", "3", "5", "7","14","21","28","30"],  required: true, defaultValue: "Disabled"
    input "runHour","enum",title: "If scheduled to run, run at which hour?",options: ["00","01", "02","03","04","05","06","07","08","09","10","11","12","13","14","15","16","17","18","19","20"," 21","22","23"], required: true, defaultValue: "00"
    input "clearTableStatesAfterRun","bool",title: "Clear out the table/state variables after run to not cluter the device status page?", required: true, defaultValue: true
}

metadata {
    definition (name: "LGK Zwave table/devices comparison.", namespace: "lgkapps", author: "larry kahn kahn@lgk.com",  singleThreaded: false) {
    capability "Actuator"

This file has been truncated. show original

kahn-hubitat · March 24, 2025, 8:32pm

Should now be in hubitat package manager as soon of manifest update code runs!!

jtp10181 · March 25, 2025, 11:55am

I checked out this code the other day, I noticed it looks like you are hitting the /device/fullJson/$deviceid endpoint for every device on every run. I suspect that is where the heavy load is coming from. Have you considered caching the zwave/zigbee true/false info in a state variable? Then you only need to check the endpoint for new devices not already saved. The device ID could only ever be for one device and the Zwave / Zigbee check should never change so I think its safe to do that.

kahn-hubitat · March 25, 2025, 1:57pm

Ya good idea . I also coukd do insertion sort and speed uo the searching. But thought it was overkill.

jtp10181 · March 25, 2025, 2:45pm

I think just reducing the endpoint hits will greatly improve it, since they are http calls. The searching for matches part should be pretty quick once most of the data is readily available.

kahn-hubitat · March 25, 2025, 2:47pm

when i was writing it i was suprised at least half the time is just the one endpoint getting back all the device data since it returns such a large structure.. would be nice if the tag whether they are zigbee are zwave was in that structure.

also may hold off on any changes right now as i assume all the endpoints and data returned are changing at least the zwave part with the new zwavejs and willhave to rewrite anyway.

kahn-hubitat · March 26, 2025, 3:43am

new version 1.3 in git hub.. thanks to jtp above
i now load the device details into a map and just use that .. it is only loaded when it is empty (on first run) or if you execute the new re-initialize function , or in the case you add a new device and it cannot find it in the table.

here are the times for the initial run and subsequent runs that uses the array in a state variable instead of loading every time and issueing an api http endpoint for each device..