As an end user - a) polling is very useful for battery powered measuring devices - temp and lux 2 obvious measures. IMO it's "truer" than interpolation. b) go for breaking changes for multiple instances.
Other than the one-shot loss of pending data when the new version is installed, breaking changes haven't actually been required yet.
That said, if there were no install base, I would switch this to parent/child in a heartbeat. 
1 Like
@dennypage Looking at a few things quickly it looks like this removes the speed improvement we found with the Influxdb2.x upgrade PR. So this change to a state value for the logger queue will cost performance. Does seem to work though.
I don't either, but I was hopefull when i saw the addChildApp method. That said I can't for the life of me figure it out.
Yea, I'm not too surprised. Updating the state.* map seems to be expensive. Balancing that against multiple instances and fixing the data loss on update issue though.
There is probably a reasonable way to get overall performance back though. Remembering that the performance degradation is beyond linear, you can scale horizontally by using multiple app instances and dividing up the device load amongst them. So instead of a single instance with a queue limit of 50, three instances with a queue limit of 15.
The other alternative is Field variable maps like @hubitrep proposed. However this requires global mutexes, setters and getters, each of which has its own performance impact. Also, with globals each app instance will compete with other instances for access to the concurrent mutex and queue maps, so you may not be able to scale out as effectively. Unless there is a significant performance difference, I think it makes sense to go with the simpler approach.
Well it was well beyond linear. I would probably say on orders of magnitude improvements in time to process the step to create the post record. Keep in mind this is also about CPU Consumption. The decrease in cpu time was also a big boost.
So I have all 3 code bases loaded on my hubs.
The current published code base seems to do the Write the queue out for posting in 1-2 ms regardless of size. Even with 34 recordswhich is where my limit is it stays at 2ms
With @hubitrep's code time to Write the queue out for posting for aprox 30 records goes to 100-120ms around around 3-4 ms per record
With your last change to use a state value time to write the queue for posting went up to 159ms for 38 records ore has gone back up to around 4.2ms per record
Another thought is that we take advantage of the current app using the same queue. We simply document and provide directions on how to use it properly so not to get bad data. Like don't use soft polling in more then one app, Don't select the same devices and attribute between apps. Do use multiple installs if you need access to all attributes as well a the legacy method.
My concern is i don't want to not think about system impact when some of this stuff is changed. I am pretty certain that in the early days of InfluxDB Logger on Hubitat there were some performance issues/impact to the hub. I think that is partly why folks use Node-Red allot to get data into InfluxDB instead of this app sometimes.
Agree simpler is better.
If I read this right, you seem to have gotten rid of the semaphore altogether. I have read other discussions here where it is said that state.* is not considered thread-safe and the app's event handler can get called from multiple threads. The alternative I've seen suggested is to use atomicState.* variables but, for those, writes go to the db directly. I suspect the performance might get even worse.
You are correct, the state map is not multi-thread safe. The semaphore isn't really gone, it's just externalized. The magic is on line 52. In Hubitat 2.2.9, they introduced the singleThreaded attribute for both apps and drivers, which provides externally managed thread exclusion. It is faster than atomicState and designed for just this kind of situation.
[Edit: singleThreaded attribute described here]
1 Like
If you want to try something with my version while I sort out my branches, try commenting out the mutex calls in queueToInfluxDb() and writeQueuedDataToInfluxDb(). I'm not sure it's needed. The mutex should be necessary only to protect the reference to the ConcurrentHashMap, not the ConcurrentHashMap itself, which is supposed to be thread-safe.
Still just over 100ms after two runs for 30 records
1 Like
The mutex is required to prevent an event thread and a post thread from accessing/modifying the loggerQueue simultaneously. Concurrent provides protection from corrupting the underlying data structure, it doesn't provide protection from stale state in the thread. I.E. loggerQueue.size() can change after you read it.
You can remove the mutex if you use singleThreaded.
That's why I set it up at the time, I saw posts of people complaining about performance issues with the app so I went the Node-Red route. But I also heavily re-wrote the example flow custom functions that people had posted and also made it work for Influx 2.x
Sometimes I wonder if there is a performance hit using MakerAPI though since I am sending every event from almost every device to Node-Red as soon as it happens. They get filtered in NR. Using this logging app I could send only the events I select.
1 Like
I am sure there is overhead, by sending the data to Node-Red, but you have to ask yourself how much would be eliminated by using InfluxDB Logger. If you are heavy into Node-Red for other stuff it very well could be zero on the hub.
I have 45 devices in Node-Red and the app shows usage of .249 of total
In InfluxDB i have a bunch of devices across multiple capabilities selected. 45 devices just with the battery capability. On a soft polling it is over 240 events. The current app uses .906% cpu of total.
That .906 would be allot lower though if soft polling wasn't used. Where you might see a benefit though is on the InfluxDB server side. Single record adds can cause high cpu usage because of allot of overhead on the server side for each post.
I really doubt though that Maker API sending the raw events to Node-red is less efficient. It also isn't exactly fair to compare the two as InfluxDB Logger also queues up the records and then formats them for the insert.
Well that is pretty convincing that the talking to Influx part is more well suited to be off loaded to NR, but like I said more complicated to setup. This app makes it pretty easy.
I am going to have to investigate if I can queue up the records in NR somehow and batch them over every 5 minutes. I am already using a "batch" message because it was easier to construct in NR, but only sending one event at a time. Problem will be same as you have with this app though, what happens if the NR server is restarted or even just a deployment of changes.
Not too worried about CPU on influx, its running on old Quad-Core micro PC media server. CPU floats around 6% and most of that is being used by Homebridge.
1 Like
I agree. CPU restrictions on Hubitat are likely to be tighter than a system running an InfluxDB server.
1 Like
Not necessarily true. I bet there are a fair number of folks runing influxDB on a raspberry Pi.
It seems that the current Hubitat is around rPi 3 A+ performance, substantially below that of the current rPi 4. At least that what I guesstimate based on CPU specs. Understand that I could be completely wrong--I am not a rPi guy, I am an Atom guy.
But I think the more important consideration is that the Hubitat has pseudo real-time responsibilities that a general purpose system running an InfluxDB server would normally not have.
[ Apologies to the non-developer users that are currently following the thread... this has temporarily turned into quite a developer topic. Things should return to normal soon. ]
I'm spinning on the state.* vs Field map issue, so I coded it up both ways to run performance tests. The test code is below. Please note that this code is not intended for production use.
There are two different approaches in the test code. Before you instantiate an app with it, you must uncomment the approach you want to test beginning line 771 and 797. Also note that you must remove the app instance and install a new one whenever you change the approach. Lastly, please note that logging for event processing and post are commented out in the test code. The reason for this is that calling these functions alone easily consumes more than 100ms (even more if you are watching the logs) which otherwise clouds the results.
What the test code does is to create a large number of soft events (100) per interval and then posts them to influx. The specific conditions that I am testing are:
- soft poll at 1 minute
- db insertions at 1 minute
- queue size limit set to 500 (so it doesn’t interfere)
- one slow moving device being monitored (in my case, a temp sensor)
- after installing the app, wait at least 10 iterations (10 minutes)
- examine the Ave ms value show in Logs->App stats
In short this does 100 insertions each minute as a single batch.
With approach 1, after 20 iterations I am seeing an average of 118ms per method.
With approach 2, after 20 iterations I am seeing an average of 109ms per method.
So, in summary, state.* ends up being around 8% more expensive than using a Field map. Of course the performance benefit of the Field map comes at the cost of pending data from all instances being lost during a code update.
I would very much like to hear people's opinions on the tradeoff between the 8% performance benefit and the dataloss on update.
Test code
/* groovylint-disable ImplementationAsType, InvertedCondition, LineLength, MethodReturnTypeRequired, MethodSize, NestedBlockDepth, NoDef, UnnecessaryGString, UnnecessaryObjectReferences, UnnecessaryToString, VariableTypeRequired */
/*****************************************************************************************************************
* Source: https://github.com/HubitatCommunity/InfluxDB-Logger
*
* Raw Source: https://raw.githubusercontent.com/HubitatCommunity/InfluxDB-Logger/master/influxdb-logger.groovy
*
* Forked from: https://github.com/codersaur/SmartThings/tree/master/smartapps/influxdb-logger
* Original Author: David Lomas (codersaur)
* Hubitat Elevation version maintained by HubitatCommunity (https://github.com/HubitatCommunity/InfluxDB-Logger)
*
* Description: A SmartApp to log Hubitat device states to an InfluxDB database.
* See Codersaur's github repo for more information.
*
* License:
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except
* in compliance with the License. You may obtain a copy of the License at:
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed
* on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License
* for the specific language governing permissions and limitations under the License.
*
* Modifcation History
* Date Name Change
* 2019-02-02 Dan Ogorchock Use asynchttpPost() instead of httpPost() call
* 2019-09-09 Caleb Morse Support deferring writes and doing buld writes to influxdb
* 2022-06-20 Denny Page Remove nested sections for device selection.
* 2023-01-08 Denny Page Address whitespace related lint issues. No functional changes.
* 2023-01-09 Craig Added InfluxDb2.x support.
* 2023=01-12 Denny Page Automatic migration of Influx 1.x settings.
* 2023-01-15 Denny Page Clean up various things:
* Remove Group ID/Name which are not supported on Hubitat.
* Remove Location ID and Hub ID which are not supported on Hubitat (always 1).
* Remove blocks of commented out code.
* Don't set page sections hidden to false where hideable is false.
* Remove state.queuedData.
* 2023=01-22 Denny Page Allow multiple instances of the application to be installed.
* NB: This requires Hubitat 2.2.9 or above.
*****************************************************************************************************************/
definition(
name: "InfluxDB Logger",
namespace: "nowhereville",
author: "Joshua Marker (tooluser)",
description: "Log device states to InfluxDB",
category: "My Apps",
importUrl: "https://raw.githubusercontent.com/HubitatCommunity/InfluxDB-Logger/master/influxdb-logger.groovy",
iconUrl: "",
iconX2Url: "",
iconX3Url: "",
singleThreaded: true
)
import groovy.transform.Field
@Field static loggerQueueMap = new java.util.concurrent.ConcurrentHashMap<String, java.util.concurrent.ConcurrentLinkedQueue>()
preferences {
page(name: "setupMain")
page(name: "connectionPage")
}
def setupMain() {
dynamicPage(name: "setupMain", title: "InfluxDB Logger Settings", install: true, uninstall: true) {
section("") {
input "appName", "text", title: "Aplication Name", multiple: false, required: false, submitOnChange: true, defaultValue: app.getLabel()
href(
name: "href",
title: "Connection Settings",
description : prefDatabaseHost == null ? "Configure database connection parameters" : prefDatabaseHost,
required: true,
page: "connectionPage"
)
input(
name: "configLoggingLevelIDE",
title: "IDE Live Logging Level:\nMessages with this level and higher will be logged to the IDE.",
type: "enum",
options: [
"0" : "None",
"1" : "Error",
"2" : "Warning",
"3" : "Info",
"4" : "Debug",
"5" : "Trace"
],
defaultValue: "3",
displayDuringSetup: true,
required: false
)
}
section("Polling / Write frequency:") {
input "prefSoftPollingInterval", "number", title:"Soft-Polling interval (minutes)", defaultValue: 10, required: true
input "writeInterval", "enum", title:"How often to write to db (minutes)", defaultValue: "5", required: true,
options: ["1", "2", "3", "4", "5", "10", "15"]
input "prefWriteQueueLimit", "number", title:"Write Interval Queue Size Limit", defaultValue: 50, required: true
}
section("System Monitoring:") {
input "prefLogModeEvents", "bool", title:"Log Mode Events?", defaultValue: false, required: true
input "prefLogHubProperties", "bool", title:"Log Hub Properties?", defaultValue: false, required: true
input "prefLogLocationProperties", "bool", title:"Log Location Properties?", defaultValue: false, required: true
}
section("Input Format Preference:") {
input "accessAllAttributes", "bool", title:"Get Access To All Attributes?", defaultValue: false, required: true, submitOnChange: true
}
if (!accessAllAttributes) {
section("Devices To Monitor:", hideable:false) {
input "accelerometers", "capability.accelerationSensor", title: "Accelerometers", multiple: true, required: false
input "alarms", "capability.alarm", title: "Alarms", multiple: true, required: false
input "batteries", "capability.battery", title: "Batteries", multiple: true, required: false
input "beacons", "capability.beacon", title: "Beacons", multiple: true, required: false
input "buttons", "capability.button", title: "Buttons", multiple: true, required: false
input "cos", "capability.carbonMonoxideDetector", title: "Carbon Monoxide Detectors", multiple: true, required: false
input "co2s", "capability.carbonDioxideMeasurement", title: "Carbon Dioxide Detectors", multiple: true, required: false
input "colors", "capability.colorControl", title: "Color Controllers", multiple: true, required: false
input "consumables", "capability.consumable", title: "Consumables", multiple: true, required: false
input "contacts", "capability.contactSensor", title: "Contact Sensors", multiple: true, required: false
input "doorsControllers", "capability.doorControl", title: "Door Controllers", multiple: true, required: false
input "energyMeters", "capability.energyMeter", title: "Energy Meters", multiple: true, required: false
input "humidities", "capability.relativeHumidityMeasurement", title: "Humidity Meters", multiple: true, required: false
input "illuminances", "capability.illuminanceMeasurement", title: "Illuminance Meters", multiple: true, required: false
input "locks", "capability.lock", title: "Locks", multiple: true, required: false
input "motions", "capability.motionSensor", title: "Motion Sensors", multiple: true, required: false
input "musicPlayers", "capability.musicPlayer", title: "Music Players", multiple: true, required: false
input "peds", "capability.stepSensor", title: "Pedometers", multiple: true, required: false
input "phMeters", "capability.pHMeasurement", title: "pH Meters", multiple: true, required: false
input "powerMeters", "capability.powerMeter", title: "Power Meters", multiple: true, required: false
input "presences", "capability.presenceSensor", title: "Presence Sensors", multiple: true, required: false
input "pressures", "capability.pressureMeasurement", title: "Pressure Sensors", multiple: true, required: false
input "shockSensors", "capability.shockSensor", title: "Shock Sensors", multiple: true, required: false
input "signalStrengthMeters", "capability.signalStrength", title: "Signal Strength Meters", multiple: true, required: false
input "sleepSensors", "capability.sleepSensor", title: "Sleep Sensors", multiple: true, required: false
input "smokeDetectors", "capability.smokeDetector", title: "Smoke Detectors", multiple: true, required: false
input "soundSensors", "capability.soundSensor", title: "Sound Sensors", multiple: true, required: false
input "spls", "capability.soundPressureLevel", title: "Sound Pressure Level Sensors", multiple: true, required: false
input "switches", "capability.switch", title: "Switches", multiple: true, required: false
input "switchLevels", "capability.switchLevel", title: "Switch Levels", multiple: true, required: false
input "tamperAlerts", "capability.tamperAlert", title: "Tamper Alerts", multiple: true, required: false
input "temperatures", "capability.temperatureMeasurement", title: "Temperature Sensors", multiple: true, required: false
input "thermostats", "capability.thermostat", title: "Thermostats", multiple: true, required: false
input "threeAxis", "capability.threeAxis", title: "Three-axis (Orientation) Sensors", multiple: true, required: false
input "touchs", "capability.touchSensor", title: "Touch Sensors", multiple: true, required: false
input "uvs", "capability.ultravioletIndex", title: "UV Sensors", multiple: true, required: false
input "valves", "capability.valve", title: "Valves", multiple: true, required: false
input "volts", "capability.voltageMeasurement", title: "Voltage Meters", multiple: true, required: false
input "waterSensors", "capability.waterSensor", title: "Water Sensors", multiple: true, required: false
input "windowShades", "capability.windowShade", title: "Window Shades", multiple: true, required: false
}
} else {
section("Devices To Monitor:", hideable:false) {
input name: "allDevices", type: "capability.*", title: "Selected Devices", multiple: true, required: false, submitOnChange: true
}
state.selectedAttr = [:]
settings.allDevices.each { deviceName ->
if (deviceName) {
deviceId = deviceName.getId()
attr = deviceName.getSupportedAttributes().unique()
if (attr) {
state.options = []
index = 0
attr.each { at ->
state.options[index] = "${at}"
index = index + 1
}
section("$deviceName", hideable: true) {
input name:"attrForDev$deviceId", type: "enum", title: "$deviceName", options: state.options, multiple: true, required: false, submitOnChange: true
}
state.selectedAttr[deviceId] = settings["attrForDev" + deviceId]
}
}
}
}
}
}
def connectionPage() {
dynamicPage(name: "connectionPage", title: "Connection Properties", install: false, uninstall: false) {
section {
input "prefDatabaseTls", "bool", title:"Use TLS?", defaultValue: false, required: true
input "prefDatabaseHost", "text", title: "Host", defaultValue: "192.168.1.100", required: true
input "prefDatabasePort", "text", title : "Port", defaultValue : prefDatabaseTls ? "443" : "8086", required : false
input(
name: "prefInfluxVer",
title: "Influx Version",
type: "enum",
options: [
"1" : "v1.x",
"2" : "v2.x"
],
defaultValue: "1",
submitOnChange: true,
required: true
)
if (prefInfluxVer == "1") {
input "prefDatabaseName", "text", title: "Database Name", defaultValue: "Hubitat", required: true
} else if (prefInfluxVer == "2") {
input "prefOrg", "text", title: "Org", defaultValue: "", required: true
input "prefBucket", "text", title: "Bucket", defaultValue: "", required: true
}
input(
name: "prefAuthType",
title: "Authorization Type",
type: "enum",
options: [
"none" : "None",
"basic" : "Username / Password",
"token" : "Token"
],
defaultValue: "none",
submitOnChange: true,
required: true
)
if (prefAuthType == "basic") {
input "prefDatabaseUser", "text", title: "Username", defaultValue: "", required: false
input "prefDatabasePass", "text", title: "Password", defaultValue: "", required: false
} else if (prefAuthType == "token") {
input "prefDatabaseToken", "text", title: "Token", required: true
}
}
}
}
def getDeviceObj(id) {
def found
settings.allDevices.each { device ->
if (device.getId() == id) {
//log.debug "Found at $device for $id with id: ${device.id}"
found = device
}
}
return found
}
/**
* installed()
*
* Runs when the app is first installed.
**/
def installed() {
state.installedAt = now()
state.loggingLevelIDE = 5
updated()
log.debug "${app.label}: Installed with settings: ${settings}"
}
/**
* uninstalled()
*
* Runs when the app is uninstalled.
**/
def uninstalled() {
removeLoggerQueue()
log.debug "${app.label}: uninstalled"
}
/**
* updated()
*
* Runs when app settings are changed.
*
* Updates device.state with input values and other hard-coded values.
* Builds state.deviceAttributes which describes the attributes that will be monitored for each device collection
* (used by manageSubscriptions() and softPoll()).
* Refreshes scheduling and subscriptions.
**/
def updated() {
logger("updated()", "trace")
// Update application name
app.updateLabel(appName)
// Update internal state:
state.loggingLevelIDE = (settings.configLoggingLevelIDE) ? settings.configLoggingLevelIDE.toInteger() : 3
// Database config:
setupDB()
// Build array of device collections and the attributes we want to report on for that collection:
// Note, the collection names are stored as strings. Adding references to the actual collection
// objects causes major issues (possibly memory issues?).
state.deviceAttributes = []
state.deviceAttributes << [ devices: 'accelerometers', attributes: ['acceleration']]
state.deviceAttributes << [ devices: 'alarms', attributes: ['alarm']]
state.deviceAttributes << [ devices: 'batteries', attributes: ['battery']]
state.deviceAttributes << [ devices: 'beacons', attributes: ['presence']]
state.deviceAttributes << [ devices: 'buttons', attributes: ['button']]
state.deviceAttributes << [ devices: 'cos', attributes: ['carbonMonoxide']]
state.deviceAttributes << [ devices: 'co2s', attributes: ['carbonDioxide']]
state.deviceAttributes << [ devices: 'colors', attributes: ['hue', 'saturation', 'color']]
state.deviceAttributes << [ devices: 'consumables', attributes: ['consumableStatus']]
state.deviceAttributes << [ devices: 'contacts', attributes: ['contact']]
state.deviceAttributes << [ devices: 'doorsControllers', attributes: ['door']]
state.deviceAttributes << [ devices: 'energyMeters', attributes: ['energy']]
state.deviceAttributes << [ devices: 'humidities', attributes: ['humidity']]
state.deviceAttributes << [ devices: 'illuminances', attributes: ['illuminance']]
state.deviceAttributes << [ devices: 'locks', attributes: ['lock']]
state.deviceAttributes << [ devices: 'motions', attributes: ['motion']]
state.deviceAttributes << [ devices: 'musicPlayers', attributes: ['status', 'level', 'trackDescription', 'trackData', 'mute']]
state.deviceAttributes << [ devices: 'peds', attributes: ['steps', 'goal']]
state.deviceAttributes << [ devices: 'phMeters', attributes: ['pH']]
state.deviceAttributes << [ devices: 'powerMeters', attributes: ['power', 'voltage', 'current', 'powerFactor']]
state.deviceAttributes << [ devices: 'presences', attributes: ['presence']]
state.deviceAttributes << [ devices: 'pressures', attributes: ['pressure']]
state.deviceAttributes << [ devices: 'shockSensors', attributes: ['shock']]
state.deviceAttributes << [ devices: 'signalStrengthMeters', attributes: ['lqi', 'rssi']]
state.deviceAttributes << [ devices: 'sleepSensors', attributes: ['sleeping']]
state.deviceAttributes << [ devices: 'smokeDetectors', attributes: ['smoke']]
state.deviceAttributes << [ devices: 'soundSensors', attributes: ['sound']]
state.deviceAttributes << [ devices: 'spls', attributes: ['soundPressureLevel']]
state.deviceAttributes << [ devices: 'switches', attributes: ['switch']]
state.deviceAttributes << [ devices: 'switchLevels', attributes: ['level']]
state.deviceAttributes << [ devices: 'tamperAlerts', attributes: ['tamper']]
state.deviceAttributes << [ devices: 'temperatures', attributes: ['temperature']]
state.deviceAttributes << [ devices: 'thermostats', attributes: ['temperature', 'heatingSetpoint', 'coolingSetpoint', 'thermostatSetpoint', 'thermostatMode', 'thermostatFanMode', 'thermostatOperatingState', 'thermostatSetpointMode', 'scheduledSetpoint', 'optimisation', 'windowFunction']]
state.deviceAttributes << [ devices: 'threeAxis', attributes: ['threeAxis']]
state.deviceAttributes << [ devices: 'touchs', attributes: ['touch']]
state.deviceAttributes << [ devices: 'uvs', attributes: ['ultravioletIndex']]
state.deviceAttributes << [ devices: 'valves', attributes: ['contact']]
state.deviceAttributes << [ devices: 'volts', attributes: ['voltage']]
state.deviceAttributes << [ devices: 'waterSensors', attributes: ['water']]
state.deviceAttributes << [ devices: 'windowShades', attributes: ['windowShade']]
// Configure Scheduling:
state.softPollingInterval = settings.prefSoftPollingInterval.toInteger()
state.writeInterval = settings.writeInterval
manageSchedules()
// Configure Subscriptions:
manageSubscriptions()
// Clean up old state variables
state.remove("queuedData")
}
/*****************************************************************************************************************
* Event Handlers:
*****************************************************************************************************************/
/**
* handleModeEvent(evt)
*
* Log Mode changes.
**/
def handleModeEvent(evt) {
logger("handleModeEvent(): Mode changed to: ${evt.value}", "info")
def locationName = escapeStringForInfluxDB(location.name)
def mode = '"' + escapeStringForInfluxDB(evt.value) + '"'
def data = "_stMode,locationName=${locationName} mode=${mode}"
queueToInfluxDb(data)
}
/**
* handleEvent(evt)
*
* Builds data to send to InfluxDB.
* - Escapes and quotes string values.
* - Calculates logical binary values where string values can be
* represented as binary values (e.g. contact: closed = 1, open = 0)
**/
def handleEvent(evt) {
//logger("handleEvent(): $evt.unit", "info")
//logger("handleEvent(): $evt.displayName($evt.name:$evt.unit) $evt.value", "info")
// Build data string to send to InfluxDB:
// Format: <measurement>[,<tag_name>=<tag_value>] field=<field_value>
// If value is an integer, it must have a trailing "i"
// If value is a string, it must be enclosed in double quotes.
String measurement = evt.name
// tags:
String deviceId = evt?.deviceId?.toString()
String deviceName = escapeStringForInfluxDB(evt?.displayName)
String hubName = escapeStringForInfluxDB(evt?.device?.device?.hub?.name?.toString())
String locationName = escapeStringForInfluxDB(location.name)
String unit = escapeStringForInfluxDB(evt.unit)
String value = escapeStringForInfluxDB(evt.value)
String valueBinary = ''
String data = "${measurement},deviceId=${deviceId},deviceName=${deviceName},hubName=${hubName},locationName=${locationName}"
// Unit tag and fields depend on the event type:
// Most string-valued attributes can be translated to a binary value too.
if ('acceleration' == evt.name) { // acceleration: Calculate a binary value (active = 1, inactive = 0)
unit = 'acceleration'
value = '"' + value + '"'
valueBinary = ('active' == evt.value) ? '1i' : '0i'
data += ",unit=${unit} value=${value},valueBinary=${valueBinary}"
}
else if ('alarm' == evt.name) { // alarm: Calculate a binary value (strobe/siren/both = 1, off = 0)
unit = 'alarm'
value = '"' + value + '"'
valueBinary = ('off' == evt.value) ? '0i' : '1i'
data += ",unit=${unit} value=${value},valueBinary=${valueBinary}"
}
else if ('button' == evt.name) { // button: Calculate a binary value (held = 1, pushed = 0)
unit = 'button'
value = '"' + value + '"'
valueBinary = ('pushed' == evt.value) ? '0i' : '1i'
data += ",unit=${unit} value=${value},valueBinary=${valueBinary}"
}
else if ('carbonMonoxide' == evt.name) { // carbonMonoxide: Calculate a binary value (detected = 1, clear/tested = 0)
unit = 'carbonMonoxide'
value = '"' + value + '"'
valueBinary = ('detected' == evt.value) ? '1i' : '0i'
data += ",unit=${unit} value=${value},valueBinary=${valueBinary}"
}
else if ('consumableStatus' == evt.name) { // consumableStatus: Calculate a binary value ("good" = 1, "missing"/"replace"/"maintenance_required"/"order" = 0)
unit = 'consumableStatus'
value = '"' + value + '"'
valueBinary = ('good' == evt.value) ? '1i' : '0i'
data += ",unit=${unit} value=${value},valueBinary=${valueBinary}"
}
else if ('contact' == evt.name) { // contact: Calculate a binary value (closed = 1, open = 0)
unit = 'contact'
value = '"' + value + '"'
valueBinary = ('closed' == evt.value) ? '1i' : '0i'
data += ",unit=${unit} value=${value},valueBinary=${valueBinary}"
}
else if ('door' == evt.name) { // door: Calculate a binary value (closed = 1, open/opening/closing/unknown = 0)
unit = 'door'
value = '"' + value + '"'
valueBinary = ('closed' == evt.value) ? '1i' : '0i'
data += ",unit=${unit} value=${value},valueBinary=${valueBinary}"
}
else if ('lock' == evt.name) { // door: Calculate a binary value (locked = 1, unlocked = 0)
unit = 'lock'
value = '"' + value + '"'
valueBinary = ('locked' == evt.value) ? '1i' : '0i'
data += ",unit=${unit} value=${value},valueBinary=${valueBinary}"
}
else if ('motion' == evt.name) { // Motion: Calculate a binary value (active = 1, inactive = 0)
unit = 'motion'
value = '"' + value + '"'
valueBinary = ('active' == evt.value) ? '1i' : '0i'
data += ",unit=${unit} value=${value},valueBinary=${valueBinary}"
}
else if ('mute' == evt.name) { // mute: Calculate a binary value (muted = 1, unmuted = 0)
unit = 'mute'
value = '"' + value + '"'
valueBinary = ('muted' == evt.value) ? '1i' : '0i'
data += ",unit=${unit} value=${value},valueBinary=${valueBinary}"
}
else if ('presence' == evt.name) { // presence: Calculate a binary value (present = 1, not present = 0)
unit = 'presence'
value = '"' + value + '"'
valueBinary = ('present' == evt.value) ? '1i' : '0i'
data += ",unit=${unit} value=${value},valueBinary=${valueBinary}"
}
else if ('shock' == evt.name) { // shock: Calculate a binary value (detected = 1, clear = 0)
unit = 'shock'
value = '"' + value + '"'
valueBinary = ('detected' == evt.value) ? '1i' : '0i'
data += ",unit=${unit} value=${value},valueBinary=${valueBinary}"
}
else if ('sleeping' == evt.name) { // sleeping: Calculate a binary value (sleeping = 1, not sleeping = 0)
unit = 'sleeping'
value = '"' + value + '"'
valueBinary = ('sleeping' == evt.value) ? '1i' : '0i'
data += ",unit=${unit} value=${value},valueBinary=${valueBinary}"
}
else if ('smoke' == evt.name) { // smoke: Calculate a binary value (detected = 1, clear/tested = 0)
unit = 'smoke'
value = '"' + value + '"'
valueBinary = ('detected' == evt.value) ? '1i' : '0i'
data += ",unit=${unit} value=${value},valueBinary=${valueBinary}"
}
else if ('sound' == evt.name) { // sound: Calculate a binary value (detected = 1, not detected = 0)
unit = 'sound'
value = '"' + value + '"'
valueBinary = ('detected' == evt.value) ? '1i' : '0i'
data += ",unit=${unit} value=${value},valueBinary=${valueBinary}"
}
else if ('switch' == evt.name) { // switch: Calculate a binary value (on = 1, off = 0)
unit = 'switch'
value = '"' + value + '"'
valueBinary = ('on' == evt.value) ? '1i' : '0i'
data += ",unit=${unit} value=${value},valueBinary=${valueBinary}"
}
else if ('tamper' == evt.name) { // tamper: Calculate a binary value (detected = 1, clear = 0)
unit = 'tamper'
value = '"' + value + '"'
valueBinary = ('detected' == evt.value) ? '1i' : '0i'
data += ",unit=${unit} value=${value},valueBinary=${valueBinary}"
}
else if ('thermostatMode' == evt.name) { // thermostatMode: Calculate a binary value (<any other value> = 1, off = 0)
unit = 'thermostatMode'
value = '"' + value + '"'
valueBinary = ('off' == evt.value) ? '0i' : '1i'
data += ",unit=${unit} value=${value},valueBinary=${valueBinary}"
}
else if ('thermostatFanMode' == evt.name) { // thermostatFanMode: Calculate a binary value (<any other value> = 1, off = 0)
unit = 'thermostatFanMode'
value = '"' + value + '"'
valueBinary = ('off' == evt.value) ? '0i' : '1i'
data += ",unit=${unit} value=${value},valueBinary=${valueBinary}"
}
else if ('thermostatOperatingState' == evt.name) { // thermostatOperatingState: Calculate a binary value (heating = 1, <any other value> = 0)
unit = 'thermostatOperatingState'
value = '"' + value + '"'
valueBinary = ('heating' == evt.value) ? '1i' : '0i'
data += ",unit=${unit} value=${value},valueBinary=${valueBinary}"
}
else if ('thermostatSetpointMode' == evt.name) { // thermostatSetpointMode: Calculate a binary value (followSchedule = 0, <any other value> = 1)
unit = 'thermostatSetpointMode'
value = '"' + value + '"'
valueBinary = ('followSchedule' == evt.value) ? '0i' : '1i'
data += ",unit=${unit} value=${value},valueBinary=${valueBinary}"
}
else if ('threeAxis' == evt.name) { // threeAxis: Format to x,y,z values.
unit = 'threeAxis'
def valueXYZ = evt.value.split(",")
def valueX = valueXYZ[0]
def valueY = valueXYZ[1]
def valueZ = valueXYZ[2]
data += ",unit=${unit} valueX=${valueX}i,valueY=${valueY}i,valueZ=${valueZ}i" // values are integers.
}
else if ('touch' == evt.name) { // touch: Calculate a binary value (touched = 1, "" = 0)
unit = 'touch'
value = '"' + value + '"'
valueBinary = ('touched' == evt.value) ? '1i' : '0i'
data += ",unit=${unit} value=${value},valueBinary=${valueBinary}"
}
else if ('optimisation' == evt.name) { // optimisation: Calculate a binary value (active = 1, inactive = 0)
unit = 'optimisation'
value = '"' + value + '"'
valueBinary = ('active' == evt.value) ? '1i' : '0i'
data += ",unit=${unit} value=${value},valueBinary=${valueBinary}"
}
else if ('windowFunction' == evt.name) { // windowFunction: Calculate a binary value (active = 1, inactive = 0)
unit = 'windowFunction'
value = '"' + value + '"'
valueBinary = ('active' == evt.value) ? '1i' : '0i'
data += ",unit=${unit} value=${value},valueBinary=${valueBinary}"
}
else if ('touch' == evt.name) { // touch: Calculate a binary value (touched = 1, <any other value> = 0)
unit = 'touch'
value = '"' + value + '"'
valueBinary = ('touched' == evt.value) ? '1i' : '0i'
data += ",unit=${unit} value=${value},valueBinary=${valueBinary}"
}
else if ('water' == evt.name) { // water: Calculate a binary value (wet = 1, dry = 0)
unit = 'water'
value = '"' + value + '"'
valueBinary = ('wet' == evt.value) ? '1i' : '0i'
data += ",unit=${unit} value=${value},valueBinary=${valueBinary}"
}
else if ('windowShade' == evt.name) { // windowShade: Calculate a binary value (closed = 1, <any other value> = 0)
unit = 'windowShade'
value = '"' + value + '"'
valueBinary = ('closed' == evt.value) ? '1i' : '0i'
data += ",unit=${unit} value=${value},valueBinary=${valueBinary}"
}
// Catch any other event with a string value that hasn't been handled:
else if (evt.value ==~ /.*[^0-9\.,-].*/) { // match if any characters are not digits, period, comma, or hyphen.
logger("handleEvent(): Found a string value that's not explicitly handled: Device Name: ${deviceName}, Event Name: ${evt.name}, Value: ${evt.value}", "warn")
value = '"' + value + '"'
data += ",unit=${unit} value=${value}"
}
// Catch any other general numerical event (carbonDioxide, power, energy, humidity, level, temperature, ultravioletIndex, voltage, etc).
else {
data += ",unit=${unit} value=${value}"
}
// Queue data for later write to InfluxDB
//logger("$data", "info")
queueToInfluxDb(data)
}
/*****************************************************************************************************************
* Main Commands:
*****************************************************************************************************************/
/**
* softPoll()
*
* Executed by schedule.
*
* Forces data to be posted to InfluxDB (even if an event has not been triggered).
* Doesn't poll devices, just builds a fake event to pass to handleEvent().
*
* Also calls LogSystemProperties().
**/
def softPoll() {
logger("softPoll()", "trace")
logSystemProperties()
if (accessAllAttributes == false) {
// Iterate over each attribute for each device, in each device collection in deviceAttributes:
def devs // temp variable to hold device collection.
state.deviceAttributes.each { da ->
devs = settings."${da.devices}"
if (devs && (da.attributes)) {
devs.each { d ->
da.attributes.each { attr ->
if (d.hasAttribute(attr) && d.latestState(attr)?.value != null) {
logger("softPoll(): Softpolling device ${d} for attribute: ${attr}", "info")
// Send fake event to handleEvent():
for (i = 0; i < 100; i++) {
handleEvent([
name: attr,
value: d.latestState(attr)?.value,
unit: d.latestState(attr)?.unit,
device: d,
deviceId: d.id,
displayName: d.displayName
])
}
}
}
}
}
}
} else {
state.selectedAttr.each { entry ->
d = getDeviceObj(entry.key)
entry.value.each { attr ->
if (d.hasAttribute(attr) && d.latestState(attr)?.value != null) {
logger("softPoll(): Softpolling device ${d} for attribute: ${attr}", "info")
// Send fake event to handleEvent():
handleEvent([
name: attr,
value: d.latestState(attr)?.value,
unit: d.latestState(attr)?.unit,
device: d,
deviceId: d.id,
displayName: d.displayName
])
}
}
}
}
}
/**
* logSystemProperties()
*
* Generates measurements for SmartThings system (hubs and locations) properties.
**/
def logSystemProperties() {
logger("logSystemProperties()", "trace")
def locationName = '"' + escapeStringForInfluxDB(location.name) + '"'
// Location Properties:
if (prefLogLocationProperties) {
try {
def tz = '"' + escapeStringForInfluxDB(location.timeZone.ID.toString()) + '"'
def mode = '"' + escapeStringForInfluxDB(location.mode) + '"'
def times = getSunriseAndSunset()
def srt = '"' + times.sunrise.format("HH:mm", location.timeZone) + '"'
def sst = '"' + times.sunset.format("HH:mm", location.timeZone) + '"'
def data = "_heLocation,locationName=${locationName},latitude=${location.latitude},longitude=${location.longitude},timeZone=${tz} mode=${mode},sunriseTime=${srt},sunsetTime=${sst}"
queueToInfluxDb(data)
//log.debug("LocationData = ${data}")
} catch (e) {
logger("logSystemProperties(): Unable to log Location properties: ${e}", "error")
}
}
// Hub Properties:
if (prefLogHubProperties) {
location.hubs.each { h ->
try {
def hubName = '"' + escapeStringForInfluxDB(h.name.toString()) + '"'
def hubIP = '"' + escapeStringForInfluxDB(h.localIP.toString()) + '"'
def firmwareVersion = '"' + escapeStringForInfluxDB(h.firmwareVersionString) + '"'
def data = "_heHub,locationName=${locationName},hubName=${hubName},hubIP=${hubIP} "
data += "firmwareVersion=${firmwareVersion}"
//log.debug("HubData = ${data}")
queueToInfluxDb(data)
} catch (e) {
logger("logSystemProperties(): Unable to log Hub properties: ${e}", "error")
}
}
}
}
def queueToInfluxDb(data) {
loggerQueue = getLoggerQueue()
// Add timestamp (influxdb does this automatically, but since we're batching writes, we need to add it
long timeNow = (new Date().time) * 1e6 // Time is in milliseconds, needs to be in nanoseconds
data += " ${timeNow}"
loggerQueue.add(data)
if (loggerQueue.size() > (settings.prefWriteQueueLimit ?: 100)) {
logger("Queue size is too big, triggering write now", "info")
writeQueuedDataToInfluxDb()
}
}
def writeQueuedDataToInfluxDb() {
loggerQueue = getLoggerQueue()
if (loggerQueue.size() == 0) {
logger("No queued data to write to InfluxDB", "info")
return
}
logger("Writing queued data of size ${loggerQueue.size()} out", "info")
String writeData = loggerQueue.toArray().join('\n')
postToInfluxDB(writeData)
loggerQueue.clear()
}
/**
* postToInfluxDB()
*
* Posts data to InfluxDB.
*
* Uses hubAction instead of httpPost() in case InfluxDB server is on the same LAN as the Smartthings Hub.
**/
def postToInfluxDB(data) {
if (state.uri == null) {
// Failsafe if using an old config
setupDB()
}
//logger("postToInfluxDB(): Posting data to InfluxDB: ${state.uri}, Data: [${data}]", "info")
// Hubitat Async http Post
try {
def postParams = [
uri: state.uri,
requestContentType: 'application/json',
contentType: 'application/json',
headers: state.headers,
body : data
]
asynchttpPost('handleInfluxResponse', postParams)
} catch (e) {
logger("postToInfluxDB(): Something went wrong when posting: ${e}", "error")
}
}
/**
* handleInfluxResponse()
*
* Handles response from post made in postToInfluxDB().
**/
def handleInfluxResponse(hubResponse, data) {
//logger("postToInfluxDB(): status of post call is: ${hubResponse.status}", "info")
if (hubResponse.status >= 400) {
logger("postToInfluxDB(): Something went wrong! Response from InfluxDB: Status: ${hubResponse.status}, Headers: ${hubResponse.headers}, Data: ${data}", "error")
}
}
/*****************************************************************************************************************
* Private Helper Functions:
*****************************************************************************************************************/
/**
* getLoggerQueue()
*
* Return the loggerQueue for this instance
**/
private getLoggerQueue() {
// Approach 1
//loggerQueueInstance = state.loggerQueue
//if (loggerQueueInstance == null) {
// loggerQueueInstance = []
// state.loggerQueue = loggerQueueInstance
//}
// Approach 2
//String id = app.getId()
//loggerQueueInstance = loggerQueueMap.get(id)
//if (loggerQueueInstance == null) {
// loggerQueueInstance = new java.util.concurrent.ConcurrentLinkedQueue()
// loggerQueueMap.put(id, loggerQueueInstance)
//}
return loggerQueueInstance
}
/**
* removeLoggerQueue()
*
* Return the loggerQueue for this instance
**/
private removeLoggerQueue() {
// Approach 1
//return
// Approach 2
//String id = app.getId()
//loggerQueueInstance = loggerQueueMap.get(id)
//if (loggerQueueInstance) {
// loggerQueueMap.remove(id)
//}
}
/**
* setupDB()
*
* Set up the database uri and header state variables.
**/
private setupDB() {
String uri
def headers = [:]
if (settings?.prefDatabaseTls) {
uri = "https://"
} else {
uri = "http://"
}
uri += settings.prefDatabaseHost
if (settings?.prefDatabasePort) {
uri += ":" + settings.prefDatabasePort
}
if (settings?.prefInfluxVer == "2") {
uri += "/api/v2/write?org=${settings.prefOrg}&bucket=${settings.prefBucket}"
} else {
// Influx version 1
uri += "/write?db=${settings.prefDatabaseName}"
}
if (settings.prefAuthType == null || settings.prefAuthType == "basic") {
if (settings.prefDatabaseUser && settings.prefDatabasePass) {
def userpass = "${settings.prefDatabaseUser}:${settings.prefDatabasePass}"
headers.put("Authorization", "Basic " + userpass.bytes.encodeBase64().toString())
}
} else if (settings.prefAuthType == "token") {
headers.put("Authorization", "Token ${settings.prefDatabaseToken}")
}
state.uri = uri
state.headers = headers
logger("New URI: ${uri}", "info")
// Clean up old state vars if present
state.remove("databaseHost")
state.remove("databasePort")
state.remove("databaseName")
state.remove("databasePass")
state.remove("databaseUser")
state.remove("path")
}
/**
* manageSchedules()
*
* Configures/restarts scheduled tasks:
* softPoll() - Run every {state.softPollingInterval} minutes.
**/
private manageSchedules() {
logger("manageSchedules()", "trace")
// Generate a random offset (1-60):
Random rand = new Random(now())
def randomOffset = 0
try {
unschedule(softPoll)
unschedule(writeQueuedDataToInfluxDb)
}
catch (e) {
// logger("manageSchedules(): Unschedule failed!", "error")
}
randomOffset = rand.nextInt(50)
if (state.softPollingInterval > 0) {
logger("manageSchedules(): Scheduling softpoll to run every ${state.softPollingInterval} minutes (offset of ${randomOffset} seconds).", "trace")
schedule("${randomOffset} 0/${state.softPollingInterval} * * * ?", "softPoll")
}
randomOffset = randomOffset + 8
schedule("${randomOffset} 0/${state.writeInterval} * * * ?", "writeQueuedDataToInfluxDb")
}
/**
* manageSubscriptions()
*
* Configures subscriptions.
**/
private manageSubscriptions() {
logger("manageSubscriptions()", "trace")
// Unsubscribe:
unsubscribe()
// Subscribe to mode events:
if (prefLogModeEvents) subscribe(location, "mode", handleModeEvent)
if (!accessAllAttributes) {
// Subscribe to device attributes (iterate over each attribute for each device collection in state.deviceAttributes):
def devs // dynamic variable holding device collection.
state.deviceAttributes.each { da ->
devs = settings."${da.devices}"
if (devs && (da.attributes)) {
da.attributes.each { attr ->
logger("manageSubscriptions(): Subscribing to attribute: ${attr}, for devices: ${da.devices}", "info")
// There is no need to check if all devices in the collection have the attribute.
subscribe(devs, attr, handleEvent)
}
}
}
} else {
state.selectedAttr.each { entry ->
d = getDeviceObj(entry.key)
entry.value.each { attr ->
logger("manageSubscriptions(): Subscribing to attribute: ${attr}, for device: ${d}", "info")
subscribe(d, attr, handleEvent)
}
}
}
}
/**
* logger()
*
* Wrapper function for all logging.
**/
private logger(msg, level = "debug") {
switch (level) {
case "error":
if (state.loggingLevelIDE >= 1) log.error msg
break
case "warn":
if (state.loggingLevelIDE >= 2) log.warn msg
break
case "info":
if (state.loggingLevelIDE >= 3) log.info msg
break
case "debug":
if (state.loggingLevelIDE >= 4) log.debug msg
break
case "trace":
if (state.loggingLevelIDE >= 5) log.trace msg
break
default:
log.debug msg
break
}
}
/**
* escapeStringForInfluxDB()
*
* Escape values to InfluxDB.
*
* If a tag key, tag value, or field key contains a space, comma, or an equals sign = it must
* be escaped using the backslash character \. Backslash characters do not need to be escaped.
* Commas and spaces will also need to be escaped for measurements, though equals signs = do not.
*
* Further info: https://docs.influxdata.com/influxdb/v0.10/write_protocols/write_syntax/
**/
private String escapeStringForInfluxDB(String str) {
//logger("$str", "info")
if (str) {
str = str.replaceAll(" ", "\\\\ ") // Escape spaces.
str = str.replaceAll(",", "\\\\,") // Escape commas.
str = str.replaceAll("=", "\\\\=") // Escape equal signs.
str = str.replaceAll("\"", "\\\\\"") // Escape double quotes.
//str = str.replaceAll("'", "_") // Replace apostrophes with underscores.
}
else {
str = 'null'
}
return str
}
