right.
reduce the number of DB updates.
users would have to build the list outside of the app then submit it. can do but not very convenient for users. 
For example...
I modified the source to move certain events to the end of the Poll() method and then surrounded everything I didn't want within an "if not..." and put a switch into the UI.
I am only needing (today) a limited subset of data:
// Top 10 Group of the most popular attributes to return from the Poll. sendEvent(name: "city", value: (cityName ?: obs.location.name), displayed: true) sendEvent(name: "temperature", value: (isFahrenheit ? obs.current.temp_f : obs.current.temp_c), unit: "${(isFahrenheit ? 'F' : 'C')}", displayed: true) sendEvent(name: "visual", value: '<img src=' + imgName + '>', displayed: true) sendEvent(name: "humidity", value: obs.current.humidity, unit: "%", displayed: true) sendEvent(name: "pressure", value: (isFahrenheit ? obs.current.pressure_in : obs.current.pressure_mb), unit: "${(isFahrenheit ? 'IN' : 'MBAR')}", displayed: true) sendEvent(name: "weather", value: obs.current.condition.text, displayed: true) sendEvent(name: "forecastIcon", value: getWUIconName(obs.current.condition.code, 1), displayed: true) sendEvent(name: "feelsLike", value: (isFahrenheit ? obs.current.feelslike_f : obs.current.feelslike_c), unit: "${(isFahrenheit ? 'F' : 'C')}", displayed: true) sendEvent(name: "wind", value: (isFahrenheit ? obs.current.wind_mph : obs.current.wind_kph), unit: "${(isFahrenheit ? 'MPH' : 'KPH')}", displayed: true) sendEvent(name: "percentPrecip", value: (isFahrenheit ? obs.current.precip_in : obs.current.precip_mm), unit: "${(isFahrenheit ? 'IN' : 'MM')}", displayed: true) sendEvent(name: "wind_dir", value: obs.current.wind_dir, displayed: true) sendEvent(name: "localSunrise", value: localSunrise, displayed: true) sendEvent(name: "localSunset", value: localSunset, displayed: true) sendEvent(name: "visualDayPlus1", value: '<img src=' + imgName + '>', displayed: true) sendEvent(name: "temperatureHighDayPlus1", value: (isFahrenheit ? obs.forecast.forecastday[0].day.maxtemp_f : obs.forecast.forecastday[0].day.maxtemp_c), unit: "${(isFahrenheit ? 'F' : 'C')}", displayed: true)
Everything else is not processed. This reduced my DB Event logs significantly. In 4 days I've gotten 2000+ event entries. By comparison, one of my well used Motion sensor is seeing 5000+ event entries since last April. (287 days since april.. (5000/287)*4 = 70 vs 2000 and that's AFTER removing half to 2/3 of the events.)
In other words, ApiXU is a busy device filling the event logs with it's every 5 min and every 30 min updates. Reducing the quantity is a benefit.
My next edit will be to change the hard coded 5 Lux reading to 10. For me, where I live, I use LUX to detect "gloomy days" and add some light. I sure don't need that light going on and off every 5 mins. Every 10 would be OK 
1 Like
I wonder if some additional logic could be added to only send events in the case that something actually changed. ST distinguished between events and state change events with isStateChange in the sendEvent method, but it sounds like you guys are more interested in just fundamentally reducing the number of base events.
My background is in the Industrial Internet of Things space and a common feature that was used in collecting data was 'compression' and 'exception' settings. In this case, I think the concept of 'exception' could help. The basic concept is to only send the event across the wire when there was a meaningful change.
So in the case of a driver with a lot of events, the driver could first check what the current value is before sending across a new value. If the value is the exact same, it doesn't need to be sent again.
The industrial space takes this a bit further with maximum time windows and exception deviation settings. The time window basically let you say that you would always send an event if another event hadn't been sent in the past X minutes. The exception deviation settings let you specify a minimum value or percentage value change before triggering the sending of an event. So if a temperature sensor was only capable of measuring 0.1° change, then you could set the deviation to 0.1 as well... that way if the temperature changed by 0.001 or some other 'noise' in the signal, it could be excluded and not sent.
sendEvent does that already and was the core change of v4.3 (remove isStateChange = true) which had been forcing every event to be logged, same or not.
But we're talking about Weather. The Wind changes Lux changes, every Poll. 12times an hour (every 5min) the Event DB gets two entries. Every 30 min the Event DB gets between 2 and 7, like this:
wind 5.6 MPH DEVICE 2019-01-26 06:57:09.880 AM PST
weather Sunny DEVICE 2019-01-26 06:57:09.880 AM PST
feelsLike 39.3 F DEVICE 2019-01-26 06:57:09.880 AM PST
humidity 56 % DEVICE 2019-01-26 06:57:09.879 AM PST
temperature 42.8 F DEVICE 2019-01-26 06:57:09.879 AM PST
wind_dir N DEVICE 2019-01-26 06:27:09.888 AM PST
wind 4.3 MPH DEVICE 2019-01-26 06:27:09.887 AM PST
feelsLike 38.2 F DEVICE 2019-01-26 06:27:09.887 AM PST
lastXUupdate 2019-01-26 06:27 DEVICE 2019-01-26 06:27:09.694 AM PST
wind_dir NNE DEVICE 2019-01-26 05:57:09.925 AM PST
wind 6.9 MPH DEVICE 2019-01-26 05:57:09.925 AM PST
feelsLike 36.4 F DEVICE 2019-01-26 05:57:09.924 AM PST
lastXUupdate 2019-01-26 05:57 DEVICE 2019-01-26 05:57:09.698 AM PST
Of the 15 pieces of data I care about, at 6:57 am 5 of them changed and got added to the Event DB.
When I go to the Event log for ApiXU, and pick 100 entries, the oldest event is from 12:27am (midnight:30 )
100 events in 7 hrs.
For me, this is significantly better and unfortunately the logs I pasted show a sunrise, in which "maximum time windows and exception deviation settings" would be exceeded. What's that old adage? "90%of the work takes 90% of the allocated time, the next 10% takes another 90%"
I see what you're saying. For me, each of those looks like a legitimate change in value, so I'd be happy with those event being sent... then again, I'm not concerned about DB event logs at the moment! 
Looking through the posts above, am I to understand the issue is it make the event page slow to load for the APIXU driver and may increase the size of the database/backup if it's done before the nightly event purge (1k retention) is done? Purge Device/Hub/Location events - #15 by csteele
In that case, wouldn't it be a valid update for the sunrise values? Or are you saying you just don't care about sunrise values at all and would want to filter them out?
It seems like having the option of filtering those unneeded attributes would be the 'easiest' solution here... sounds like the remaining issue preventing implemention of that is not having a multi-select input in the device drivers. Has anyone asked @mike.maxwell @chuck.schwer or any of the HE guys if they have plans to implement a multi-select for device driver preferences?
@csteele had shared this data and his changes of selecting only a subset of data to publish … with me. thats what prompted me to look in to this.
for obvious reasons i didnt want to be the arbiter of the subset of attributes that should be published. so the plan was to allow user selection of attributes to publish.
that plan is currently on hold because currently drivers dont support multi-selection from list.
I was trying to say we can't visualize the importance of "maximum time windows and exception deviation settings" with only 3 samples I pasted and even then, values that one would expect to exceed and therefore be logged. In other words, I was attempting to say: "that's a cool thing to investigate, but the data set isn't the part I'd want to filter."
@bangali has been very gracious in accepting my ramblings and trying to give us a benefit. I really appreciate it. I have another idea, just this minute to perhaps assist with multi-select. I'll cobble it together and see where it goes. Could be a complete waste of time.
4 Likes
@bangali at the risk of being painfully american, can you make either a 12/24hr toggle or assume if the units are imperial then time == 12hr format?
Just helps cover a mental deficiency on my side where I stare perplexed at the dashboard icon for a while doing conversions in my head.
Thanks for any consideration!
2 Likes
Although I use imperial units, I prefer the 24hour time format. Please make this a 12/24hr toggle.
@keithcroshaw & @arnb will do. traveling a bit for work so will be a couple of weeks. but will do.
2 Likes
I'm in the same boat. Thanks for the reply.
1 Like
Logs were getting eaten up...I just went through and commented out all of the log.debug entries, and changed the one entry that logs the poll command from a log.debug to log.info. Much cleaner logging now. Any change of getting a debug logging on/off switch at some point in the future?
I had a question also...is the forecast information, specifically perfect precip, for the next day or the next hour?
WU Night Icon Variants
@bangali I noticed that the code appears to have support for the night variants of Weather Underground keys in the forecastIcon attribute, but it appears to be hard coded to day.
On line 285 of the current release, the getWUIconName() call can be changed from:
sendEvent(name: "forecastIcon", value: getWUIconName(obs.current.condition.code, 1), displayed: true)
To:
sendEvent(name: "forecastIcon", value: getWUIconName(obs.current.condition.code, obs.current.is_day), displayed: true)
Was it hard coded for some particular reason or maybe just a remnant from testing that could be corrected?
Attributes Now Reporting in SharpTools.io
Just an update that the Hubitat 2.0.6 update includes some enhancements around this. SharpTools.io can now dynamically request which attributes to subscribe to so it can get events from 'non-standard' attributes like those used in several weather drivers.
localSunrise and localSunset impact
Also just a comment that the removal of the localSunrise and localSunset attributes means that sunrise and sunset will no longer be reflected/updated in the SharpTools weather tile.
2 Likes
yes will add this weekend. there is some pending work to add selectable attributes to publish intend to finish and push that as well.
1 Like
will check this weekend on why it was only publishing the day time icons. dont remember explicitly coding it to do so … so might just be a left over or bug.
did you want me to put the localSunrise and loclSunset attributes back in or that was a general notification for users so they update it in their sharptools config? 
1 Like
Bringing them back would be great. Otherwise users miss out on sunrise/sunset altogether. 
1 Like
updated to github with the following changes:
* Version: 5.0.0
* 3/10/2019: allow selection of which attributes to publish
* 3/10/2019: restore localSunrise and localSunset attributes
* 3/10/2019: added option for lux polling interval
* 3/10/2019: added expanded weather polling interval
2 Likes
Excellent update, @bangali!
One thing I noticed is that there is no wind or feelsLike entry in the attributesMap, so there's no way for the user to turn those on and thus they wont get published under the new sendEventPublish approach.
I also posted that as a comment to your GitHub along with a request to distinguish the description of localSunrise and local_sunrise in the attributesMap as they both read as 'Local Sunrise' (same with the sunset).
--
I haven't tested to see if it's an issue here, but one thing I've run into in the past is that if a device is defined as including a certain capability, but it doesn't publish the attributes associated with that capability, it can cause Hubitat to hang when trying to introspect the attributes.
// For Example:
// If a device indicates that is has the 'Switch' capability
// but it never defines a 'switch' attribute
// then code like this can cause the hub to hang
device.supportedAttributes?.collect {[
name: it.name,
dataType: it.dataType,
values: it.values
]},
Again, I haven't run into this in a while, so I haven't tested it recently. Ideally the platform would be able to work around this without hanging, but it's something I wanted to throw out there since you are now defining capabilities like Temperature, Illuminance, Humidity, Pressure, and UV and users have the options of disabling the publishing of those attributes.
--
One other minor comment is that once an attribute is published, there doesn't seem to be a way to 'unpublish' it... so perhaps the first post could be updated to call this out as an important configuration step?
While unchecking the option for those attributes would at least reduce the events on the system, it still keeps the attribute defined on the device which increases the burden on apps / the hub when trying to iterate through properties of the device.
Again, awesome work on the updates! Keep up the good work!
1 Like
