Also, will be possible to distinguish between device/app with no events and device/app with events but still loading?
Maybe adding a loading message/spinning..
Thanks
We have talked about it, no imediate plans to implement adjustable retention.
Some attributes are only set once, but referenced by apps, numberOfButtons, maxCodes etc.
So pruning on overall event count really isn't viable.
Thank you for the response, problem is that actually we have lots of unneeded events that just slow down the UI and increase DB and backup size, so sooner or later something should be done to improve it: I have few devices (like Apixu) that really takes 1 minute or more to load the event page..
Sure, if they are not needed, they should not be produced, or their production interval should be reduced.
Cleaning them up is one thing, however that won't help with reducing their production and the system load imposed by that.
Keep in mind, HE is an event driven system, events are supposed to trigger things, with few exceptions, if an attribute event had no purpose, it should not be produced.
If your custom device is producing spam like events, I would just fix the driver.
Yep, infact I've fixed my own written driver for thermostat but how to remove all old bloatware?
not following, bloatware in the driver?: comment out what you don't need, then fix anything what breaks as a result...
In regards to the extra events, you're stuck with the distinct 1K (unless you want to start over that is), however I suspect the issue you're having is driven more by the excess processing of the events the driver used to be producing, rather than those stored in the db, but I don't know for sure.
The clean up runs every day, early in the morning.
[quote="mike.maxwell, post:11, topic:2621"]
In regards to the extra events, you're stuck with the distinct 1K (unless you want to start over that is),
[/quote]Yes, I was referring to old events that I don't need anymore.
Anyway also cleaning up the driver is not a solution, because if it's not after 1 month, maybe after 1 year I'll anyway have all attributes reached 1000 events, so I'll have (considering apixu with 50 attributes) more than 50000 events, and also now that are around 30000 it takes more than 1 minute to load
clean up the driver, then delete the device (the events belonging to this device will be deleted), re install the cleaned up device and you should be good to go.
What do you mean for cleanup? If the driver exposes 30 properties/attributes, how I can cleanup it? Do I need to create two drivers with 15 properties each? It will be a mess..
My ApiXu device (events) takes 15seconds to populate.
"Showing 1 to 10 of 58,665 entries"
If I'm reading Mike's reply correctly, he's saying: ok, you have 50 attributes, go into YOUR copy of the driver and comment out all but the 8 you use. Then, delete the virtual device, and create a new. The new one will only have the 8 events.
In fact, it's such a good idea, perhaps @bangali is interested in exposing a Selection of Attributes.
I'm only using 15, for example. I could easily get rid of the rest BUT with only 15 seconds load time, I'm not very motivated. 
1 Like
Then on each update I need to manually remind which one were commented out and I can't just copy&paste new version
I've tried again now and it took the same time, so maybe there was something else earlier that has slowed it down.
Up to 20/30 secs I think it's reasonable, sorry to had bothered you @mike.maxwell
So as a result of the recent exchange on this thread, now @gabriele is requesting that I add an preferences toggle option to be able to turn on/off all the custom date/time stamp attributes in the set of Xiaomi / Aqara device drivers I am maintaining.
The reason I added those custom attributes was per some users' request, so that they would be able to view on a Hubitat Dashboard when a particular event last occurred. So for example, they could create a dashboard tile that displays the time/date stamp of the most recent motion detected event for a motion sensor, or for the most recent button pressed event for a button.
So these custom attribute events would fall into what you call "exceptions" because they do not trigger things.
Personally, I would just look at the events list page for a particular device if I want to see the last instance of a device-related event, but I can totally understand how some users would prefer a much easier way to view that information at a glance.
I don't consider the events frequency of any Xiaomi / Aqara devices to be of a particular high level, so it seemed reasonable to add those custom attributes to create time/date stamp events which only some portion of users will make use of.
If these types of events are actually considered "spam-like" events, and are actually having a major impact on hub performance, then this is the first time I've heard it. Where is the developer documentation, or FAQ entries that educate developers and users that custom attributes used for non-triggering events is a frivolous use of system resources?
I am also very open to using an alternate method of allowing users easy access to view the most recent time/date stamp for a particular event without the use of custom attributes. Is there a way to do this on the Dashboard - say for example, by selecting a supported attribute (such as motion detected) and then setting the tile to display the most recent event's time/date stamp?
I think this would be my test of "excessive" or not....
I open a motion sensor device page and then click Events.. the page displays "instantly" and down at the bottom there's:
Showing 1 to 10 of 2,110 entries
When I look at ApiXU virtual device and click Events, the page takes 15 seconds or more and at the bottom is:
Showing 1 to 10 of 58,812 entries
Almost 30 times as many AND there's 53 attributes.
Somewhere between lies the line I suspect. I'm getting weather and ALL of the attributes are being updated every time. "wind" seems like it deserves to be often, but "twilight_begin" ? Do I need that updated more than once a day? In other words, there's probably a lot of tuning possible with ApiXU and maybe some other drivers.
I have zero Xiaomi / Aqara devices so I can't perform my crude test.
1 Like
I have some Xiaomi devices and they don't produce 50,000 events for sure, it was a request to cleanup a bit the events because when I'm looking for old events they are quite hard to understand, considering that there are a lots of battery entries (followed by checkin entry), so they will be a bit more "tidy", but it wasn't related to performances in this case.
Both devices referenced in this thread are sending events with "isStateChange: true" I would check to see if that is truly necessary. If that flag is not set then the system will determine if the attribute has changed and will store it in the event table only if that is the case.
1 Like
For the Xiaomi / Aqara device drivers I maintain, I'll gladly remove isStateChange: true from all events sent, but this won't reduce the number of events seen for the custom time/date stamp attributes that the device drivers make available to users who want to see the time/date of the most recent XXX event in their Hubitat Dashboard.
So, I'd still like to know if there's a way for users to display just the date/time of the most recent occurrence of a posted event in a Hubitat Dashboard. If that functionality already exists, then I could get rid of all of the custom time/date stamp attributes I added to the device drivers just for that purpose.
If that functionality isn't available in the Dashboard, then I'll gladly start a new thread in the Requests section to ask for it, as it seems to be something a lot of users want.
First I'm seeing this request. What would the use case be? Just a tile that shows last activity for that device? Or is this per attribute?
I decided to experiment with this and took Bangali's code and removed every occurrence of "isStateChange: true" and then created a new virtual with that modified driver. It made a big difference in the Event log growth, but there were still a lot of "daily values" being polled every 30 mins and some once-and-done values, such as lat, long, city name, etc. What I decided is: I only use about 20 of the 50+ attributes. I split the attributes into two portions and then only doing a sendEvent() for "my 20".
After 4 days, I now have:
_
Showing 1 to 10 of 1,281 entries
_
When opening the Events page of the device.
@bangali has been out of town and I've discussed these alterations with him and I think he might include some in a future update.
I've noticed this job might not be running / working correctly. I have a device with 3,600+ entries dating back over a month. Any better way to clear these out then deleting the device?
It's the last 1k per attribute, per device...
