Purge Device/Hub/Location events

is there a way to purge the events?
(all and/or between two dates)

There is a pruning job that runs every night that leaves the last 1K events, currently that number can't be changed.
The device events leave 1K of each device attribute set.
We have discussed internally exposing the retainage count as a user configurable item, however from a data integrity standpoint it wouldn't be a good idea to delete a range in the middle of the existing set.

3 Likes

thank you for the details :slight_smile:

1 Like

I don't think this is working. I have almost 2000 events on 1 device alone. How can I manually purge?
I just checked other devices. They seem ok at around 400.
So I have 1 device with 1990 events.

Hi @mike.maxwell do you have any plan to customize events retention?

I have setup my Hubitat just three weeks ago, but I already have some troubles when looking at devices\apps events. Do you know if it's cleaning up also app events, other than devices events?

I think that clenaup should consider overall events, not attributes: consider for example ApiXU weather, it has more than 50 attributes, multiplied by 1000 for each attribute, means more than 50000 events are kept..

Or maybe add a global setting and an override on specific devices\app (if needed, otherwise it will keep global one)?

Also, will be possible to distinguish between device/app with no events and device/app with events but still loading?
Maybe adding a loading message/spinning..

Thanks

We have talked about it, no imediate plans to implement adjustable retention.

Some attributes are only set once, but referenced by apps, numberOfButtons, maxCodes etc.

So pruning on overall event count really isn't viable.

Thank you for the response, problem is that actually we have lots of unneeded events that just slow down the UI and increase DB and backup size, so sooner or later something should be done to improve it: I have few devices (like Apixu) that really takes 1 minute or more to load the event page..

Sure, if they are not needed, they should not be produced, or their production interval should be reduced.

Cleaning them up is one thing, however that won't help with reducing their production and the system load imposed by that.

Keep in mind, HE is an event driven system, events are supposed to trigger things, with few exceptions, if an attribute event had no purpose, it should not be produced.

If your custom device is producing spam like events, I would just fix the driver.

Yep, infact I've fixed my own written driver for thermostat but how to remove all old bloatware?

not following, bloatware in the driver?: comment out what you don't need, then fix anything what breaks as a result...
In regards to the extra events, you're stuck with the distinct 1K (unless you want to start over that is), however I suspect the issue you're having is driven more by the excess processing of the events the driver used to be producing, rather than those stored in the db, but I don't know for sure.
The clean up runs every day, early in the morning.

[quote="mike.maxwell, post:11, topic:2621"]
In regards to the extra events, you're stuck with the distinct 1K (unless you want to start over that is),
[/quote]Yes, I was referring to old events that I don't need anymore.
Anyway also cleaning up the driver is not a solution, because if it's not after 1 month, maybe after 1 year I'll anyway have all attributes reached 1000 events, so I'll have (considering apixu with 50 attributes) more than 50000 events, and also now that are around 30000 it takes more than 1 minute to load

clean up the driver, then delete the device (the events belonging to this device will be deleted), re install the cleaned up device and you should be good to go.

What do you mean for cleanup? If the driver exposes 30 properties/attributes, how I can cleanup it? Do I need to create two drivers with 15 properties each? It will be a mess..

My ApiXu device (events) takes 15seconds to populate.

"Showing 1 to 10 of 58,665 entries"

If I'm reading Mike's reply correctly, he's saying: ok, you have 50 attributes, go into YOUR copy of the driver and comment out all but the 8 you use. Then, delete the virtual device, and create a new. The new one will only have the 8 events.

In fact, it's such a good idea, perhaps @bangali is interested in exposing a Selection of Attributes.

I'm only using 15, for example. I could easily get rid of the rest BUT with only 15 seconds load time, I'm not very motivated. :frowning:

1 Like

Then on each update I need to manually remind which one were commented out and I can't just copy&paste new version

I've tried again now and it took the same time, so maybe there was something else earlier that has slowed it down.
Up to 20/30 secs I think it's reasonable, sorry to had bothered you @mike.maxwell

So as a result of the recent exchange on this thread, now @gabriele is requesting that I add an preferences toggle option to be able to turn on/off all the custom date/time stamp attributes in the set of Xiaomi / Aqara device drivers I am maintaining.

The reason I added those custom attributes was per some users' request, so that they would be able to view on a Hubitat Dashboard when a particular event last occurred. So for example, they could create a dashboard tile that displays the time/date stamp of the most recent motion detected event for a motion sensor, or for the most recent button pressed event for a button.

So these custom attribute events would fall into what you call "exceptions" because they do not trigger things.

Personally, I would just look at the events list page for a particular device if I want to see the last instance of a device-related event, but I can totally understand how some users would prefer a much easier way to view that information at a glance.

I don't consider the events frequency of any Xiaomi / Aqara devices to be of a particular high level, so it seemed reasonable to add those custom attributes to create time/date stamp events which only some portion of users will make use of.

If these types of events are actually considered "spam-like" events, and are actually having a major impact on hub performance, then this is the first time I've heard it. Where is the developer documentation, or FAQ entries that educate developers and users that custom attributes used for non-triggering events is a frivolous use of system resources?

I am also very open to using an alternate method of allowing users easy access to view the most recent time/date stamp for a particular event without the use of custom attributes. Is there a way to do this on the Dashboard - say for example, by selecting a supported attribute (such as motion detected) and then setting the tile to display the most recent event's time/date stamp?

I think this would be my test of "excessive" or not....

I open a motion sensor device page and then click Events.. the page displays "instantly" and down at the bottom there's:

Showing 1 to 10 of 2,110 entries

When I look at ApiXU virtual device and click Events, the page takes 15 seconds or more and at the bottom is:

Showing 1 to 10 of 58,812 entries

Almost 30 times as many AND there's 53 attributes.

Somewhere between lies the line I suspect. I'm getting weather and ALL of the attributes are being updated every time. "wind" seems like it deserves to be often, but "twilight_begin" ? Do I need that updated more than once a day? In other words, there's probably a lot of tuning possible with ApiXU and maybe some other drivers.

I have zero Xiaomi / Aqara devices so I can't perform my crude test.

1 Like

I have some Xiaomi devices and they don't produce 50,000 events for sure, it was a request to cleanup a bit the events because when I'm looking for old events they are quite hard to understand, considering that there are a lots of battery entries (followed by checkin entry), so they will be a bit more "tidy", but it wasn't related to performances in this case.

Both devices referenced in this thread are sending events with "isStateChange: true" I would check to see if that is truly necessary. If that flag is not set then the system will determine if the attribute has changed and will store it in the event table only if that is the case.

1 Like