C-4 / C-5 / C-7 Hubs - free memory decline over time

I posted the stats and @gopher.ny looked into it already but didn't see anything interesting. He's recommending that I cycle power my hub twice a week for now since it's a C4 HUB.

I might jump to a C7 soon so waiting to see how things are for you guys with the C7 first :slight_smile:

1 Like

I see your "total JVM" increased. Up to now I assumed total JVM was hardware dependent. Kinda like total hard disk capacity.

Actually as I am reading this thread I have it stuck in my mind that the Hubitat folks are working on this and some major modifications of how the 700 series chip is handled.

1 Like

@JasonJoel - which endpoint do you poll to get the JVM memory stats?

http://192.168.x.x/hub/advanced/freeOSMemoryHistory

Then I throw out all of the data except the last row in the table. I poll every 5 minutes.

Still hoping @gopher.ny in the future adds an endpoint to only get the latest data, instead of fetching all history each time.

2 Likes

Thanks! What we need is a stats and health endpoint like: https://micrometer.io/

I was also suffering from a severe memory decline (halving over a period of a week after a reboot) but @lewis.heidrick's post here RM and a Reboot got me thinking about runcounts.

Turns out it was a maker API issue where I was sending MQTT data to node-red and then to a virtual device. A lot of the MQTT data was redundant but it kept refreshing the virtual device. Simply limiting the refresh to an appropriate rate and doing the same for all my MQTT links seems to have made a huge reduction in the loss of free memory.

So it really does highlight the need to optimise the IO to the hub wherever possible.

3 Likes

Yup which is why I point out those tools. They're often overlooked but very useful when you have no idea where to start.

3 Likes

That sounds odd to me, though... I do many hundreds, often well over a thousand, writes from Node-RED to the hub via Maker API every day. I haven't seen any memory leak from that.

How many times a day were you writing to that virtual device? Maybe I just haven't pounded it hard enough yet. lol

In the end, if that is what fixed your issue - then good! Just weird that Maker API would cause issues for you and not others.

Like I said above, I had the same problem with Maker slowing my hub to a crawl. So it isn't just one person. Admittedly Maker in my case wasn't being used with Node Red, but I am not sure that makes any difference?

I suspect that others are also having issues too. I have seen many instances of people saying they use Maker and they are having general slowness issues or having to constantly reboot and so on. But they aren't connecting the slowdown with Maker for some reason.

Anyhow, I strongly suspect there is some bug, memory leak, or whatever in Maker. Maybe only when used a certain way, or only enough of a bug when combined with something specific (a built in OR community app/driver?) that it becomes apparent.

1 Like

Interesting! Certainly anything is possible!

I use Maker API on 3 hubs, and haven't seen it. BUT I also use it more or less the same way on all 3 hubs. So if it only triggers in some uses cases, or with some certain devices, I might not see it. (3 of basically the same is really only "1" from a testing scenario).

Even with the slow decline in memory (aging sucks haha) my and my client's systems have been very responsive with Maker and NR - to be fair there are no other apps etc so maybe that offsets a bit. I have always suspected it's something related to "groups/scenes" but that's purely anecdotal and likely wrong.

Just on that one device alone - it was doing probably 20 or so a minute (multiple attirbutes involved) so 30k a day !!!! I've probably managed to cut it down to a thousand.

1 Like

What is "hub stats" - pls explain little it more. See no such a staff in Settings page.
Thank You.

Hub Stats

View and enable stats
http://hubitat.local/hub/stats
http://hubitat.local/hub/enableStats
http://hubitat.local/hub/disableStats

Taken from this post:

2 Likes

Make sure to turn it off when you are done. It can cause issues if you leave it running.

2 Likes

Curious now your graph looks on 2.2.5? Mine looked a lot like yours before: new graphs look different, but I don’t think it will fix reboot requirement on my C4.

Since upgrade; weird it didn’t start out at 700 K though, more like 460 K

I will have to check! Thanks for the update..

New free memory is similar to your plot. Need a few more days to see if I need to reboot or not. Hoping for the best.

i have a slow leak as well. .. do you have to remove or just disable maker api to test?

You can just disable it with the X thing. It may level out though. A lot of that is just caching and whatnot. Especially after a reboot and about a week after that.