Not only is the flood exactly hourly, it's got some correlation to the hub's start time.
I looked at the hub event logs and saw:
systemStart System startup with build: 2.3.8.138 2.3.8.138 4/08/2024 4:55:37.746 pm
and thought I could predict that the flood would occur on minute 55 of any hour. I was able to capture the flood in that minute with another 2400 packets in a second: 07:55:17.212149 to 07:55:18.296927
A temp workaround might be to disable mDNS on the ESP32. If you remove the name on the WiFi setup screen, I wonder if it would cause the network stack to ignore all the broadcast traffic from the Hubitat?
Wondering if this is also tied to the issues some people are having with Homekit going "no response" randomly. Since it uses mDNS HomeKit “No Response”, any resolution?
At wake-up this morning, my connection to HomeKit was dead from my C8. No Response mode.
I did the normal restart of the C8 and it came back.
In order to debug, I created a rule where every minute I would set a virtual switch called switch A, and when HomeKit saw switch A turn on it would turn on another switch called switch B.
In the rule I would check if switch B did not come on within 10 seconds I would declare HomeKit dead.
Interestingly enough HomeKit died exactly 1 hour after I rebooted my C8.
I am still investigating.
It's ONE second every 3600 seconds. It's 2400 packets (of 100mb) on a gig network. I saw 16-20 mDNS packets scattered within the middle of the flood: apple products (Homepod, TV, computer), cameras, dd-wrt, and a couple other Hubitat Hubs. I can't see how this could lead to that. Anything's possible, but it's mostly just 1 second of latency.
There seem to be some “quirks” in Hubitat’s mDNS implementation. Most of the time there’s not much operational impact for most people. Another example:
The "quirks"..... that's the thing that's weird to me... Under the Hubitat environment is just a Linux system, and most Linux systems use a core suite of tools and libraries to handle all the networking services like DNS, DHCP, etc.
So I am always trying to figure out WHY does Hubitat have networking issues? WHY can't Hubitat exist on LANs with jumbo frames? WHY doesn't the DHCP service honor DNS servers in my DHCP scope (you have to change them manually)? WHY are there mDNS abnormalities? I just don't understand, these basic networking issues should not exist on any up-to-date Linux based system.
Just a quick check in on this. I scanned last night and the packet 'storms' are now around 800 per second every hour. Its definitely growing over time and my LED controllers have started rebooting or becoming unresponsive because of Hubitat doing this.
Is there a way to disable mDNS on the Hubitat? I only have one hub and I have its IP reserved in my DHCP server.
I'm willing to share my network logs with the devs if that helps solve this.
This issue is also affecting others and discovered in March:
@gopher.ny is probably the one to examine the network issue and was tagged in the other thread. You might want to document your network layout - model of router, network configuration (VLANs), and how your hub is connected - ethernet or WiFi. My guess is that there is some interaction with users' network setup as these issues are fairly uncommon. For example, the only "quirk" I have is that hubitat.local gets flipped to the IP address. It's the only .local device I have that does that. I'm using a fairly generic flat network on UniFi equipment.
It's unfortunate. Maybe it's not well recognized but the storm is of informational/broadcast packets. They don't do anything but update an existing table of values. In the storm's case, it's repeating the updates and so they can all be discarded, and would be under normal circumstances. I think the ESP32 is just underpowered enough that it is unable to process one packet in the time it takes to receive a packet. That should mean that any packet burst would overwhelm that processor. By that I mean, the network is capable of overwhelming them. Hubitat hubs are only 100mb network devices and most of us have a 1gig network.
I am not trying to downplay the mDNS storm that Hubitat is producing, but instead point out the precarious position all of us are in with any processor that can't dispatch a packet in the time it takes to receive a packet. The buffer size is just too limited but you might want to research increasing the network buffer size on the ESP32 for that specific use: WLED. It might be that it isn't a memory hog on it's own and could live side by side with a bigger buffer.