We can agree to disagree there. 
Sure - 
But Wireshark and a port mirror is simpler than building a Zniffer (IMHO)..
I read your post to indicate that HE added a built in broker. But if I interpret the docs right that’s not the case.
I’ve had very good results with Matter Devices connected to HE.. Same with Thread as long as I don’t connect them directly to HE. I use virtual HE devices and Home automations to get the Thread devices to HE. That has worked well.
According to the documents you still need a broker. That’s how I read it anyway
Sorry, I read it again and see where the option is for internal broker. My bad.
Again, I don’t see why MQTT payload cannot be somewhat standartisized.
I believe this should be very obvious. The base reason is the same reason we do not have a worldwide electrical standard, standardized measuring system, common language, etc.
Stuff evolves in isolation, which results in everyone having a significant investment in their own particular implementation, which is optimized for their use case. Standardization generally asks everyone to move away from an existing design that is optimized for their use case, to a new design which isn't as optimized for their use case and comes with a bunch of stuff they didn't ask for and don't need. Along the way, they have to break backward compatibility for their customers and manage a huge transition. It's a huge effort just to get parity with what you had previously. In the short term, it's a very low return on investment.
Yes, I've been involved in a few messaging standardization efforts, including MQTT.
I’m a huge fan of MQTT and part of the joy is its lack of standardisation beyond the transport mechanism, optional certificates, and the JSON format. Just as developers can make an API any way they choose, the same can be achieved with MQTT, often with the same JSON formatting and no-one pushing a stricter framework. Within that lies a world of flexibility.
The more rules one imposes, and the stricter one tries to impose them, the more people will find quirky workarounds when the inevitable edge case or lack of functionality crops up. Zigbee’s pretty good… but manufacturer-specific clusters (to cover those edge cases) really do make it very tricky to decipher sometimes.
Can you send me the topic names and the sample messages published on them? I can reproduce the setup here and modify the integration to pick/parse it correctly. That will help others in the same boat, too
Here is a a link to the entire project:
It contains HE Drivers, Arduino Sketch and KiCAD Schematic and PCB files.
(Anyone is very welcome to reuse this project without any restrictions.)
@gopher.ny MqttManager.cpp in the AC_Monitor\src\system has a code how MQTT Payload is generated and MultiIO_MQTT_Parent_HA_HE.txt in the MQTT_HE_Drivers location is a parent HE Driver for processing MQTT messages.
NOTE: The SW Code is created with MS Copilot AI help (I am not so good to design a sw).
Please let me know if you need anything else and I will be very happy to address all questions.
This DIY Device is a configurable (at a compile time via config.h file) which currently supports number of Relays (Switches), Contact Sensors, Temp Sensors (DS18B20 OneWire) and very unique multilevel liquid sensor:
The Device has a local Web UI interface:
Each Relay has an individual run time configurable Auto OFF and Max ON Timeouts. Realays could be locally controlled (configurable at a compile time) by Contact Sensors or Liquid Level Sensor. And there is a Real Time Scheduler for controlling a selected Relay locally.
Plan is to add a PWM Modules (for say LED Brightness control and ADC Modules for Sensors with Analog Output.
Home Assistant picked everything automatically but HE required a custom drivers. HE was seeing everything correctly but it was impossible to created a mapping.
WebUI, HA and HE are 100% synchronized. I.e. any of them can control a Device and all are seeing a correct status instantly.