Mira uses Maker API to sync ST devices with HE. Occasionally users of Mira would say the two would get out of sync but I could never nail down what might be at least one of the causes of it. A user reached out over the last couple days who has a pretty complex setup with a bunch of devices that all end up triggering updates at once and I think I've nailed down a possible issue with that config. To take ST/Edge out of the equation I built a test case without it.
Test environment: C7 running 2.4.3.173. Run ncat on another machine, Windows/Linux/Mac as the receiver. I rebooted the C7 just to make sure it was fresh.
To duplicate:
-
Create some virtual devices for testing. I created Virtual Switch Test, Virtual Switch Test A, Virtual Switch Test B, Virtual Switch Test C, Virtual Switch Test D and Virtual Switch Test E. I believe the tipping point at more than 4 updates in a small window of time.
-
Create two simple rules. I used basic rules to create one for on and another for the off version.
-
For easy debugging/testing, I created an EZ dashboard with the 6 switches on it.
-
Add a new Maker API instance. Add the 6 virtual test switches to it. Turn logging for debugging on. Add your test server machine's ip address and listening port in the Post device events field. I used 8080 in this example.
-
Startup ncat or whatever you use to listen on a port on your server machine.
$ ncat -l -k -m 10 -v -v 8080 -
On the dashboard, if you click the controlling device, you should see the other 5 change state as expected due to the Basic Rules. All good.
-
But looking at my ncat window, I'm seeing that only 3 or 4 of the devices are getting updates. 192.168.1.113 is my ncat machine. 192.168.1.155 is the HE C-7.
Ncat: Version 7.98 ( https://nmap.org/ncat )
Ncat: Listening on [::]:8080
Ncat: Listening on 0.0.0.0:8080
Ncat: Connection from 192.168.1.155:34920.
POST /deviceUpdate HTTP/1.1
Accept: */*
Content-Length: 185
Content-Type: application/json
Host: 192.168.1.113:8080
Connection: Keep-Alive
User-Agent: Apache-HttpClient/4.5.13 (Java/11.0.28)
Accept-Encoding: gzip,deflate
{"content":{"name":"switch","value":"on","displayName":"Virtual Test Switch","deviceId":"276","descriptionText":"Virtual Test Switch was turned on","unit":null,"type":null,"data":null}}Ncat: Connection from 192.168.1.155:34924.
POST /deviceUpdate HTTP/1.1
Accept: */*
Content-Length: 189
Content-Type: application/json
Host: 192.168.1.113:8080
Connection: Keep-Alive
User-Agent: Apache-HttpClient/4.5.13 (Java/11.0.28)
Accept-Encoding: gzip,deflate
{"content":{"name":"switch","value":"on","displayName":"Virtual Test Switch D","deviceId":"280","descriptionText":"Virtual Test Switch D was turned on","unit":null,"type":null,"data":null}}NCAT DEBUG: Closing connection.
Ncat: Connection from 192.168.1.155:34928.
POST /deviceUpdate HTTP/1.1
Accept: */*
Content-Length: 189
Content-Type: application/json
Host: 192.168.1.113:8080
Connection: Keep-Alive
User-Agent: Apache-HttpClient/4.5.13 (Java/11.0.28)
Accept-Encoding: gzip,deflate
{"content":{"name":"switch","value":"on","displayName":"Virtual Test Switch E","deviceId":"281","descriptionText":"Virtual Test Switch E was turned on","unit":null,"type":null,"data":null}}NCAT DEBUG: Closing connection.
Ncat: Connection from 192.168.1.155:34930.
POST /deviceUpdate HTTP/1.1
Accept: */*
Content-Length: 189
Content-Type: application/json
Host: 192.168.1.113:8080
Connection: Keep-Alive
User-Agent: Apache-HttpClient/4.5.13 (Java/11.0.28)
Accept-Encoding: gzip,deflate
{"content":{"name":"switch","value":"on","displayName":"Virtual Test Switch A","deviceId":"277","descriptionText":"Virtual Test Switch A was turned on","unit":null,"type":null,"data":null}}NCAT DEBUG: Closing connection.
NCAT DEBUG: Closing connection.
- If you check the logs tab for your new Maker API instance, it does appear that Maker API noted the new state and built events for each to be sent out.
- The ncat data received on the server side seems to align with this, it logged it in order of Virtual Switch, Switch D, Switch E, Switch A. But B and C were never received in ncat.
You can repeat this behavior pretty easily. Any action that generates a bunch of POST http calls at the same "moment', some seem to be discarded.
This seems to show that when a bunch of events are all generated to the same Maker API destination at about the same time via http POST, some are discarded.
- Is there documented rate limiting?
- Is this a matter of socket limits on listening on the ncat side? I bumped it to 10 (I believe 5 is the default) in case that was a limiter as well, but no change.
- Is this the expected behavior? I realize that there is no documented queuing or fault tolerance of any kind but at the very least some logging may help indicate that yeah theres a overload and some updates via http POST may have been discarded.
As an aside, are there any additional logs available for the actual network communication by Maker API? I looked around and nothing stood out.
