Problematic connection with HTTP Get

[MrFarmer]

Hi,
I am writing a driver for the HomeWizard water meter. It is a very simple API, just a GET that returns a simple json. The problem I have is that I get a lot of Connection reset by peer errors on Hubitat. When I do the same in python, it keeps working, no error in hundredths of requests.

This is the code snippet for Hubitat giving Connection reset by peers half the time when called every two, five or ten seconds (did try normal httpGet as well, same result):

void pollDevice() {
  logger "info", "pollDevice()"
  try {
    Map params = [
      uri: "http://${ip}", 
      path: "/api/${state.api_version}/data", 
      timeout: 5, 
      headers: ["Host": ${ip}, "User-Agent": "Hubitat", "Accept-Encoding": "gzip, deflate", "Accept": "*/*", "Connection": "keep-alive"]
    ]
    asynchttpGet(parse, params)
  } catch(Exception e) {
    logger ("error", "error occured calling httpget ${e}")
  }
}

This is the python code that never get the error unless the Hubitat is polling as well:

if __name__ == "__main__":
    for attempt in range(250):
        resp = requests.get("http://192.168.178.29/api/v1/data")
        print(str(attempt ) + ": "+ resp.text)
        time.sleep(2)

It almost seems that the Hubitat requests cause some odd issues with this device what should not be possible.
Update: If I poll once per 30 seconds, I do not get this problem. Does Hubitat have a problem with too many http requests in a short time?

Running on a pretty empty C7 with latest firmware.

Anyone any suggestions how to tackle this issue so this will work on Hubitat?

Cheers Rene

[tomw]

Maybe you are over specifying the headers and timeout on the Hubitat side. Is there a reason you're setting so many explicitly?

I'd try bouncing both requests off of an echo server (like postman echo) to see how the requests differ.

[MrFarmer]

Thanks for your reply Tom. I actually have replicated these headers from what I see in the python request. With or without it does not seem to matter. The only thing I see is if I bring the poll frequency under 15 seconds it becomes problematic on my Hubitat and it does not on a PI3 with python, hitting it over and over with a browser, of from the Home Assistant driver for this device. (but I try to keep as much as I can on Hubitat to reduce the number of things I have to keep running)

[tomw]

@MrFarmer

I actually have replicated these headers from what I see in the python request

What do you mean by this?



Here's what I see from this simple test:

import requests
r = requests.get("https://postman-echo.com/get")
print(r.text)

response:

{
  "args": {},
  "headers": {
    "x-forwarded-proto": "https",
    "x-forwarded-port": "443",
    "host": "postman-echo.com",
    "x-amzn-trace-id": "Root=1-65bfc34a-3acd09137e5713c01aeac955",
    "user-agent": "python-requests/2.28.2",
    "accept-encoding": "gzip, deflate",
    "accept": "*/*"
  },
  "url": "https://postman-echo.com/get"
}

Same thing in Hubitat:

httpGet("https://postman-echo.com/get")
{
    resp->
    log.debug resp.data
}

Response:

[args:[:], headers:[x-forwarded-proto:https, x-forwarded-port:443, host:postman-echo.com, x-amzn-trace-id:Root=1-65bfc2f0-3434d4e04f6497fb0fa2178b, accept:*/*, user-agent:Apache-HttpClient/4.5.13 (Java/1.8.0_181), accept-encoding:gzip,deflate], url:https://postman-echo.com/get]

I would remove the headers from your request, just to see if it works with default values set for the Hubitat request. Especially Connection=keep-alive, which seems like it could be holding multiple server connections open for the requests you're sending.

[MrFarmer]

Hi Tom,
In python you can enable low-level debugging for requests and that will show the headers used for a get request and response. As an other test I created a http server that gives an identical response as the Water Meter device, If I set the polling frequency to two seconds in Hubitat it works without missing a beat. So it seems only the combo of the two (Water Meter & Hubitat) has this issue. Only thing I did not mimic is having the web-server over WiFi. So, on to the wifes laptop :-). Also the Connection Reset by peer status last for about 14 seconds, then it works again for a number of requests, until it stops again for 14 secs.

Sending without any headers does not make a difference. It is how I started initially.
Cheers.

[tomw]

@MrFarmer

As an other test I created a http server that gives an identical response as the Water Meter device

This is an excellent experiment. And I'd say it points to an issues with the server for the water meter.

This issue looks sort of similar, and it seems like the developer for the meter is actively looking into it: Homewizard devices are randomly disappearing · Issue #87432 · home-assistant/core · GitHub

[MrFarmer]

Thanks, that looks too similar to be a coincidence. I'll ask them about the water meter. I did move it to another wifi hub as a test, but that did not make a difference.