Maker API - SLOW /devices/all and Invalid JSON on /devices endpoints

🚧 Developers
1

Hi,

I'm using the Maker API /devices and /devices/all endpoints and struggling to handle 2 different issues with them

  1. /devices/all is SLOW.. takes 4-5 seconds to return for my ~25 devices but takes minutes to return on users with lots (ie: 100) devices..
  2. I can use /devices to get all of the device ID's quickly but then I need to use /deices/ID to get info for each device. But the JSON response to this endpoint isn't JSON and it's causing my JSON parser to crash. The "capabilities" field mixes String's and Objects in an array which isn't valid AFAIK.

I'm curious how anyone is able to parse this output actually.. at least everything I've tried using the GSON parser fails miserably..

http://192.168.0.202/apps/api/38/devices/[Device ID]

    "capabilities" : 
    [
        "Switch",
        
        {
            "attributes" : 
            [
                
                {
                    "name" : "switch",
                    "dataType" : null
                }
            ]
        },
        "Polling",
        "Refresh",
        "Valve",
        
        {
            "attributes" : 
            [
                
                {
                    "name" : "valve",
                    "dataType" : null
                }
            ]
        },
        "Sensor",
        "Actuator"
    ],

For the /devices/all endpoint, why is it so slow? Shouldn't the hub just return all of the current state info it has cached in memory? Would it be possible to put in a parameter to return 'pages' of devices at a time (vs everything)?

2

Also, I find it very odd/confusing why the /devices/all endpoint returns fields that the /devices/ID endpoint doesn't (manufacturer, date, type and a few others).

There's also changes in the attributes/commands/capabilities fields. I would expect the /ID endpoint to simply return the exact same data which is contained in the /devices/all - but just for 1 device instead of a list of them

3

devices/all is slow, always has been slow, and probably always will be slow. It is best used sparingly for initial device inventory/cache building, and never again (until the whole cache needs rebuilt again). It is not designed or intended for frequent use, nor for regularly refreshing all parameters in a cache - that should be done with the POST events.

In terms of /devices/id json format, I don't have a good answer. I can parse the response just fine in node-red, and if I paste it into a few different online json viewers it looks/renders fine (such as http://jsonviewer.stack.hu/ ). I am not a json expert though, and am not saying it is "compliant".

In terms of differences in the responses, only Hubitat can answer that. Not sure if that is on purpose, or a bug/unintentional thing.

1 Like
4

Here's an example of the device info mismatch between API's..

device details from /devices/all endpoint:

  {
    "name": "Generic Zigbee Contact Sensor",
    "label": "Patio Door",
    "type": "Generic Zigbee Contact Sensor",
    "id": "67",
    "date": "2020-05-23T13:33:34+0000",
    "model": null,
    "manufacturer": null,
    "capabilities": [
      "TemperatureMeasurement",
      "Battery",
      "ContactSensor",
      "Configuration",
      "Refresh",
      "Sensor"
    ],
    "attributes": {
      "battery": "24",
      "dataType": "NUMBER",
      "values": null,
      "contact": "open",
      "temperature": "69.80"
    },
    "commands": [
      {
        "command": "configure"
      },
      {
        "command": "refresh"
      }
    ]
  },

Here's the response from the /devices/67 endpoint

{
  "id": "67",
  "name": "Generic Zigbee Contact Sensor",
  "label": "Patio Door",
  "attributes": [
    {
      "name": "battery",
      "currentValue": 24,
      "dataType": "NUMBER"
    },
    {
      "name": "contact",
      "currentValue": "open",
      "dataType": "ENUM",
      "values": [
        "closed",
        "open"
      ]
    },
    {
      "name": "temperature",
      "currentValue": 69.80,
      "dataType": "NUMBER"
    }
  ],
  "capabilities": [
    "TemperatureMeasurement",
    {
      "attributes": [
        {
          "name": "temperature",
          "dataType": null
        }
      ]
    },
    "Battery",
    {
      "attributes": [
        {
          "name": "battery",
          "dataType": null
        }
      ]
    },
    "ContactSensor",
    {
      "attributes": [
        {
          "name": "contact",
          "dataType": null
        }
      ]
    },
    "Configuration",
    "Refresh",
    "Sensor"
  ],
  "commands": [
    "configure",
    "refresh"
  ]
}

I think both endpoints contain the same data more or less... at least what I care about. But, the formats differ between the 2 which makes it more of a pain.

5

The problem at this point is that many people/integrations use the format as-is, so they can't really change it without breaking many things. Or maybe forking it and making a new endpoint, or new "MakerAPI 2" app I guess.

1 Like
6

All of my Hubitat-dependent automations would stop working if the format was changed. Not something I want to see happening -- unless there's a phenomenally good reason to do so.

7

Yeah, I didn't really expect it to change at this point.. just got frustrated at trying to switch from the /devices/all endpoint to the /devices/ID endpoint and how inconsistent the data was.

I'm not sure if the /devices/ID JSON is valid or not.. I do know my GSON parser failed to handle it since it needs a consistent format. I was able to figure it out by creating a custom deserializer.. but frustration still..

I would still like to know why the /devices/all endpoint is so slow and if it could be improved though.. adding some optional parameters to fetch X devices at a time would be great or some kind of paging system.

$.02

8

I've also found the All option for Maker API very slow, especially if it includes my Sensibo devices (which are cloud). I noticed when I excluded these it ran a fair bit quicker. I wonder if it forces these to update out to the cloud or something before returning all the values. Anyway I've found it much much much quicker just to interrogate the individual devices one at a time than use the All function.

9

Or do devices/all once at the startup of the external integration, and then keep the cache up-to-date via the POST events. That is how a number of 3rd party hubitat integrations work.

Even then, if you are only using a subset of all devices, it is probably still faster to fetch them individually like @Angus_M said.

1 Like
10

Yeah, I'm building a bespoke dashboard using eventsocket and Maker API. I've been shocked at how fast an individual get is with Maker API. It is blisteringly quick. I use Javascript to cycle through my devices (by type, using arrays) and it's really quick and reliable. This way I can set my lights, AC, presence, mode etc when the page loads and then keep them up to date using the eventsocket stream. It works very nicely! But I also do wonder why the All option is sometimes so slow.

11

Add me to the list of those curious whether the "all" endpoint is fetching from a local memory cache or polling & waiting for individual devices.

That's kind of what I expected too. If you replace "all" with "*" it will return data in the same format. But since it returns more data it could take even longer to get.

Not sure what the best practice is when starting up. I'm currently using the "all" endpoint & then listening to the WebSocket for events. Based on what I'm reading here, I may rework that & send individual requests for each device like @Angus_M is doing.

12

@jpage4500 - just wanted to make sure you’re aware that you can write your own Groovy OAuth2 endpoint App, with local and cloud endpoints. This would allow you complete control over how the hub responds to your remote client application.

Just another option to consider. :sunglasses:

13

old topic but wanted to update with something I just learned and might be helpful to someone else who ends up here:

I'd have a big learning curve ahead as I've been a mobile app developer for 20+ years.. but, no experience with Groovy or server development.. The MakerAPI is a huge win for me -- also I think the only way to do cloud endpoints is with a 3rd party server so I'm happy Hubitat offers this as well.

That said -- I have been toying with writing my own app to supplement the MakerAPI. The biggest thing missing today with MakerAPI is the ability to push updates to a mobile device. There's a callback URL but that's a no-go with a mobile device(s) - at least without some kind of 3rd party server/service.

However if I can have an app that sends out push notifications (FCM) - this would fix that limitation for me. I know Hubitat uses FCM for their own app so it can be done. But, I'd need some help getting the app side of this figured out.. maybe I'll open a new topic and see if anyone is able or willing to help

14

If you write your own OAUTH App on Hubitat, it will use Hubitat's cloud server. No need for a third-party server.