SOLVED: [Feature Request] HTTP SUBSCRIBE and NOTIFY support

Would it be possible to get support for sending (preferably async) http SUBSCRIBE requests? And on the other side, in the "mappings", could support be added for "action: [ NOTIFY 'callbackHandler' ]" for apps?

I'm working on an app that will support a much more complete feature set of what Sonos speakers have than the built-in driver provides, but in order to implement parts of the UPnP architecture I need to send SUBSCRIBE requests to the speakers, and they then perform a NOTIFY request at the specified endpoint that's provided in the SUBSCRIBE request. So the mappings would need to accept NOTIFY requests and allow handling those in a callback, much like how GET and POST work presently.

Beyond Sonos, any other UPnP supporting network speakers would benefit from this, as the code and libraries I'm working on should work with minimal changes for other brands as well if they implement UPnP.

Thanks in advance!

EDIT: As pointed out, HubAction may be able to send SUBSCRIBE messages, in which case adding "asyncHttpSubscribe() isn't really needed, just some clarification in the documentation on how to use HubAction to send non-standard HTTP verbs would be fine, I can write a wrapper method for it if I had better documentation on how to use it. But the "mappings" definitely needs a code change to allow NOTIFY verbs as it currently responds with 501 errors.

Nothing in the current app architecture stops you from making http calls and receiving data, etc. on an endpoint so not understanding what you are requesting. Can you elaborate?

This is 2 requests. First is the addition of a method, "asyncHttpSubscribe()" which will use the "SUBSCRIBE" verb, not GET, POST, PUT, DELETE, etc.

The second is expanding the "mappings" config for apps to allow "NOTIFY" verbs to be received. I've tried this, and used Postman to send NOTIFY messages to my app, it did not process the callback:

mappings {
  path('/notify') {
  action: [ NOTIFY: 'testNotifyCallback' ]
  }
}

Don't see SUBSCRIBE HTTP Method in the HTTP specs.

From RFC 9110: HTTP Semantics the defined methods are:

Method NameDescriptionSection
GETTransfer a current representation of the target resource.9.3.1
HEADSame as GET, but do not transfer the response content.9.3.2
POSTPerform resource-specific processing on the request content.9.3.3
PUTReplace all current representations of the target resource with the request content.9.3.4
DELETERemove all current representations of the target resource.9.3.5
CONNECTEstablish a tunnel to the server identified by the target resource.9.3.6
OPTIONSDescribe the communication options for the target resource.9.3.7
TRACEPerform a message loop-back test along the path to the target resource.9.3.8

and your action mapping will need to be one of POST, GET and possibly PUT

Take a look at the UPnP architecture document I linked above. It uses SUBSCRIBE and NOTIFY verbs.

Here's a Wireshark capture of some SUBSCRIBE and NOTIFY messages. The SUBSCRIBE here is sent from the official Sonos app for Mac, and the NOTIFY is being sent from the coordinator speaker in my Sonos group.

Per the document you linked: "This specification defines a number of standardized methods that are commonly used in HTTP, as outlined by the following table." The list in that document is not somehow inclusive of all possible HTTP verbs. It merely lists some of the common ones.

If you want to use a uncommon method suggest you look at HubAction Object | Hubitat Documentation

That may work for sending a SUBSCRIBE message, but it doesn't help with receiving NOTIFY ones in an app.

Here's a Flask app I put together in a few seconds to receive the NOTIFY messages from a subscription:
image

When I use Postman to send a SUBSCRIBE, with this Flask app as the "CALLBACK", I get the XML messages from my UPnP speaker, as I'd expect to:

When I send a SUBSCRIBE with this, which causes the speaker to send NOTIFY messages to the Hubitat app, I get nothing from my Hubitat app:

So, even if I can use the HubAction to send a SUBSCRIBE (which may well work, I have't tried it because it's documented in any way to describe how to use it to send non-standard HTTP verb messages), there's still the issue of the app Mappings not passing NOTIFY messages into the app.

If I send a NOTIFY message manually to the app, using Postman, I get a 501 Not Implemented error:

So as it turns out, there's a "magic port" at 39501 that's documented NOWHERE on the Hubitat documentation site, but I found it by digging around with a bunch of google searches. As for the SUBSCRIBE, it can be accomplished like this (which is also not documented anywhere I could find):

new hubitat.device.HubAction(
"""SUBSCRIBE /upnp/event/basicevent1 HTTP/1.1
HOST: 192.168.1.35:1400
CALLBACK: <http://192.168.1.4:39501/notify>
NT: upnp:event
TIMEOUT: Second-300
""", hubitat.device.Protocol.LAN)

By using the magic 39501 port, my SUBSCRIBE results in my driver receiving messages to its parse() method.

So at this point, I change this from "FEATURE REQUEST" to "Please update documentation site". Maybe consider making the documentation area a WIKI if there's not enough official personnel to maintain it. Scouring google until one stumbles upon a 5 year old forum post isn't really a valid alternative to proper documentation.

4 Likes

It is a wiki, technically. But credentials to login and edit are, AFAICT, not granted to many people.

Nonetheless, @bertabcd1234 has done a great job of updating the hub documentation over the past year or so, IMO.

2 Likes

Yeah, 39501 is used for receiving unsolicited HTTP communications from devices. The idea is that devices are created in HE with a Device Network Id (DNI) that matches the physical devices MAC Address, then when HE receives the data via HTTP it looks for a device on the hub that matches the MAC address and calls the parse() method, providing the HTTP request object. This is used in the EcoWitt weather station / sensor drivers I look after, where the EcoWitt gateway periodically send a data feed via HTTP to the HE hub based on a configured HTTP address on the EcoWitt Gateway, i.e. the data is not sent as a response to a HTTP call initiated by the HE hub. This data feed is sent on 39501.

1 Like

It happens to be a wiki underneath, just because that's a format that worked well internally. It is only editable by staff. The old docs site was also a wiki and users could be granted access to submit changes to a queue (which some seem to mis-remember as being directly editable, but that was also not the case). Further, what was submitted was filled with more often spam than anything. I don't believe there are any plans to change how this works -- for various reasons.

Regarding the actual issue, there has been much added to the developer docs over the last year or so, and there are more to come. This is somewhat of an advanced case that rarely comes up, but it's certainly not outside the scope of what should be there; other work has simply been prioritized. Respecting the humanity of the relatively small staff, the effort it takes to identify gaps in the first place, the time and effort it takes to address them, and the fact that what one user/developer is looking for isn't the same as what everyone else is looking for are the best things to keep in mind if you ask me, :slight_smile:

4 Likes

Oh for sure. That's why I suggested a community wiki instead of a Hubitat maintained area. If it were a wiki, I'd be editing a few pages myself right now, so others don't need to spend 6+ hours figuring out something that would take 30 seconds to read on a docs page.

Or if a wiki doesn't work, what about posting the source for the docs area on GitHub and letting people put in PRs for changes/updates. This is how Home Assistant handles their documentation, for example.

No one is stopping anyone from starting that if they want to. :slight_smile: But the goal is for these to get good enough where no one will feel the need to suggest this anymore.

I don't see a big difference between GitHub source/PRs vs. using a wiki as a wiki and wouldn't expect any changes there, but the same should (eventually) apply...