Thanks @dkilgore90 for your reponse. Testing the API using Postman shows me that it requires Digest authentication, the call fails using Basic. Is there a similar option to the above but using Digest? I have a suspicion the answer will be be no but here is to hoping!
No experience with "Digest Authentication" -- but the concepts are outlined here, hope this helps! Digest access authentication - Wikipedia
This might help:
There would be some work required to port to Hubitat and your specific use case. But the general structure might be a good reference.
This looks really promising. I'll have a play. Thanks.
I'll post the ported code if and when I manage to get it sorted.
@dkilgore90 FYI I had found https://stackoverflow.com/questions/45361721/groovy-digest-authentication as a pointer to digest auth but haven't managed to get much further as yet.
Nope, still struggling I'm afraid. Here is what I think that I know (feel free to shoot me down where I have it wrong):
- The HubAction class (method?) example from @tomw and the subsequent parsing of the response is only applicable to device drivers - I am trying to code an app example (at least initially)
- Digest Auth requires a two part request- the first request requires you to interrogate the response headers to obtain the nonce. It also returns a 401 which indicates that authentication is required. You can catch the 401 and interrogate the reponse header to get the value that you need.
- The second request then uses that nonce value and provides the credentials to the server which authenticates and returns the complete headers and data
I am stuck at interrogating the returned request from the first call. I simply dont know enough about groovy coding and am not able to find a similar example online to serve as an example. I have used the httpGet method and encapsulated this in a try catch construct (sorry if these are not the right terms). I do get a 401 response but it is handled in the exception (catch) part of the code and I don't know how to use that to interrogate the headers and obtain the nonce value.
2021-08-01 13:44:09.993 debug Exception: groovyx.net.http.HttpResponseException: status code: 401, reason phrase: Unauthorized
What I am trying to do is to connect to an API that requires digest auth and obtain a string of a server that can then be used in further API calls. Details are at Myenergi API. The end game is to have parsed the asn value from the returned header, which would contain the server string to be used in all other calls (also digest auth) but I am having no luck at all.
Reaching out to the developer community in the hope that this sparks somebody's imagination enough to give it a go and start me in the right direction. If I can get a working example of a digest auth call in an app code I would more than likely be away. I am more than happy to learn but I have really hit a brick wall here in my knowledge of HE dev and groovy in general.
All contributions gratefully received.
I trap HTTP 401 in a few of my drivers and re-auth (for an expired JWT in my case, but the same general structure should probably work). Here's an example -- see httpExecWithAuthCheck() in hubitat_unifiEvents/unifiController at main · tomwpublic/hubitat_unifiEvents · GitHub
For synchronous HTTP operations (like httpGet()), Hubitat returns an HttpResponseDecorator. You can get the headers from that response with <response>.getHeaders().
It's only slightly more complicated with async HTTP operations, as described here: Async HTTP calls
Looks like the HTTP headers are what you need to extract, based on your original post and the linked post (https://s7.myenergi.net/cgi-status-*), seeing that 401 is an "expected" response with key info.
https://director.myenergi.net/cgi-jstatus-*
Simon
1 Like
I left my post a little light... looks like you need to identify the "asn" in the headers and use that in subsequent HTTP calls I expect...
As I thought about it some more, I think you will have to use the async HTTP calls, because the 401 response will throw an exception immediately from the synchronous HTTP call, and you won't be able to get at the response and headers.
Here's a stab at how to handle this based on reading the Myenergi thread that you linked to:
def callApi(someJsonBody)
{
def params =
[
uri: "https://director.myenergi.net",
body: someJsonBody
]
try
{
asynchttpGet(handler, params, [body: someJsonBody])
}
catch (Exception e)
{
log.debug "failed: ${e.message}"
}
}
def handler(response, data)
{
if(response?.getStatus() == 401)
{
log.debug "response headers = ${response?.getHeaders()}"
def params =
[
uri: "https://" + response?.getHeaders()?.getAt('X_MYENERGI-asn'),
body: data?.body
]
httpGet(params)
{ resp ->
if (resp)
{
log.debug "resp.data = ${resp.data}"
doSomethingWith(resp)
}
}
}
}
@tomw and @sburke781 thanks heaps for your help. I had to step away from the code for a bit due to time limitations but I had a good stab at it yesterday and have got the digest authentication working with async calls (good tip @tomw). If you are interested in how I have done it (and this may well be the only publicly available example of digest auth working in Hubitat at the moment - not saying it is a good one though!) then the code is at https://github.com/VeloWulf/myenergiIntegration. It is definitely still a work in progress but the basic structure around the digest auth is in place.
The next problem that I need to overcome is that by doing all of the calls as asynchronous it means that I struggle to interact with the UI. The specific example that I have is the loading of devices into the app config pages but I am certain that there wil be others down the road. I am going to try reworking the code to use synchrronous calls and rtap the 401 error (and hopefully the associated headers) in the exception, store the info in the state variable and then reuse that same info is the subsequent calls to both the director server (to obtain the ASN) and the main data server that has the hub data I need. Going to use the
technique described here by @tomw. Fingers crossed...
Do you have any tips for how to deal with the async calls and interact with the UI that I mightnot have thought about? The crux of it all is (obviously) that the async code continues to run after functions that call the code have completed. They would need to be told to wait for the async code to finish somehow but I haven't worked out a solution for this yet.
Thanks again for your help
2 Likes
Nice! To your async or delayed return question, I haven't had great success with the app config pages since you don't have as much control over when they load. I usually just brute force it and give the user an extra page to click through or an input that triggers a refresh of the same page to get updated data values into the input fields.
For an example of the latter:
dynamicPage(name: "mainPage2", title: "", install: true, uninstall: true)
{
section
{
input name: "beaconsToWatch", type: "enum", title: "Select all devices that indicate presence at your location.", options: state?.knownBeacons?.sort(), required: false, multiple: true
input name: "refreshList", type: "bool", title: "Refresh this list", defaultValue: false, submitOnChange: true
input name: "selectAll", type: "bool", title: "Select all devices?", defaultValue: false, submitOnChange: true
}
if(refreshList)
{
app.updateSetting("refreshList", false)
}
if(selectAll)
{
app.updateSetting("beaconsToWatch", state.knownBeacons ?: [])
app.updateSetting("selectAll", false)
}
}
2 Likes
Yeah, unfortunately I can't claim any experience or knowledge in app's myself, sorry.
I now have it working in a synchronous call and I have the second server (the referred asn) returning data. Woohoo!! The latest code is at the github link. It isn't too pretty, I'm afraid. In particular the pollASNServer function that returns the body data sends back something that looks like this, which is nested maps and lists all over the place. The code that I have written tried to parse it and it works (I think) but was done through a lot of trial and error and reading of online knowledge sources. If anybody has a better way to get to the data do let me know!! I am trying to identify which devices are active in the system (have nested values - just the eddi in the example below).
[
[eddi:[
[sno:12345678,
dat:15-09-2021,
tim:05:48:32,
ectp2:-1549,
ectp3:3516,
ectt1:Internal Load,
ectt2:Grid,
ectt3:Generation,
bsm:0,
bst:0,
cmt:254,
dst:1,
div:0,
frq:50.02,
fwv:3200S3.048,
gen:3516,
grd:-1525,
pha:1,
pri:1,
sta:1,
tz:9,
vol:2326,
che:7.56,
hpri:1,
hno:1,
ht1:Tank 2,
ht2:None,
r1a:0,
r2a:0,
r1b:0,
r2b:0,
rbc:1,
tp1:62,
tp2:127]
]
],
[zappi:[
]
],
[harvi:[
]
],
[asn:s18.myenergi.net,
fwv:3401S3.051]
]
Have you tried something like:
HashMap responseMap = (HashMap)evaluate(response)
which could theoretically allow you to reference the values semi-directly like:
myFwv = responseMap.eddi.fwv
I've picked this up as a starting point for getting digest auth working in a new Venstar thermostat integration I'm doing right now, so thanks for that!
Just something I noticed right away, the opaque is not required in a server response, so you need to guard for not getting it and trying to send it back (null exception will be thrown).
I've embellished on your code a little as well to support other hashes. It's possible for the server to return multiple www-authenticate headers, each with a different supported algorithm, and the client is expected to choose. MD5 has fallen from grace.
Poking along, I'm new to groovy and HE (but not Java and others) so half of my time is spent figuring out how to do basic things the "right" way. Do you happen to know if there's any persistent/static storage for a driver outside of the state structure and the device attributes? The www-authenticate information can be stored so subsequent requests can use them to build auth headers up front, but I'm not sure where to keep it (state gets displayed in the UI, so not great). I think this would help smooth out async calls a lot.
Could use the encrypt() function when storing and the unencrypt() when you need to use it.
1 Like
Sounds good. Please feel free to create a pull request on the repo with any suggested changes - all contributions are very welcome!
That is interesting - at the moment I am coding for the specific API servers which (touch wood) have so far responded with the opaque info but I will bear this in mind. I have noticed that even within the same API ecosystem the www-auth response varies in composition.
I attempt to do something similar with the latest version of the code (will commit the latest shortly but is still WIP so bear that in mind). The particular service that I have to use always returns a 401 when talking to the first [director] server so the response needs to be constructed afresh each time. The latest code moves onto the next server (that I call the ASN server) where 200 codes are returned and there is merit in constructing an auth header from stored values. Have a look and let me know what you think.
I can appreciate your point regarding the state variable being visible but from my perspective this is an acceptable security risk. You could always use @thebearmay idea of encryption or explore using th new concept of hub variables brought in with 2.2.8
P.S. if you are creating a driver then you might want to take a look at some of the earlier examples posted by @tomw. I do plan on creating a driver as well once I have the app working but it is slow going for me! 
2 Likes
Actually it caught me a bit off guard, too. The "server" in my case is a Venstar thermostat, so I'm not surprised that an embedded webserver is taking as many shortcuts as possible. RFC7617 says that component of the header is a "SHOULD", and darned if I didn't run across one that avails itself of the option that creates. But it's a first out of many.
In general, I agree, but as you know, code like this has a habit of migrating its way into other things (ahem), so if the problem can be solved and the example that persists is strong in this regard, I think it's doing the community a service. If it can be done easily, it should be done, IMO. But also not a hill to die on.
By the way... I note in the example you linked, it returns an empty opaque for its Authorization header. So now two examples!
Random question along similar lines (maybe)... I'm grappling with conversions between charsets that I think I need to do, anyone converted a potentially UTF-16 byte array to UTF-8? I can convert the byte array to a string, is that enough, is there (hopefully) some kind of implicit conversion? It does display the same output in the logs as is displayed in Notepadd++ when I display the same decoded output, with Notepadd++ indicating it is working in UTF-8.
Basically, I can't seem to include the Charset.forName("UTF-8") in a new String constructor call.
EDIT: Solved my issue, using getBytes("UTF-8") on either a byte array or String seems to do the trick.
Thanks for @kahn-hubitat for sharing his work here, which I'd almost forgotten about...
This topic was automatically closed 365 days after the last reply. New replies are no longer allowed.
