Can httpget read raw html?

I tried calling httpGet with a content Type of text/html expecting to get a raw html response but it looks like it is auto parsing it somehow and just returning the inner text. It's there any way to read raw html?

So is this not doable? I am trying to call mykevo.com/login and grab the text of a meta tag needed to create an integration, however, when I try to read the HTML using log.debug response.data.head.meta.find {it.@name == "csrf-token"}.content I'm not able to read it. I'm suspecting it's because an XMLSlurper isn't exactly ideal at reading HTML (which isn't always valid XML). Can anyone lend a hand?

I don’t own a Kevo but did a quick google search and found someone created an integration for SmartThings. You should look at that and see if you can port it over:

If I'm remembering right, I was able to get a raw response in the past by passing

contentType: "text/plain"

to prevent the autoparser from parsing the html.

@ritchierich thanks, I actually started with that but it's a pretty basic integration. For example, rather than having an app to manage the integration and child devices, you just create devices. This means you do automatically get a list of your locks, you have to look up guids on their website and manually add. It also means if you have multiple locks, rather than polling the API once per interval, you poll it lock_count times per interval because each lock has to request its status individually. That's not ideal.

@jp0550 thanks for the suggestion, unfortunately I tried that. When you do that you're telling the server "send me back a text/plain response." In the case of the kevo server it says "I don't know how to do that" and returns a 500 error. I need something that tells Hubitat what to do without impacting what I'm asking the server to do.

What happens if you try the following in your headers...???

requestContentType: "text/html",
contentType: "text/plain",

Unfortunately that results in - groovyx.net.http.HttpResponseException: Internal Server Error on line 63 (prefKevoDevices)

try without requestContentType and only supply contentType: "text/plain"

If that doesn't work, I recommend asking @chuck.schwer for his advice.

Same error. When you specify contentType it tells the HTTP request to specify that in the Accept header. The server wants to send html so when you tell it "I only support text" it is basically saying "I don't know what to do" @chuck.schwer happy to hear any suggestions :slight_smile:

The docs for httpGet say the response is a HttpResponseDecorator object.

Did you try response.entity.content.text?
(ref: StackOverflow - How to get the raw response and URL from HttpResponseDecorator)

Yup, when you try to access response.entity.content you get an error stating that the stream is already closed.

Use aynchttpGet, the parsing in response there is different than the synchronous httpGet and it should give you back the raw text.

1 Like

@chuck.schwer thank you. So this may be a stupid question, but how do I use the result of an asynchronous call to populate a preferences page in an app? Meaning if it's completing asynchronously... how do I make it populate the UI? Can I return a dynamicPage from an async call?

Unfortunately no, I have created an issue to fix the sync call so you should be able to do this once 2.1.1 is out.

Thank you! I look forward to it.

@chuck.schwer did this make it into 2.1.1? I don't see it in the notes. How do I use this functionality?

yes, it is an additional parameter you can pass to httpGet:

https://docs.hubitat.com/index.php?title=Common_Methods_Object#httpGet

httpGet([uri: "https://mykevo.com/login", textParser: true], { response ->
    log.debug groovy.xml.XmlUtil.escapeXml(response.data.text)
})```
4 Likes

Well it works great... But kevo added a recaptcha to their website and broke my whole idea :frowning:

I love you! I have been searching for a way to get the raw xml back from my httpgets in certain cases and this works great! My Hikvision camera driver will soon get two new features, and more.

I do wish the doc was better though. I think there are a lot of "obscure methods" available behind the scenes that we don't get to see, or are simply not well documented or difficult to understand if they are, because none have a good example or two just like yours. My diligence has paid off. I have finally found it. Thank you and Happy New Year!
Tom

3 Likes

Actually, that method caused me a moment of panic this morning until I looked it up in the Apache doc and found out that is the escape method for making things pretty by changing <> to &lt &gt.

So that didn't work but what I did get out of this is to use xml = response.data.text instead of just response.data when using textParser true and requestContentType = application/xml for httpget.

So all is groovy now!

2 Likes