How to parse XML in a multipart/form-data POST? Hikvision Cams

[kleung1]

My custom driver ([RELEASE] Hikvision Alarm (HTTP Listening/Data Streaming/Alarm Server)) waits for POSTs from Hikvision IP cameras on port 39501 and parses the XML with this code snippet:

255    def msg = parseLanMessage(description)
256    def body = new XmlSlurper().parseText(new String(msg.body))

However, it turns out some newer cameras don't send the XML in this way and I get an error:

org.xml.sax.SAXParseException: Content is not allowed in prolog. on line 256 (method parse)

Comparing the old and new cameras. I noticed the older camera sends POSTs with content-type application/xml; charset="UTF-8" and the XML is in the body of the message.

On newer cameras, it sends POSTs with content-type multipart/form-data; boundary=boundary and the XML looks like a file.

How can I handle this scenario? Thanks for any tips in the right direction.

[sburke781]

@thebearmay - Do you have any thoughts on this, wondering if your work with files may come in handy somehow?

[kleung1]

The resulting msg string from parseLanMessage():

[mac:4CF5DCB55351, ip:c0a801ec, port:f6d2, headers:[POST / HTTP/1.1:null, Connection:close, Host:192.168.1.218:39501, Content-Length:738, Content-Type:multipart/form-data; boundary=boundary], body:--boundary
Content-Disposition: form-data; name="MoveDetection.xml"; filename="MoveDetection.xml"
Content-Type: application/xml
Content-Length: 568



192.168.1.236
39501
HTTP
4c:f5:dc:b5:53:51
1
2023-04-07T01:46:08-07:00
1
VMD
active
Motion alarm
DS-2CD2347G2-LU(G15477384)


--boundary--
, header:POST / HTTP/1.1
Connection: close
Host: 192.168.1.218:39501
Content-Length: 738
Content-Type: multipart/form-data; boundary=boundary
]

Everything I need is right there. I could probably just parse the string manually...

Here's an older camera that POSTs XML directly:

[mac:64F2FBAC247B, ip:c0a801e1, port:d460, headers:[POST / HTTP/1.1:null, Connection:keep-alive, Host:192.168.1.218, Content-Length:886, Content-Type:application/xml; charset="UTF-8"], body:

192.168.1.225
39501
HTTP
64:f2:fb:ac:24:7b
1
2023-04-07T01:49:23-08:00
93362
PIR
active
PIR alarm
Bonus Room (E94085953)

DS-2CD2455FWD-IW20201104AAWRE94085953
PIR&&DS-2CD2455FWD-IW20201104AAWRE94085953,2023-04-07T01:49:23-08:00,1,1.0


, header:POST / HTTP/1.1
Connection: keep-alive
Host: 192.168.1.218
Content-Length: 886
Content-Type: application/xml; charset="UTF-8"
, xml:192.168.1.22539501HTTP64:f2:fb:ac:24:7b12023-04-07T01:49:23-08:0093362PIRactivePIR alarmBonus Room (E94085953)DS-2CD2455FWD-IW20201104AAWRE94085953PIR&&DS-2CD2455FWD-IW20201104AAWRE94085953,2023-04-07T01:49:23-08:00,1,1.0, data:192.168.1.22539501HTTP64:f2:fb:ac:24:7b12023-04-07T01:49:23-08:0093362PIRactivePIR alarmBonus Room (E94085953)DS-2CD2455FWD-IW20201104AAWRE94085953PIR&&DS-2CD2455FWD-IW20201104AAWRE94085953,2023-04-07T01:49:23-08:00,1,1.0]

[thebearmay]

If it is coming in as form data it’s probably in a pair:value format , possibly JSON. If you temporarily remove the parser and just print the raw data what does it look like?

[kleung1]

I've added logDebug to output description and the parseLanMessage() result:

def parse(String description) {
    logDebug "Parsing '${description}'"

	def msg = parseLanMessage(description)
    logDebug "msg: '${msg}'"

This is directly copy-pasted from the Hubitat Log display in the web-based UI:

msg: '[mac:4CF5DCB55351, ip:c0a801ec, port:d0d0, headers:[POST / HTTP/1.1:null, Connection:close, Host:192.168.1.218:39501, Content-Length:738, Content-Type:multipart/form-data; boundary=boundary], body:--boundary Content-Disposition: form-data; name="MoveDetection.xml"; filename="MoveDetection.xml" Content-Type: application/xml Content-Length: 568 192.168.1.236 39501 HTTP 4c:f5:dc:b5:53:51 1 2023-04-08T16:01:07-07:00 1 VMD active Motion alarm DS-2CD2347G2-LU(G15477384) --boundary-- , header:POST / HTTP/1.1 Connection: close Host: 192.168.1.218:39501 Content-Length: 738 Content-Type: multipart/form-data; boundary=boundary ]'

Parsing 'mac:4CF5DCB55351, ip:c0a801ec, port:d0d0, headers:UE9TVCAvIEhUVFAvMS4xDQpDb25uZWN0aW9uOiBjbG9zZQ0KSG9zdDogMTkyLjE2OC4xLjIxODozOTUwMQ0KQ29udGVudC1MZW5ndGg6IDczOA0KQ29udGVudC1UeXBlOiBtdWx0aXBhcnQvZm9ybS1kYXRhOyBib3VuZGFyeT1ib3VuZGFyeQ0K, body:LS1ib3VuZGFyeQ0KQ29udGVudC1EaXNwb3NpdGlvbjogZm9ybS1kYXRhOyBuYW1lPSJNb3ZlRGV0ZWN0aW9uLnhtbCI7IGZpbGVuYW1lPSJNb3ZlRGV0ZWN0aW9uLnhtbCINCkNvbnRlbnQtVHlwZTogYXBwbGljYXRpb24veG1sDQpDb250ZW50LUxlbmd0aDogNTY4DQoNCjw/eG1sIHZlcnNpb249IjEuMCIgZW5jb2Rpbmc9IlVURi04Ij8+DQo8RXZlbnROb3RpZmljYXRpb25BbGVydCB2ZXJzaW9uPSIyLjAiIHhtbG5zPSJodHRwOi8vd3d3Lmhpa3Zpc2lvbi5jb20vdmVyMjAvWE1MU2NoZW1hIj4NCjxpcEFkZHJlc3M+MTkyLjE2OC4xLjIzNjwvaXBBZGRyZXNzPg0KPHBvcnRObz4zOTUwMTwvcG9ydE5vPg0KPHByb3RvY29sPkhUVFA8L3Byb3RvY29sPg0KPG1hY0FkZHJlc3M+NGM6ZjU6ZGM6YjU6NTM6NTE8L21hY0FkZHJlc3M+DQo8Y2hhbm5lbElEPjE8L2NoYW5uZWxJRD4NCjxkYXRlVGltZT4yMDIzLTA0LTA4VDE2OjAxOjA3LTA3OjAwPC9kYXRlVGltZT4NCjxhY3RpdmVQb3N0Q291bnQ+MTwvYWN0aXZlUG9zdENvdW50Pg0KPGV2ZW50VHlwZT5WTUQ8L2V2ZW50VHlwZT4NCjxldmVudFN0YXRlPmFjdGl2ZTwvZXZlbnRTdGF0ZT4NCjxldmVudERlc2NyaXB0aW9uPk1vdGlvbiBhbGFybTwvZXZlbnREZXNjcmlwdGlvbj4NCjxjaGFubmVsTmFtZT5EUy0yQ0QyMzQ3RzItTFUoRzE1NDc3Mzg0KTwvY2hhbm5lbE5hbWU+DQo8L0V2ZW50Tm90aWZpY2F0aW9uQWxlcnQ+DQoNCi0tYm91bmRhcnktLQ0K'

[kleung1]

I directed the cameras to POST to webhook.site. Here's what the older camera sends:

And the newer camera:

[kampto]

Anyone figure this out? Im trying to make a custom Hikvision listening DH and hung up on how to receive and parse out the events from the various cameras. I know there's a built in Hikvision driver but I have some different needs and that code(the parts I need) is not publicly available for me to "borrow".

[kleung1]

No, but let me know if you ever do.

[TomS]

@kleung1 , @kampto
Hey guys... hot off the presses... I've got a solution for you with regards to the multi-part alert messages sent by newer Hikvision cameras. It's easy and it works. Came up with it over the weekend and been testing with it all morning.

Parse Method for Hikvision Alarm Server, supports ALL cameras! Maybe...

[kleung1]

@kleung1

ef msg = parseLanMe

Kudos!! I just took a quick glance at your code... I will need to find time to test this out in my driver!

[TomS]

Just wanted to share this utility I came across from Dahua called Network Assistant that lets you open up an http listener on your windows pc to receive alarms from your cameras and inspect the contents. There are other choices for http debugging and app development but this one is super easy to use if all you want is the basics. I found it being used in this Hikvision doc that comes off the EU portal, which is the place to go for Hikvision doc, "How to get real-time alarm/event in HTTP listening mode"

[kampto]

I see this when I copy, paste, and save this driver

[sburke781]

Can you show line 127? Also, the platform version you are running may be important.

[thebearmay]

add this

import groovy.transform.Field

[sburke781]

Ah, yes... I see I have had to import this as well when I have used static fields...

Not quite an existing driver...?

[thebearmay]

Probably included in the full code that the OP pulled this from and just got dropped.

[sburke781]

Yeah, can see that @TomS 's driver does not include the import... And can now see that row 127 includes a static field reference. Can only assume use of the field must produce the error, which may not have happened in all situations.

[TomS]

Sorry about that! I did forget to include the import in that sample code. I'm going to refresh it here shortly with what I have now for my driver.

[kampto]

Tried again, C7, ver 2.3.8.106

[TomS]

Sorry but this is only the Parse method from my driver, that I will be releasing soon with HPM.