How do I extract values from this parse() XML data?

A LAN device is pushing XML data to HE. My device driver's parse() function is being driven as expected. Here's the code that I'm using so far to just look at the data that comes in...

Here's the data that is written to the log by the "log.debug(xmlData)" statement. Sorry for the way it strings it out on one line, but I couldn't figure out how to post it so that it could be copied by someone...

------------------------------3cbec9ce8f05 Content-Disposition: form-data; name="owServerData"; filename="details.xml" Content-Type: text/plain <?xml version="1.0" encoding="UTF-8"?> <Devices-Detail-Response xmlns="http://www.embeddeddatasystems.com/schema/owserver" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> <PollCount>10922</PollCount> <DevicesConnected>1</DevicesConnected> <LoopTime>0.193</LoopTime> <DevicesConnectedChannel1>1</DevicesConnectedChannel1> <DevicesConnectedChannel2>0</DevicesConnectedChannel2> <DevicesConnectedChannel3>0</DevicesConnectedChannel3> <DataErrorsChannel1>0</DataErrorsChannel1> <DataErrorsChannel2>0</DataErrorsChannel2> <DataErrorsChannel3>0</DataErrorsChannel3> <VoltageChannel1>4.79</VoltageChannel1> <VoltageChannel2>4.76</VoltageChannel2> <VoltageChannel3>4.77</VoltageChannel3> <VoltagePower>5.00</VoltagePower> <DeviceName>OWServer_v2-Enet</DeviceName> <HostName>EDSOWSERVER2</HostName> <MACAddress>44:B7:D0:81:AB:09</MACAddress> <DateTime>2024-02-07 16:47:39</DateTime> <owd_EDS0068 Description="Temperature, Humidity, Barometric Pressure and Light Sensor"> <Name>EDS0068</Name> <Family>7E</Family> <ROMId>D60010000036DD7E</ROMId> <Health>7</Health> <Channel>1</Channel> <RawData>68007101F102B3008F018A01B8BA1FDDEF001C0000020000E1070000E1070000000026EE64007DD87DD87DD800803E000000002003000000A0860100000009020000000000000000</RawData> <PrimaryValue>47.0625 %RH</PrimaryValue> <Temperature Units="Centigrade">23.0625</Temperature> <Humidity Units="PercentRelativeHumidity">47.0625</Humidity> <DewPoint Units="Centigrade">11.1875</DewPoint> <Humidex>24.9375</Humidex> <HeatIndex Units="Centigrade">24.6250</HeatIndex> <BarometricPressureMb Units="Millibars">1015.340</BarometricPressureMb> <BarometricPressureHg Units="InchesOfMercury">29.983</BarometricPressureHg> <Light Units="Lux">28</Light> <LED>1</LED> <Relay>0</Relay> <Counter1>2017</Counter1> <Counter2>2017</Counter2> <TemperatureHighAlarmState>0</TemperatureHighAlarmState> <TemperatureLowAlarmState>0</TemperatureLowAlarmState> <HumidityHighAlarmState>0</HumidityHighAlarmState> <HumidityLowAlarmState>0</HumidityLowAlarmState> <DewPointHighAlarmState>0</DewPointHighAlarmState> <DewPointLowAlarmState>0</DewPointLowAlarmState> <HumidexHighAlarmState>0</HumidexHighAlarmState> <HumidexLowAlarmState>0</HumidexLowAlarmState> <HeatIndexHighAlarmState>0</HeatIndexHighAlarmState> <HeatIndexLowAlarmState>0</HeatIndexLowAlarmState> <BarometricPressureMbHighAlarmState>0</BarometricPressureMbHighAlarmState> <BarometricPressureMbLowAlarmState>0</BarometricPressureMbLowAlarmState> <BarometricPressureHgHighAlarmState>0</BarometricPressureHgHighAlarmState> <BarometricPressureHgLowAlarmState>0</BarometricPressureHgLowAlarmState> <LightHighAlarmState>0</LightHighAlarmState> <LightLowAlarmState>0</LightLowAlarmState> <ClearAlarms Writable="True">-</ClearAlarms> <TemperatureHighConditionalSearchState Writable="True">0</TemperatureHighConditionalSearchState> <TemperatureLowConditionalSearchState Writable="True">0</TemperatureLowConditionalSearchState> <HumidityHighConditionalSearchState Writable="True">0</HumidityHighConditionalSearchState> <HumidityLowConditionalSearchState Writable="True">0</HumidityLowConditionalSearchState> <DewPointHighConditionalSearchState Writable="True">0</DewPointHighConditionalSearchState> <DewPointLowConditionalSearchState Writable="True">0</DewPointLowConditionalSearchState> <HumidexHighConditionalSearchState Writable="True">0</HumidexHighConditionalSearchState> <HumidexLowConditionalSearchState Writable="True">0</HumidexLowConditionalSearchState> <HeatIndexHighConditionalSearchState Writable="True">0</HeatIndexHighConditionalSearchState> <HeatIndexLowConditionalSearchState Writable="True">0</HeatIndexLowConditionalSearchState> <BarometricPressureMbHighConditionalSearchState Writable="True">0</BarometricPressureMbHighConditionalSearchState> <BarometricPressureMbLowConditionalSearchState Writable="True">0</BarometricPressureMbLowConditionalSearchState> <BarometricPressureHgHighConditionalSearchState Writable="True">0</BarometricPressureHgHighConditionalSearchState> <BarometricPressureHgLowConditionalSearchState Writable="True">0</BarometricPressureHgLowConditionalSearchState> <LightHighConditionalSearchState Writable="True">0</LightHighConditionalSearchState> <LightLowConditionalSearchState Writable="True">0</LightLowConditionalSearchState> <TemperatureHighAlarmValue Writable="True" Units="Centigrade">38</TemperatureHighAlarmValue> <TemperatureLowAlarmValue Writable="True" Units="Centigrade">-18</TemperatureLowAlarmValue> <HumidityHighAlarmValue Writable="True" Units="PercentRelativeHumidity">100</HumidityHighAlarmValue> <HumidityLowAlarmValue Writable="True" Units="PercentRelativeHumidity">0</HumidityLowAlarmValue> <DewPointHighAlarmValue Writable="True" Units="Centigrade">125</DewPointHighAlarmValue> <DewPointLowAlarmValue Writable="True" Units="Centigrade">-40</DewPointLowAlarmValue> <HumidexHighAlarmValue Writable="True">125</HumidexHighAlarmValue> <HumidexLowAlarmValue Writable="True">-40</HumidexLowAlarmValue> <HeatIndexHighAlarmValue Writable="True" Units="Centigrade">125</HeatIndexHighAlarmValue> <HeatIndexLowAlarmValue Writable="True" Units="Centigrade">-40</HeatIndexLowAlarmValue> <BarometricPressureMbHighAlarmValue Writable="True" Units="Millibars">2000.000</BarometricPressureMbHighAlarmValue> <BarometricPressureMbLowAlarmValue Writable="True" Units="Millibars">0.000</BarometricPressureMbLowAlarmValue> <BarometricPressureHgHighAlarmValue Writable="True" Units="InchesOfMercury">100.000</BarometricPressureHgHighAlarmValue> <BarometricPressureHgLowAlarmValue Writable="True" Units="InchesOfMercury">0.000</BarometricPressureHgLowAlarmValue> <LightHighAlarmValue Writable="True" Units="Lux">100000</LightHighAlarmValue> <LightLowAlarmValue Writable="True" Units="Lux">0</LightLowAlarmValue> <LEDFunction Writable="True">2</LEDFunction> <RelayFunction Writable="True">1</RelayFunction> <LEDState Writable="True">1</LEDState> <RelayState Writable="True">0</RelayState> <Version>1.04</Version> </owd_EDS0068> </Devices-Detail-Response> ------------------------------3cbec9ce8f05

As an example, I understand that I should be able to extract the value for the humidity in the "owd_EDS0068" sensor element data by doing something like...

Float basementHumidity = Devices-Detail-Response.owd_EDS0068.Humidity

My problem is I can't figure out how to get from the XML-ish data shown to XML data that can be processed. Part of the problem, I think, is that I need to strip off all of the data before and after the "Device-Detail-Response" element, but I don't have any idea how to do that.

I've written a lot of code, but none using XML, Groovy, or Java.

If someone can explain to me how to add the XML in a vertically scrollable form (instead of it appearing as a single scrollable horizontal line of text), I will edit the post.

@tomw let me know if any of this helps.

Can you post a sample of the raw message received?

Also curious, what type of LAN device is it? Do they have any description of their data method?

UPDATE:
Not sure if this is based on the same type of data... but this appears similar to what you might have been receiving:

UPDATE2:
Some success... The below example gets the value for the Humidex. You could replace that with whatever specific values you want. Still working on a map method that is not turning it into garbage...

log.debug( "Humidex = ${ Data.split( 'Humidex' )[ 1 ].split( '>' )[ 1 ].split( '<' )[ 0 ] }" )

Humidex = 24.9375

@snell The data in the stackoverflow thread is for the same device I'm using, although the data received by the parse function contains additional header data at the beginning, e.g., MAC address, IP address, etc. Maybe HE is prepending the additional data to the data it received before calling the driver's parse function?

The device is the OW-Server 1-wire Ethernet server. There's already a community driver for it, but it polls the device instead waiting for the device to push the data. That's what I'm trying to change as a learning exercise.

I'm hoping there's a better way to extract values than using a bunch of regex's. There's over 50 values for the sensor connected to the server, and the server supports up to 22 1-wire sensors of multiple types.

You are talking about this thread correct? From what I can see the data is just in screenshots, not in any text I can toss in and "throw code at". Or did I miss it (or a link to it) in there?

Yes, I am sure having them all there would be a pain (I had to do that with something a long time ago). I am trying to bang away at the sample above... but it really does not want to allow it to act as a string so I can "cut" the garbage at the beginning and end.

UPDATE:
Seemed like it would be easy enough to "minus" the extra (Data is just a copy of the line from above):

    String Data = '------------------------------3cbec9ce8f05 Content-Disposition: form-data; name="owServerData"; filename="details.xml" Content-Type: text/plain <?xml version="1.0" encoding="UTF-8"?> <Devices-Detail-Response xmlns="http://www.embeddeddatasystems.com/schema/owserver" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> <PollCount>10922</PollCount> <DevicesConnected>1</DevicesConnected> <LoopTime>0.193</LoopTime> <DevicesConnectedChannel1>1</DevicesConnectedChannel1> <DevicesConnectedChannel2>0</DevicesConnectedChannel2> <DevicesConnectedChannel3>0</DevicesConnectedChannel3> <DataErrorsChannel1>0</DataErrorsChannel1> <DataErrorsChannel2>0</DataErrorsChannel2> <DataErrorsChannel3>0</DataErrorsChannel3> <VoltageChannel1>4.79</VoltageChannel1> <VoltageChannel2>4.76</VoltageChannel2> <VoltageChannel3>4.77</VoltageChannel3> <VoltagePower>5.00</VoltagePower> <DeviceName>OWServer_v2-Enet</DeviceName> <HostName>EDSOWSERVER2</HostName> <MACAddress>44:B7:D0:81:AB:09</MACAddress> <DateTime>2024-02-07 16:47:39</DateTime> <owd_EDS0068 Description="Temperature, Humidity, Barometric Pressure and Light Sensor"> <Name>EDS0068</Name> <Family>7E</Family> <ROMId>D60010000036DD7E</ROMId> <Health>7</Health> <Channel>1</Channel> <RawData>68007101F102B3008F018A01B8BA1FDDEF001C0000020000E1070000E1070000000026EE64007DD87DD87DD800803E000000002003000000A0860100000009020000000000000000</RawData> <PrimaryValue>47.0625 %RH</PrimaryValue> <Temperature Units="Centigrade">23.0625</Temperature> <Humidity Units="PercentRelativeHumidity">47.0625</Humidity> <DewPoint Units="Centigrade">11.1875</DewPoint> <Humidex>24.9375</Humidex> <HeatIndex Units="Centigrade">24.6250</HeatIndex> <BarometricPressureMb Units="Millibars">1015.340</BarometricPressureMb> <BarometricPressureHg Units="InchesOfMercury">29.983</BarometricPressureHg> <Light Units="Lux">28</Light> <LED>1</LED> <Relay>0</Relay> <Counter1>2017</Counter1> <Counter2>2017</Counter2> <TemperatureHighAlarmState>0</TemperatureHighAlarmState> <TemperatureLowAlarmState>0</TemperatureLowAlarmState> <HumidityHighAlarmState>0</HumidityHighAlarmState> <HumidityLowAlarmState>0</HumidityLowAlarmState> <DewPointHighAlarmState>0</DewPointHighAlarmState> <DewPointLowAlarmState>0</DewPointLowAlarmState> <HumidexHighAlarmState>0</HumidexHighAlarmState> <HumidexLowAlarmState>0</HumidexLowAlarmState> <HeatIndexHighAlarmState>0</HeatIndexHighAlarmState> <HeatIndexLowAlarmState>0</HeatIndexLowAlarmState> <BarometricPressureMbHighAlarmState>0</BarometricPressureMbHighAlarmState> <BarometricPressureMbLowAlarmState>0</BarometricPressureMbLowAlarmState> <BarometricPressureHgHighAlarmState>0</BarometricPressureHgHighAlarmState> <BarometricPressureHgLowAlarmState>0</BarometricPressureHgLowAlarmState> <LightHighAlarmState>0</LightHighAlarmState> <LightLowAlarmState>0</LightLowAlarmState> <ClearAlarms Writable="True">-</ClearAlarms> <TemperatureHighConditionalSearchState Writable="True">0</TemperatureHighConditionalSearchState> <TemperatureLowConditionalSearchState Writable="True">0</TemperatureLowConditionalSearchState> <HumidityHighConditionalSearchState Writable="True">0</HumidityHighConditionalSearchState> <HumidityLowConditionalSearchState Writable="True">0</HumidityLowConditionalSearchState> <DewPointHighConditionalSearchState Writable="True">0</DewPointHighConditionalSearchState> <DewPointLowConditionalSearchState Writable="True">0</DewPointLowConditionalSearchState> <HumidexHighConditionalSearchState Writable="True">0</HumidexHighConditionalSearchState> <HumidexLowConditionalSearchState Writable="True">0</HumidexLowConditionalSearchState> <HeatIndexHighConditionalSearchState Writable="True">0</HeatIndexHighConditionalSearchState> <HeatIndexLowConditionalSearchState Writable="True">0</HeatIndexLowConditionalSearchState> <BarometricPressureMbHighConditionalSearchState Writable="True">0</BarometricPressureMbHighConditionalSearchState> <BarometricPressureMbLowConditionalSearchState Writable="True">0</BarometricPressureMbLowConditionalSearchState> <BarometricPressureHgHighConditionalSearchState Writable="True">0</BarometricPressureHgHighConditionalSearchState> <BarometricPressureHgLowConditionalSearchState Writable="True">0</BarometricPressureHgLowConditionalSearchState> <LightHighConditionalSearchState Writable="True">0</LightHighConditionalSearchState> <LightLowConditionalSearchState Writable="True">0</LightLowConditionalSearchState> <TemperatureHighAlarmValue Writable="True" Units="Centigrade">38</TemperatureHighAlarmValue> <TemperatureLowAlarmValue Writable="True" Units="Centigrade">-18</TemperatureLowAlarmValue> <HumidityHighAlarmValue Writable="True" Units="PercentRelativeHumidity">100</HumidityHighAlarmValue> <HumidityLowAlarmValue Writable="True" Units="PercentRelativeHumidity">0</HumidityLowAlarmValue> <DewPointHighAlarmValue Writable="True" Units="Centigrade">125</DewPointHighAlarmValue> <DewPointLowAlarmValue Writable="True" Units="Centigrade">-40</DewPointLowAlarmValue> <HumidexHighAlarmValue Writable="True">125</HumidexHighAlarmValue> <HumidexLowAlarmValue Writable="True">-40</HumidexLowAlarmValue> <HeatIndexHighAlarmValue Writable="True" Units="Centigrade">125</HeatIndexHighAlarmValue> <HeatIndexLowAlarmValue Writable="True" Units="Centigrade">-40</HeatIndexLowAlarmValue> <BarometricPressureMbHighAlarmValue Writable="True" Units="Millibars">2000.000</BarometricPressureMbHighAlarmValue> <BarometricPressureMbLowAlarmValue Writable="True" Units="Millibars">0.000</BarometricPressureMbLowAlarmValue> <BarometricPressureHgHighAlarmValue Writable="True" Units="InchesOfMercury">100.000</BarometricPressureHgHighAlarmValue> <BarometricPressureHgLowAlarmValue Writable="True" Units="InchesOfMercury">0.000</BarometricPressureHgLowAlarmValue> <LightHighAlarmValue Writable="True" Units="Lux">100000</LightHighAlarmValue> <LightLowAlarmValue Writable="True" Units="Lux">0</LightLowAlarmValue> <LEDFunction Writable="True">2</LEDFunction> <RelayFunction Writable="True">1</RelayFunction> <LEDState Writable="True">1</LEDState> <RelayState Writable="True">0</RelayState> <Version>1.04</Version> </owd_EDS0068> </Devices-Detail-Response> ------------------------------3cbec9ce8f05'
    log.debug( Data )
    String TempData1 = Data.minus( '------------------------------3cbec9ce8f05 Content-Disposition: form-data; name="owServerData"; filename="details.xml" Content-Type: text/plain ' )
    String TempData2 = TempData1.minus( ' ------------------------------3cbec9ce8f05' )
    log.debug( TempData2 )

But it looks like it REALLY does not want to be treated as a string, as the debug logs show below. It loses all the node names for each one even when it is just immediately dumping the string to the log BEFORE any manipulations:

[dev:1146]2024-02-07 10:22:05.294 PM[debug]10922 1 0.193 1 0 0 0 0 0 4.79 4.76 4.77 5.00 OWServer_v2-Enet EDSOWSERVER2 44:B7:D0:81:AB:09 2024-02-07 16:47:39 EDS0068 7E D60010000036DD7E 7 1 68007101F102B3008F018A01B8BA1FDDEF001C0000020000E1070000E1070000000026EE64007DD87DD87DD800803E000000002003000000A0860100000009020000000000000000 47.0625 %RH 23.0625 47.0625 11.1875 24.9375 24.6250 1015.340 29.983 28 1 0 2017 2017 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 - 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 38 -18 100 0 125 -40 125 -40 125 -40 2000.000 0.000 100.000 0.000 100000 0 2 1 1 0 1.04
[dev:1146]2024-02-07 10:22:05.112 PM[debug]------------------------------3cbec9ce8f05 Content-Disposition: form-data; name="owServerData"; filename="details.xml" Content-Type: text/plain 10922 1 0.193 1 0 0 0 0 0 4.79 4.76 4.77 5.00 OWServer_v2-Enet EDSOWSERVER2 44:B7:D0:81:AB:09 2024-02-07 16:47:39 EDS0068 7E D60010000036DD7E 7 1 68007101F102B3008F018A01B8BA1FDDEF001C0000020000E1070000E1070000000026EE64007DD87DD87DD800803E000000002003000000A0860100000009020000000000000000 47.0625 %RH 23.0625 47.0625 11.1875 24.9375 24.6250 1015.340 29.983 28 1 0 2017 2017 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 - 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 38 -18 100 0 125 -40 125 -40 125 -40 2000.000 0.000 100.000 0.000 100000 0 2 1 1 0 1.04 ------------------------------3cbec9ce8f05
1 Like

The device's web UI can display the XML document it is pushing. Here's a partial screenshot...

The XML data I provided in my first post above (the single, long text line that is horizontally scrollable) is the same thing except it has the extra data before and after the Devices-Detail-Response parent element.

I copied the XML document from the device's web UI page and pasted it into the driver's parse function (just to make it easy to run the code) to create an XML object as part of the parse function at runtime. I added this code after the XML object definition...

Parse code #2

When it ran, it logged the correct value for the humidity.

Bottom line is it looks like the XML data being pushed by the device is good if the data before and after the parent element is removed. I don't have any way to see the data exactly as it is pushed to HE, so I don't know if the device is adding the extra data or if HE is doing it.

Part of my confusion stems from not having any idea what HE is doing with the data when it receives it, when I use the log function to write it to the log, why the tags sometimes get stripped out when the data is written to the log, etc.

I wonder if the data you are logging after minus'ing the extra stuff still has the XML tags, but the HE log is stripping them out when it writes it to the log.

The frustrating thing is I could strip the extra data out in a few minutes of coding if I used a "legacy language", e.g., C, COBOL, REXX, or even assembler. I guess that's revealing my age.

Closing this out...

Thanks to @tomw for providing the following code that strips out the leading and trailing data in the initial LAN message. This was exactly what I was hoping for...

def parse(String message)
{
    Map parsedMessage = parseLanMessage(message)
    
    // extract the boundary string from the headers
    String boundary = parsedMessage?.headers?.getAt('Content-Type')?.split('boundary=')?.getAt(1)
    
    // get the 'middle' part of the multi-part body, which has the XML (and then some)
    String subBody = parsedMessage?.body?.split(boundary)?.getAt(1)
    
    // strip off the extraneous content from beginning and end
    subBody = subBody?.split('text/plain\r\n\r\n')?.getAt(1)
    subBody -= "--"
    
    // parse the XML so we can work with it
    def xmlObj = parseXML(subBody)

    xmlObj.owd_EDS0068.each{ sensor->
        log.debug ("ROM ID: ${sensor.ROMId}  Temperature: ${sensor.Temperature}  Humidity: ${sensor.Humidity}")
    }

    return
}

Thanks also to @snell for working on this.

2 Likes