Updated state field returns incorrect data on next execution

Sequence of events (same result on an app or device)

  1. add new element to an existing state Map field, state.wkcodes
  2. sort the Map into keyed order
  3. save the new ordered map back to state.wkcodes
  4. On next execution state.wkcodes is out of order

Sample Code

			log.debug state.wkcodes
            Map codeEntry = ["x": "4321"]
            state.wkcodes << codeEntry
			log.debug state.wkcodes
			keys=state.wkcodes.keySet().sort()
			maps=[:]
			keys.each()
				{
				maps<< [(it):(state.wkcodes.get(it))]
				}
	        state.wkcodes = maps
			log.debug state.wkcodes

Debug data with annotations

  1. Start
    [ff:522345, a:11, q:2345, b:22345, f:522345, fff:522345, w:20011111, z:102, ffff:522345]

  2. add element ["x", "4321"]
    [ff:522345, a:11, q:2345, b:22345, f:522345, fff:522345, w:20011111, x:4321, z:102, ffff:522345]

  3. Sort and stored value, then exit
    [a:11, b:22345, f:522345, ff:522345, fff:522345, ffff:522345, q:2345, w:20011111, x:4321, z:102]

  4. next execution unsorted map from step 2 above returned
    [ff:522345, a:11, q:2345, b:22345, f:522345, fff:522345, w:20011111, x:4321, z:102, ffff:522345]

What I tried that did not work
setting state.wkcodes to [:] before saving ordered map

@bravenel @mike.maxwell

Maps cant be saved sorted.

Bummer.

Why not?

Dunno, i didn't create groovy...

2 Likes

Well we should definitely petition the person who did :o

Well the person or persons responsible took the time to create a SortedMap. Is that supported as a state field?

@mike.maxwell

no idea, give it a shot and let us know

Can't even get it to compile and use it as a non state field. Why are TreeMaps unsupported?

Just attempted to create a sorted map per this example but this code
SortedMap smap = new TreeMap();

throws this error at the compile phase
Expression [DeclarationExpression] is not allowed: (smap = new java.util.TreeMap()) at line number 48

No idea why TreeMap is unsupported in HE

BTW I also added these two statements just to be sure

import java.io.*;
import java.util.*;

Update added these statements to the code

import java.util.Iterator;
import java.util.Map;
import java.util.Set;
import java.util.SortedMap;
import java.util.TreeMap;

throwing error at save
Importing [java.util.SortedMap] is not allowed

You may find this interesting regarding HashMaps that I think are used to store all Maps in HE's state fields. (Never knew there where so many map types in Java/Groovy)

  1. You will need to use a different container. The very nature of a hashmap implementation all but guarantees that the original order will not be preserved

  2. As the documentation clearly states, HashMaps are unordered.
    The enumeration order is determined by the hascodes of the keys.
    If you want to preserve insertion order when enumerating the map, use LinkedHashMap. If you want enumeration order to follow the natural ordering of the keys, use TreeMap

So that brings me to requesting support for TreeMaps in State. LinkedHashMaps would also work since the map is built in sorted order.

@mike.maxwell

State in the database is a string so we can only return a HashMap type

I don't fully comprehend how state is handled in HE other than state itself is a hashMap and is stored in the HE database when the task completes.

You mention maps are converted to a string in state. So if that string includes the original Groovy/Java map type versus saying it was a generic Map, it should be trivial to create the original map type rather than make all returned maps, HashMaps.

Should none of this be possible, please allow Hub Variables to be > 255 bytes. That's what got me down this rabbit hole in the first place :man_shrugging:

It doesn't and this isn't something were going to change either.

Maybe if you post what you're try to do where you feel you need this you might get some work arounds.

Assuming I have a very large collection, hundreds each perhaps a hundred bytes long, of named data codes that are stored in state, I simply wanted to view them on the device page state fields in some rational ordered manner.

Also wanted to share the codes as a cache in a non parent app child device situation, but hub variables are limited to 255 bytes.

You might look at building an app to deal with this data. State was never intended to be a ui element.

Would your problem be solved if String Hub Variables size limit was increased to 1024?

I understand the device interface and page was not designed to be a visual interface for the stored data, but the page does function as a visual interface.

I'm assisting someone who is writing the app and devices. It's easy enough to get the codes from a device to a non parent app by defining the device to the app, but I don't know how to have a device get data from a non-parent app other than perhaps using a hub variable or file.

We gave up on hub variables (too small) and files (no system provided interface), and having the non child devices use a app based cache. So to make things easier for our testing and debugging we decided to sort the device's codes to make it easier to see what was going on, then we bumped into all state stored maps become HashMaps.

Well I learned a lot more about maps than I expected, so that's a good thing.

1 Like

Appreciate the offer and I feel it may help many other folks. However, my current codes file is in the 16K range and likely will grow.

That's clearly beyond the scope of Hub Variables.

About sorting state based maps: That is done for UI purposes at the point where the map is presented, presumably in an enum or a table of some sort. That being said, there really is no reason to need to keep a map sorted in state.

App to app is easy, and app to child is easy, and child to parent app is easy. You can use location variables as a means of app to app communication. So device makes a request of its parent app, and that makes a request of the other app...

I understand this was mainly for visual confirmation when looking at the large lump of data on the device page. So we sorted it, and honestly were surprised when the order changed after it was stored then viewed the next time. It has no impact on the function. Creating a HashMap explains why it happens.