Base64 Decoding Issue

🚧 Developers
1

I am trying to decode a string returned from an REST API call and am having problems with unrecognised characters. I have been able to decode the same string in Javascript on my laptop, but not using Groovy on my C-4 hub.

As far as I can tell, I am receiving a UTF-8 string in the HTTP response, both in terms of the header settings and my ability to read it on screen. I am storing this in a String variable without setting the encoding. I then create a second String variable, taking in the byte array from a decodeBase64 call on the first variable, and setting the encoding to Unicode. When I output the result, I can see all but one of the characters, with the problematic output showing as a question mark. I know it is not being recognised because I take this decoded string and attempt to convert it to a hex string / int array, which produces an 0xfffd result, which I understand to be the indication for an unrecognised character. Does anyone have any tips?

Here's some snippets of the code:

String password = unit.value.password;
debugLog("retrieveAuthCode_KumoCloud: password = ${password}")
String decodedPassword = new String(password.decodeBase64(), "Unicode");

I think my issues are in the code above, but here is the hex conversion part anyway. I need to tidy it up, still very much a WIP....

int[] testPassword = new int[decodedPassword.codePointCount(0,decodedPassword.length())*2];
for(int i=0; i < decodedPassword.codePointCount(0,decodedPassword.length()); i++) {
                          
  String hex = Integer.toHexString(decodedPassword.codePointAt(i));
  debugLog("retrieveAuthCode_KumoCloud: char = ${decodedPassword.codePointAt(i)}, hex = ${hex}");
  String hex2 = ("0000" + hex).substring(hex.length()).substring(0,2);

  testPassword[i*2]   = Integer.decode("0x${hex2}") & 0xFF;
  testPassword[i*2+1] = Integer.decode("0x${hex}") & 0xFF;
                          
}

I won't put the values in here, given this is password related.

Thanks,
Simon

2

I feel like this is related. The Javascript code I am using as a comparison iterates over the byte array it gets from the Base64 decoding and calls m2.charCodeAt(0), where m2 is the byte-array entry. In Javascript this charCodeAt returns the Unicode of the first part of the surrogate pair (I'm way out of my depth here...). Unfortunately we don't appear to have access to the Character class from what I can see.... so can't use the solution in the stack overflow post...

3

I'll whitelist Character class in the next hotfix. Other java.lang classes are there, and there's no reason why Character is not.

2 Likes
4

String decodedPassword = new String(password.decodeBase64(), "Unicode");

Have you tried using "UTF-8" rather than "Unicode" as the encoding? From the Java docs, the word "Unicode" is not listed among the standard encodings/Charsets, where "UTF-8" should be and is.

5

Thanks @gopher.ny , that would be great

6

I have tried using UTF-8 as the charset and that gives me mostly unrecognized characters. I had also noticed after I posted this that Unicode was not one of the standard charsets and that UTF-8 should include Unicode characters. So not sure what is going on ... I also tried UTF-16.

The Character class should also give me access to the standard charsets object(s) to be able to reference in the String constructor, rather than just a string reference to the charset name, maybe that will make a difference....

7

When I read the Javascript description then you are dealing with UTF-16 string encoded in Base64. So try UTF-16 after the base64.decode.

8

2.2.9.130 has java.lang.Character whitelisted.

3 Likes
9

Thanks for the tip. From the first of my code snippets I had tried UTF-16 instead of Unicode, but still no success. I will upgrade to the latest 2.2.9 and try using the Character class to see if that gives me any joy. Thanks @gopher.ny

Simon

10

Still no joy with the Character class unfortunately.... Will keep trying....
EDIT: Turns out the Character class did work, I just needed to include a bitwise transform of the byte array elements to get the result I was after. See post further down for explanation...

11

Can you give more info about what you're trying to do? Maybe some sample replies to show the data format as received? And what output you are trying to get to? If there is existing code somewhere for another platform that you know works, please share that also.

12

I'm happy to keep playing with it for a few days yet, but the use case is:

This is a thermostat driver for Mitsubishi Wi-Fi adaptors, specifically those systems hooked into their Kumo Cloud-hosted system in the U.S. They have an option (though not documented I believe) to control the thermostats locally using an encrypted(?) message. So you do periodic call to their cloud API and get back an encoded password and a "cryptoSerial" (I'm assuming private key...). The base64 encoded password is decoded, mashed together with the text for the command (turn on the heating), then a SHA-256 hash is done and a few other transformations with the crypto serial and what I assume is a Kumo public key (I didn't pay enough attention during data security at Uni). You can then send this encrypted message to the thermostat over the LAN without needing to involve the cloud.

It's the decoding of the password that I am struggling with. I am essentially copying a Typescript example for the integration on HomeBridge. I am close to contacting the developer to run it past him. Here's the link (see the encodeToken method right near the end on line 467):

13

Problem solved, needed to include the "unsigning" of the integers I was getting out of the decoded byte array, something I had used in various places a few days ago in a blind attempt to get this working, I just needed to introduce as I converted the byte array coming out of the decoding call. I'll add some sample code here later, once I tidy it up, but here's a link to the article that reminded me of this:

2 Likes
14

This topic was automatically closed 365 days after the last reply. New replies are no longer allowed.