Problems converting byte array to string and back to byte array

Question

There are a lot of questions with this topic, the same solution, but this doesn't work for me. I have a simple test with an encryption. The encryption/decryption itself works (as long as I handle this test with the byte array itself and not as Strings). The problem is that don't want to handle it as byte array but as String, but when I encode the byte array to string and back, the resulting byte array differs from the original byte array, so the decryption doesn't work anymore. I tried the following parameters in the corresponding string methods: UTF-8, UTF8, UTF-16, UTF8. None of them work. The resulting byte array differs from the original. Any ideas why this is so?

Encrypter:

public class NewEncrypter
{
    private String algorithm = "DESede";
    private Key key = null;
    private Cipher cipher = null;

    public NewEncrypter() throws NoSuchAlgorithmException, NoSuchPaddingException
    {
         key = KeyGenerator.getInstance(algorithm).generateKey();
         cipher = Cipher.getInstance(algorithm);
    }

    public byte[] encrypt(String input) throws Exception
    {
        cipher.init(Cipher.ENCRYPT_MODE, key);
        byte[] inputBytes = input.getBytes("UTF-16");

        return cipher.doFinal(inputBytes);
    }

    public String decrypt(byte[] encryptionBytes) throws Exception
    {
        cipher.init(Cipher.DECRYPT_MODE, key);
        byte[] recoveredBytes = cipher.doFinal(encryptionBytes);
        String recovered = new String(recoveredBytes, "UTF-16");

        return recovered;
    }
}

This is the test where I try it:

public class NewEncrypterTest
{
    @Test
    public void canEncryptAndDecrypt() throws Exception
    {
        String toEncrypt = "FOOBAR";

        NewEncrypter encrypter = new NewEncrypter();

        byte[] encryptedByteArray = encrypter.encrypt(toEncrypt);
        System.out.println("encryptedByteArray:" + encryptedByteArray);

        String decoded = new String(encryptedByteArray, "UTF-16");
        System.out.println("decoded:" + decoded);

        byte[] encoded = decoded.getBytes("UTF-16");
        System.out.println("encoded:" + encoded);

        String decryptedText = encrypter.decrypt(encoded); //Exception here
        System.out.println("decryptedText:" + decryptedText);

        assertEquals(toEncrypt, decryptedText);
    }
}

You first need to convert the bytes to something that can be presented as a string. Usually by converting to hex or base64. — Roger Lindsjö
– Roger Lindsjö, Commented Feb 1, 2012 at 15:07
What's the actual difference you see in the byte arrays before and after converting to a string? — Herms
– Herms, Commented Feb 1, 2012 at 15:08
@Roger Lindsjö: thanks for the tipp. I will try it immediately. — Bevor
– Bevor, Commented Feb 1, 2012 at 15:10
@Herms: An example -> encryptedByteArray:[B@7df17e77, encoded:[B@79a5f739 — Bevor
– Bevor, Commented Feb 1, 2012 at 15:11
Those look like memory addresses, not the actual contents of the array. — Herms
– Herms, Commented Feb 1, 2012 at 15:29

Joni · Accepted Answer · 2012-02-02 11:09:37Z

124

It is not a good idea to store encrypted data in Strings because they are for human-readable text, not for arbitrary binary data. For binary data it's best to use byte[].

However, if you must do it you should use an encoding that has a 1-to-1 mapping between bytes and characters, that is, where every byte sequence can be mapped to a unique sequence of characters, and back. One such encoding is ISO-8859-1, that is:

    String decoded = new String(encryptedByteArray, "ISO-8859-1");
    System.out.println("decoded:" + decoded);

    byte[] encoded = decoded.getBytes("ISO-8859-1"); 
    System.out.println("encoded:" + java.util.Arrays.toString(encoded));

    String decryptedText = encrypter.decrypt(encoded);

Other common encodings that don't lose data are hexadecimal and base64, but sadly you need a helper library for them. The standard API doesn't define classes for them.

With UTF-16 the program would fail for two reasons:

String.getBytes("UTF-16") adds a byte-order-marker character to the output to identify the order of the bytes. You should use UTF-16LE or UTF-16BE for this to not happen.
Not all sequences of bytes can be mapped to characters in UTF-16. First, text encoded in UTF-16 must have an even number of bytes. Second, UTF-16 has a mechanism for encoding unicode characters beyond U+FFFF. This means that e.g. there are sequences of 4 bytes that map to only one unicode character. For this to be possible the first 2 bytes of the 4 don't encode any character in UTF-16.

edited Feb 2, 2012 at 11:09

answered Feb 1, 2012 at 15:53

Joni

112k14 gold badges151 silver badges201 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Bevor Over a year ago

Today I saw that I have problem while using encryption and decryption in different VMs. Obviously only your solution works.

Joni Over a year ago

Your approach of using Apache Commons Codec should also work, but you have to distribute the commons-codec library with your application.

Aleksandar Ilic · Accepted Answer · 2015-10-15 15:40:27Z

31

Accepted solution will not work if your String has some non-typical charcaters such as š, ž, ć, Ō, ō, Ū, etc.

Following code worked nicely for me.

byte[] myBytes = Something.getMyBytes();
String encodedString = Base64.encodeToString(bytes, Base64.NO_WRAP);
byte[] decodedBytes = Base64.decode(encodedString, Base64.NO_WRAP);

answered Oct 15, 2015 at 15:40

Aleksandar Ilic

1,57616 silver badges20 bronze badges

5 Comments

AntJavaDev Over a year ago

the only working solution , although you have to depend to ApacheCommons

Aleksandar Ilic Over a year ago

I was using android.util.Base64. If you don't want to include ApacheCommons I guess you can always copy the file in your project. Here is the source: github.com/android/platform_frameworks_base/blob/master/core/…

AntJavaDev Over a year ago

yes i pointed that for java applications , this utility class comes with JDK 1.8 as well , but for previous java Versions you have to depend to ApacheCommons as it is the only working method.

Odin Over a year ago

I don't know how to start thanking you. I am very very very grateful. More codes to you brain.

ObviousChild Over a year ago

Another alternative is to use javax.xml.bind.DatatypeConverter.printHexBinary(bytesToHexString[]) and javax.xml.bind.DatatypeConverter.parseHexBinary("hexStringTobytes")

Bevor · Accepted Answer · 2012-02-02 10:08:14Z

6

Now, I found another solution too...

    public class NewEncrypterTest
    {
        @Test
        public void canEncryptAndDecrypt() throws Exception
        {
            String toEncrypt = "FOOBAR";

            NewEncrypter encrypter = new NewEncrypter();

            byte[] encryptedByteArray = encrypter.encrypt(toEncrypt);
            String encoded = String.valueOf(Hex.encodeHex(encryptedByteArray));

            byte[] byteArrayToDecrypt = Hex.decodeHex(encoded.toCharArray());
            String decryptedText = encrypter.decrypt(byteArrayToDecrypt); 

            System.out.println("decryptedText:" + decryptedText);

            assertEquals(toEncrypt, decryptedText);
        }
    }

edited Feb 2, 2012 at 10:08

answered Feb 1, 2012 at 16:00

Bevor

8,63415 gold badges83 silver badges150 bronze badges

Comments

Alexander Pavlov · Accepted Answer · 2012-02-01 15:50:47Z

0

Your problem is that you cannot build a UTF-16 (or any other encoding) String from an arbitrary byte array (see UTF-16 on Wikipedia). It is up to you, however, to serialize and deserialize the encrypted byte array without any loss, in order to, say, persist it, and make use of it later. Here's the modified client code that should give you some insight of what's actually happening with the byte arrays:

public static void main(String[] args) throws Exception {
  String toEncrypt = "FOOBAR";

  NewEncrypter encrypter = new NewEncrypter();

  byte[] encryptedByteArray = encrypter.encrypt(toEncrypt);
  System.out.println("encryptedByteArray:" + Arrays.toString(encryptedByteArray));

  String decoded = new String(encryptedByteArray, "UTF-16");
  System.out.println("decoded:" + decoded);

  byte[] encoded = decoded.getBytes("UTF-16");
  System.out.println("encoded:" + Arrays.toString(encoded));

  String decryptedText = encrypter.decrypt(encryptedByteArray); // NOT the "encoded" value!
  System.out.println("decryptedText:" + decryptedText);
}

This is the output:

encryptedByteArray:[90, -40, -39, -56, -90, 51, 96, 95, -65, -54, -61, 51, 6, 15, -114, 88]
decoded:<some garbage>
encoded:[-2, -1, 90, -40, -1, -3, 96, 95, -65, -54, -61, 51, 6, 15, -114, 88]
decryptedText:FOOBAR

The decryptedText is correct, when restored from the original encryptedByteArray. Please note that the encoded value is not the same as encryptedByteArray, due to the data loss during the byte[] -> String("UTF-16")->byte[] conversion.

answered Feb 1, 2012 at 15:50

Alexander Pavlov

32.4k5 gold badges71 silver badges95 bronze badges

3 Comments

Bevor Over a year ago

Thanks, at the moment I found another solution too (see my answer). Nevertheless I will accept your answer, because your solution works too.

Bevor Over a year ago

Today I saw that you decrypt the original byte array. This is actually not what I wanted (and unfortunately I had problem again decrypting it when using this in different VMs), so only Joni's solution work.

Alexander Pavlov Over a year ago

My code was just an illustration of the error you had in your code, not a serialization solution ("It is up to you, however, to serialize and deserialize the encrypted byte array without any loss"), as you might want to use tons of different serialization techniques (using DataIn/OutputStreams etc.)

Collectives™ on Stack Overflow

Problems converting byte array to string and back to byte array

4 Answers 4

2 Comments

5 Comments

Comments

3 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

2 Comments

5 Comments

Comments

3 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related