Java Encoding string not converting properly

Question

I have a couple of string that aren't converting properly. The issue that I had was that all my strings had weird characters on it (They are in spanish with the top accent) and I was able to convert them with the following code:

Connection to DB: After is connected I proceed to get the information from the DB as follows (it comes with json format) DB information shows as PerÃƒÂº:

if (responseCode == HttpURLConnection.HTTP_OK) {
    InputStream in = new BufferedInputStream(urlConnection.getInputStream());

    BufferedReader reader = new BufferedReader(new InputStreamReader(in));
    //BufferedReader reader = new BufferedReader(new InputStreamReader(urlConnection.getInputStream(), "iso-8859-1"), 8);
    StringBuilder sb = new StringBuilder();
    String line = null;
    while ((line = reader.readLine()) != null) {
        sb.append(line + "\n");
    }
    result = sb.toString();
    //is.close();
    Log.i("Tag:", result);
}

retrieve JSON list:

try{
    jArray = new JSONArray(welcome.result);
    for(int i=0;i<jArray.length();i++){
        JSONObject json_data = jArray.getJSONObject(i);
        Log.i("log_tag", "title:" + json_data.getString("title"));

        try {
            country = new String(json_data.getString("country").getBytes("ISO-8859-1"), "UTF-8");
        }
        catch (UnsupportedEncodingException e){

        }

However I grabbed another string in spanish and it reverted back but the other strings still show up properly. The current string looks like this PER�?º Any ideas? Im guessing is using a different encoding. The database where I'm pulling them from uses utf8_general_ci. Thank you in advance!

If json_data is a String retrieved from the database using getString() on a ResultSet, and you're getting PER�?º, then whoever inserted that data messed up. — Andreas
– Andreas, Commented Jun 4, 2016 at 17:13
you should indeed try to query your DB with MySQLWorkbench (assuming that you use MySQL) to see what you have in your table, It may not have been loaded properly as suggested by @Andreas — Nicolas Filotto
– Nicolas Filotto, Commented Jun 4, 2016 at 18:04

Nicolas Filotto · Accepted Answer · 2016-06-04 17:43:13Z

2

Simply do this:

country = json_data.getString("country");

getString returns already a String so no need to encode it into ISO-8859-1 to decode it into UTF-8 which cannot work for obvious reasons.

If you get incorrect characters even with the code above, it means that your problem comes before, probably while parsing your JSON content, it has probably not been parsed with the correct character encoding.

You need to explicitly set the character encoding to UTF-8 otherwise it will use the default one as next:

BufferedReader reader = new BufferedReader(new InputStreamReader(in, "UTF-8"));

edited Jun 4, 2016 at 17:43

answered Jun 4, 2016 at 16:46

Nicolas Filotto

45.2k11 gold badges97 silver badges129 bronze badges

Sign up to request clarification or add additional context in comments.

7 Comments

paul590 Over a year ago

Thank you, I was doing this at the beginning but the characters were coming weird and thats when I added the encoding

Nicolas Filotto Over a year ago

so even with this, you don't have correct characters?

paul590 Over a year ago

unfortunately half of the list looks ok but the other half is still coming in with this weird characters, im guessing they are in a different encoding from ISO-8859-1

Nicolas Filotto Over a year ago

if so it means that your problem comes before while parsing your Json data, it has probably not been parsed with the correct encoding, your problem is not here for sure

Andreas Over a year ago

@NicolasFilotto See my answer/rambling to maybe get an idea of what is going on.

|

Andreas · Accepted Answer · 2016-06-04 18:06:00Z

2

Trying to make sense of what you show.

You say database shows PerÃƒÂº.

In CP1252, that is the bytes 50 65 72 C3 83 C2 BA.

Those bytes as UTF-8 is PerÃº.

In CP1252, that is the bytes 50 65 72 C3 BA.

Those bytes as UTF-8 is Perú. <== Right value

It would seem you have a double bad encoding going on here.

One bad encoding could be from "database shows", i.e. how do you see what database shows?

Other bad encoding is that whoever inserted the data messed up.

I would postulate that database actually has PerÃº stored in UTF-8. The tools you use to show the value is doing it wrong. Of course, whoever inserted the value was doing it wrong too, hence the double error.

Solution: You start by fixing the code that inserted bad data. You also use a tool that can correctly show what's in the database.

Then you remove any encoding hacks from your Java code, because it very likely is doing it right to being with. Or at least, it's not the Java to database part of the code that is bad.

edited Jun 4, 2016 at 18:06

answered Jun 4, 2016 at 17:57

Andreas

160k13 gold badges164 silver badges262 bronze badges

1 Comment

paul590 Over a year ago

@Andreas Thank you for all your help! If I could choose two responses that helped me solve my issue I would select yours as well! I looked at my db and solve the issue there!

Collectives™ on Stack Overflow

Java Encoding string not converting properly

2 Answers 2

7 Comments

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

7 Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related