Java UTF-8 encoding

Question

I have a String like this

String str = "\u0e04\u0e38\u0e13\u0e23\u0e39\u0e49\u0e21\u0e31\u0e49\u0e22\u0e44\u0e14\u0e42\u0e19";

It actually looks like ช1: คุณรู้มั้ยไดโนเสาร์ตั

What I want is to keep the string as a string format so that str.charAt(3) is 'e' rather than a strange character.

How to do this? Help

Further explain: I get this string from a file. I read a line in the file to a string, and this line appears to be "\u0e04\u0e38\u0e13\u0e23\u0e39\u0e49\u0e21\u0e31\u0e49\u0e22\u0e44\u0e14\u0e42\u0e19". So in memory, this string is like this.

Code here:

FileReader fr = new FileReader("sample2.json");
BufferedReader br = new BufferedReader(fr);

String line;
while((line = br.readLine()) != null)
{
    JSONObject data = new JSONObject(line);
        String text = data.getString("text");

This line in the file is "\u0e04\u0e38\u0e13\u0e23\u0e39\u0e49\u0e21\u0e31\u0e49\u0e22\u0e44\u0e14\u0e42\u0e19"

Now I want to keep the string text as its original format.

You posted the actual rendered text instead of a screenshot, which is going to be tricky here. Can you confirm that the font being used has the actual correct glyph you're wanting to display? — chrylis -cautiouslyoptimistic-
– chrylis -cautiouslyoptimistic-, Commented Nov 22, 2013 at 22:28
What is the actual input and what is the desired output? Imagine File -> JSON -> HTML. — Paul Vargas
– Paul Vargas, Commented Nov 23, 2013 at 22:37

JB Nizet · Accepted Answer · 2013-11-22 22:24:19Z

5

You just need to escape every backslash:

String str = "\\u0e04\\u0e38...";

answered Nov 22, 2013 at 22:24

JB Nizet

694k94 gold badges1.3k silver badges1.3k bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

chrylis -cautiouslyoptimistic- Over a year ago

Standard answer, nonstandard question. (I didn't downvote, but you answered "how to keep Java from treating these as Unicode escapes", which is what OP actually wants in this case.)

fleetC0m Over a year ago

upvoted. This is the most elegant solution for now, and it works for the op's requirement str.charAt(4) is 'e'. The double slash occupies one char slot

peter.petrov · Accepted Answer · 2013-11-22 22:26:20Z

1

I guess you've read this string from a file or stream. Seems you've read it using the wrong encoding (not the one the String was encoded with when it was written to that file/stream). That's why you get this issue, I think.

We don't worry about encodings when Strings are in memory (in the memory of the JVM for example). Encodings start to matter when you need to write your in-memory data/String to file/stream or to read it from file/stream.

answered Nov 22, 2013 at 22:26

peter.petrov

39.6k23 gold badges114 silver badges193 bronze badges

3 Comments

Duran Over a year ago

Thank you peter. Yes I read this string from a file to do some pre-processing. Now I want to write this string to a file. What can I do to keep the string as its original format?

peter.petrov Over a year ago

Well, you just need to find out the encoding of the file (is it UTF-8, is it UTF-16, is it Windows-1252, etc.). Whoever created the file usually defines the encoding. Then once you know this, you just have to specify that same encoding explicitly in the Java code which you have and which reads the String from this file in the memory of your JVM.

peter.petrov Over a year ago

See also the answer of JB Nizet. Seems more concrete than mine and seems useful.

ljgw · Accepted Answer · 2013-11-22 23:06:36Z

0

Okay, this looks dumb, but it will work in your case:

Instead of:

JSONObject data = new JSONObject(line);

JSONObject data = new JSONObject(line.replaceAll("\\\\", "\\\\\\\\"));

The problem is that JSON converts your unicode chars for your 'convenience'.

answered Nov 22, 2013 at 23:06

ljgw

2,7811 gold badge22 silver badges39 bronze badges

2 Comments

Duran Over a year ago

JSON still recognize it as UTF-8

ljgw Over a year ago

odd.. I'm wondering what I'm doing differently than you. What do you get when you: System.out.println(line); ? Also, what is the exact content of your file (perhaps it is online somewhere?)

Collectives™ on Stack Overflow

Java UTF-8 encoding

3 Answers 3

2 Comments

3 Comments

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

2 Comments

3 Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related