Efficient JSON encoding for data that may be binary, but is often text

Question

I need to send a JSON packet across the wire with the contents of an arbitrary file. This may be a binary file (like a ZIP file), but most often it will be plain ASCII text.

I'm currently using base64 encoding, which handles all files, but it increases the size of the data significantly - even if the file is ASCII to begin with. Is there a more efficient way I can encode the data, other than manually checking for any non-ASCII characters and then deciding whether or not to base64-encode it?

I'm currently writing this in Python, but will probably need to do the same in Java, C# and C++, so an easily portable solution would be preferable.

moshez · Accepted Answer · 2010-04-14 00:34:26Z

2

Use quoted-printable encoding. Any language should support that. http://en.wikipedia.org/wiki/Quoted-printable

answered Apr 14, 2010 at 0:34

moshez

37.8k1 gold badge24 silver badges14 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

EMP Over a year ago

Interesting idea, thanks. QP definitely has less overhead for text data, but a lot more for binary data, so I'm not too sure. +1, anyway.

the Tin Man Over a year ago

Quoted-printable is ok for text with occasional binary but for a binary file it will really balloon the size. Base-64 should be more efficient in that case. I would try to avoid passing the file via JSON, and instead would pass a URL to it and let the software at the other end retrieve the URL as a side process after decoding the JSON structure.

moshez Over a year ago

Another thing you can do (if what Greg suggests fails, for example for network topology reasons) is to use either QP or Base64, and put one character at the beginning to let the other side know which one it is. This is certainly easy enough, on both encoder and decoder...

Collectives™ on Stack Overflow

Efficient JSON encoding for data that may be binary, but is often text

1 Answer 1

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related