How do I sort objects inside of objects in JSON? (using Python 2.7)

Question

I have the following code, which is a function to export a transaction history from a digital currency wallet to a json file.

The problems I am facing are two:

I would like to allow the json file to be written in utf-8, as the property 'label' can be utf-8 characters, and if I do not account for that, it will show up in the file as \u\u\u etc. However, no matter what combinations and orderings of encode/decode('utf-8'), I can not get the final output file to print to utf-8.
I would like to order each iteration in the order I wrote them in the code. I tried OrderedDict from collection, but it did not order the items so that Date comes first, etc.

Any help with figuring out how to print this to my file using utf-8 and with the order inside each item as I wrote it.

Thank you very much.

# This line is the last line of a for loop iterating through
# the list of transactions, "for each item in list"
wallet_history.append({"Date": time_string, "TXHash": tx_hash, "Label": label, "Confirmations":
                      confirmations, "Amount": value_string, "Fee": fee_string, "Balance": balance_string})
try:
    history_str = json.dumps(
        wallet_history, ensure_ascii=False, sort_keys=False, indent=4)
except TypeError:
    QMessageBox.critical(
        None, _("Unable to create json"), _("Unable to create json"))
    jsonfile.close()
    os.remove(fileName)
    return
jsonfile.write(history_str)

Please post only the part of the code which is relevant to the problem — thefourtheye
– thefourtheye, Commented Mar 29, 2014 at 6:21

m.wasowski · Accepted Answer · 2014-03-29 07:38:41Z

2

You need to ensure that both json does not escape characters, and you write your json output as unicode:

import codecs
import json

with codecs.open('tmp.json', 'w', encoding='utf-8') as f:
    f.write(json.dumps({u'hello' : u'привет!'}, ensure_ascii=False) + '\n')


$ cat tmp.json
{"hello": "привет!"}

As for your second question: you can use collections.OrderedDict, but you need to be careful to pass it directly to json.dumps without changing it to simple dict. See the difference:

from collections import OrderedDict
data = OrderedDict(zip(('first', 'second', 'last'), (1, 10, 3)))
print json.dumps(dict(data)) # {"second": 10, "last": 3, "first": 1}
print json.dumps(data) # {"first": 1, "second": 10, "last": 3}

edited Mar 29, 2014 at 7:38

answered Mar 29, 2014 at 7:15

m.wasowski

6,3761 gold badge25 silver badges30 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Community · Accepted Answer · 2017-05-23 10:31:50Z

0

To make the generated JSON encoded in UTF-8 use ensure_ascii=False:

Python 2.7.6 (default, Mar 22 2014, 22:59:56) 
>>> import json
>>> json.dumps(u'привет!', ensure_ascii=False)
u'"\u043f\u0440\u0438\u0432\u0435\u0442!"'
>>> print json.dumps(u'привет!', ensure_ascii=False)
"привет!"
>>> with open('test', 'w') as f:
...     f.write(json.dumps(u'привет!', ensure_ascii=False))
... 
Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
UnicodeEncodeError: 'ascii' codec can't encode characters in position 1-6: ordinal not in range(128)
>>> with open('test', 'w') as f:
...     f.write(json.dumps(u'привет!'.encode('utf-8'), ensure_ascii=False))
... 
>>> with open('test') as f:
...     f.read()
... 
'"\xd0\xbf\xd1\x80\xd0\xb8\xd0\xb2\xd0\xb5\xd1\x82!"'
>>> print '"\xd0\xbf\xd1\x80\xd0\xb8\xd0\xb2\xd0\xb5\xd1\x82!"'
"привет!"
>>>

As for the second question about order. It's not possible (except writing you own serializer) or it doesn't make sense: Items in JSON object are out of order using "json.dumps"?

When you pass wallet_history to json.dumps it's already lost its order, as it contains dictionaries, which are unordered.

edited May 23, 2017 at 10:31

CommunityBot

11 silver badge

answered Mar 29, 2014 at 6:26

warvariuc

60.1k45 gold badges183 silver badges234 bronze badges

5 Comments

user3074620 Over a year ago

As you can see, I do have ensure_ascii=False, and it still doesn't write in utf-8... I think maybe it's because I am storing the dumps into a string first? Is it something to do with the file.write() function?...

warvariuc Over a year ago

It's working here (see my update). How are you checking that it still doesn't write in utf-8?

user3074620 Over a year ago

I am checking by opening the file in a text editor set to utf8 encoding. Only the labels are written in /u bytes

warvariuc Over a year ago

How are you opening the file jsonfile? Why do you pass it as the second argument to json.dumps?

warvariuc Over a year ago

Also, are you sure that label isn't already ASCII encoded when passed to wallet_history.append?

Collectives™ on Stack Overflow

How do I sort objects inside of objects in JSON? (using Python 2.7)

2 Answers 2

Comments

5 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

5 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related