Parsing datetime in python json loads

Question

I want to parse (json.loads) a json string that contains datetime values sent from a http client.

I know that I can write a custom json encoder by extending the default encoder and overriding the default method

class MyJSONEncoder(json.JSONEncoder):
    def default(self, obj):
        if isinstance(obj, (datetime.datetime,)):
            return obj.isoformat()
        elif isinstance(obj, (decimal.Decimal,)):
            return str(obj)
        else:
            return json.JSONEncoder.default(self, obj)

My questions are -

How do I customize the default json decoder? Do I need to override the decode method? Can I in some way, override/add a callback function for every field/value in the json string? (I have seen the code in json.decoder.JSONDecoder and json.scanner but am not sure what to do)
Is there an easy way to identify a specific value as a datetime string? The date values are strings in ISO format.

Thanks,

Mause · Accepted Answer · 2015-06-15 12:38:58Z

8

There are likely other solutions, but json.load & json.loads both take an object_hook argument1 that is called with every parsed object, with its return value being used in place of the provided object in the end result.

Combining this with a little tag in the object, something like this is possible;

import json
import datetime
import dateutil.parser
import decimal

CONVERTERS = {
    'datetime': dateutil.parser.parse,
    'decimal': decimal.Decimal,
}


class MyJSONEncoder(json.JSONEncoder):
    def default(self, obj):
        if isinstance(obj, (datetime.datetime,)):
            return {"val": obj.isoformat(), "_spec_type": "datetime"}
        elif isinstance(obj, (decimal.Decimal,)):
            return {"val": str(obj), "_spec_type": "decimal"}
        else:
            return super().default(obj)


def object_hook(obj):
    _spec_type = obj.get('_spec_type')
    if not _spec_type:
        return obj

    if _spec_type in CONVERTERS:
        return CONVERTERS[_spec_type](obj['val'])
    else:
        raise Exception('Unknown {}'.format(_spec_type))


def main():
    data = {
        "hello": "world",
        "thing": datetime.datetime.now(),
        "other": decimal.Decimal(0)
    }
    thing = json.dumps(data, cls=MyJSONEncoder)

    print(json.loads(thing, object_hook=object_hook))

if __name__ == '__main__':
    main()

edited Jun 15, 2015 at 12:38

answered Jun 15, 2015 at 11:31

Mause

6481 gold badge5 silver badges9 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

ksrini Over a year ago

Thanks Dominic. That makes sense. I was thinking along the lines of a creating an object with a 'type' attr, but didn't notice the object_hook argument.

AlAmilar · Accepted Answer · 2015-06-15 08:56:26Z

0

As for 2nd question, you should propably just use

import dateutil.parser dateutil.parser.parse('Your string')

method, it will try to parse your date string, and if it wouldn't be able to recognize it, it will throw Value Error. You can also use regular expressions to find fields that at least look like a dates (depending on the format you use, of course)

answered Jun 15, 2015 at 8:56

AlAmilar

1

1 Comment

ksrini Over a year ago

Are you suggesting that I apply the regex on all values? I would need to iterate over the entire dict that I get from the json string to apply the regex on each value?

Juergen · Accepted Answer · 2022-05-06 08:04:20Z

The answer from Mause is very nice and complete in the sense you cannot make a wrong conversion due to the type specifier given. But typically JSON files aren't structured like this... More often you will find key-value or key-list pairs. However, using the object_hook approach still works for this, but you need to have keys only binding to one single type. For example, a key "start" will always contain a datetime value.

import json
import datetime


class MyJSONEncoder(json.JSONEncoder):
    def default(self, obj):
        if isinstance(obj, (datetime.datetime,)):
            return obj.isoformat()
        else:
            return super().default(obj)


def cnv2DateTime(obj):
    if isinstance(obj, list):
        return [datetime.datetime.fromisoformat(i) for i in obj]
    elif isinstance(obj, str):
        return datetime.datetime.fromisoformat(obj)
    else:
        return obj


dtentries = ['range', 'start', 'end']


def object_hook(obj):
    outDict = {}
    for k, v in obj.items():
        if k in dtentries:
            vn = cnv2DateTime(v)
            outDict[k] = vn
        else:
            outDict[k] = v
    return outDict


def main():
    data = {
        "version": "1.0.0",
        "range": [datetime.datetime.now(), datetime.datetime.now() + + datetime.timedelta(0, 60)],
        "files": ['1', '2', '3'],
        "entries": {'1': {'start': datetime.datetime(2022, 1, 1, 8, 0, 0), 'end': datetime.datetime(2022, 1, 12, 17, 45, 0)},
                    '2': {'start': datetime.datetime(2022, 2, 1, 8, 0, 0), 'end': datetime.datetime(2022, 2, 12, 17, 45, 0)}}
    }
    jdata = json.dumps(data, cls=MyJSONEncoder)

    ddata = json.loads(jdata, object_hook=object_hook)
    print('Done')


if __name__ == '__main__':
    main()

As you can see from the example this works fine also for nested constructs or lists of datetime values. Hope, this is useful for somebody.

Collectives™ on Stack Overflow

Parsing datetime in python json loads

3 Answers 3

1 Comment

1 Comment

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

1 Comment

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Related