Convert JSON to set of nested Python classes

Question

I need to process multiple messages in JSON format. Each message has its own nested structure. I would like to create a Python SDK to process these messages. My idea is to map each JSON structure into a set of nested Python classes. Currently, I'm doing it manually. But it is a tedious task.

Please find an example JSON message below:

{
    "GlobalGnbId": {
        "PlmnIdentity": {
            "Data": [
                 19,
                241,
                132
            ]
        },
        "GnbId": {
            "Value":   1,
            "Length": 22
        }
    },
    "OptionalGnbDuId": 1
}

Please find below my own handcrafted set of Python classes to work with this JSON message:

class PlmnIdentity(BaseModel):
    """Class for PLMN identity"""
    Data: list[int]

class GnbId(BaseModel):
    """Class for gNodeB ID"""
    Value:  int
    Length: int

class GlobalGnbId(BaseModel):
    """Class for global gNodeB ID"""
    PlmnIdentity:    PlmnIdentity
    GnbId:           GnbId

class NodeId(BaseModel):
    """Class for node ID"""
    GlobalGnbId:     GlobalGnbId
    OptionalGnbDuId: int

Finally, please find below a full minimal example:

from pydantic import BaseModel, TypeAdapter

import json

class PlmnIdentity(BaseModel):
    """Class for PLMN identity"""
    Data: list[int]

class GnbId(BaseModel):
    """Class for gNodeB ID"""
    Value:  int
    Length: int

class GlobalGnbId(BaseModel):
    """Class for global gNodeB ID"""
    PlmnIdentity:    PlmnIdentity
    GnbId:           GnbId

class NodeId(BaseModel):
    """Class for node ID"""
    GlobalGnbId:     GlobalGnbId
    OptionalGnbDuId: int

node_id_str = \
"""
{
    "GlobalGnbId": {
        "PlmnIdentity": {
            "Data": [
                 19,
                241,
                132
            ]
        },
        "GnbId": {
            "Value":   1,
            "Length": 22
        }
    },
    "OptionalGnbDuId": 1
}
"""

# NodeId as class
node_id_class = TypeAdapter(NodeId).validate_json(node_id_str)

print(node_id_class)

print(node_id_class.GlobalGnbId)
print(node_id_class.GlobalGnbId.PlmnIdentity)
print(node_id_class.GlobalGnbId.PlmnIdentity.Data)

print(node_id_class.GlobalGnbId.GnbId)
print(node_id_class.GlobalGnbId.GnbId.Value)
print(node_id_class.GlobalGnbId.GnbId.Length)

print(node_id_class.OptionalGnbDuId)

# NodeId as dictionary
node_id_dict = node_id_class.model_dump()

print(node_id_dict)

My question is there an automatic or semi-automatic way to map a nested JSON message to a set of Python classes?

You might check out How to unpack nested JSON into Python Dataclass ans see if there is anything you can use there — JonSG
– JonSG, Commented Nov 27, 2024 at 18:00
Seems like a lot of trouble to map it into classes. Why not just work with it as-is? — John Gordon
– John Gordon, Commented Nov 27, 2024 at 18:26
With classes I can use Pydantic to ensure all the fields are in their right places. Also I prefer to use the "dot" notation for accessing the fields, e.g. NodeId.GlobalGnbId.GnbId instead of a nested dictionary notation dict['NodeId']['GlobalGnbId']['GnbId']. — mabalenk
– mabalenk, Commented Nov 27, 2024 at 18:31
You can use SimpleNamespace from the built-in types library to wrap the dict object and then access fields using "dot" notation. See a basic example here. You can couple this with JSON schema validation to validate the incoming JSON. See a basic example here. The schema can be defined in code (like the example) or loaded from file. — Jason Snouffer
– Jason Snouffer, Commented Nov 28, 2024 at 4:06
@JasonSnouffer The JSON schema validation approach is interesting, but I’m curious about its specific use case. If you have time, I’d appreciate it if you could check out my library, dataclass-wizard. It might be worth considering adding support for JSON schema validation there. I believe users would find it valuable. For context, the library is lightweight and, in the upcoming v1 major release, will be approximately 2x faster than Pydantic. Just sharing for awareness. Thanks! — Wizard.Ritvik
– Wizard.Ritvik, Commented Dec 8, 2024 at 18:08

Wizard.Ritvik · Accepted Answer · 2024-11-28 02:10:50Z

I strongly recommend exploring dataclass-wizard! This fast, strongly-typed de/serialization library is built on dataclasses and features minimal dependencies. It's well-tested and ready for production use.

Disclaimer: I am the author/maintainer of this library.

Generating Dataclass Schema

First, install the package:

pip install dataclass-wizard

Next, use the included wiz CLI utility to generate a schema for your JSON data:

echo '{
    "GlobalGnbId": {
        "PlmnIdentity": {
            "Data": [
                 19,
                241,
                132
            ]
        },
        "GnbId": {
            "Value":   1,
            "Length": 22
        }
    },
    "OptionalGnbDuId": 1
}' | wiz gs -x

That gave me Python dataclass schema to use:

from __future__ import annotations

from dataclasses import dataclass

from dataclass_wizard import JSONWizard


@dataclass
class Data(JSONWizard):
    """
    Data dataclass

    """
    global_gnb_id: GlobalGnbId
    optional_gnb_du_id: int


@dataclass
class GlobalGnbId:
    """
    GlobalGnbId dataclass

    """
    plmn_identity: PlmnIdentity
    gnb_id: GnbId


@dataclass
class PlmnIdentity:
    """
    PlmnIdentity dataclass

    """
    data: list[int]


@dataclass
class GnbId:
    """
    GnbId dataclass

    """
    value: int
    length: int

Load JSON Data to Dataclass

Now I can proceed to de/serialize the data using the Dataclass Wizard library:

from __future__ import annotations

from dataclasses import dataclass

from dataclass_wizard import JSONWizard


@dataclass
class Data(JSONWizard):
    """
    Data dataclass

    """
    global_gnb_id: GlobalGnbId
    optional_gnb_du_id: int


@dataclass
class GlobalGnbId:
    """
    GlobalGnbId dataclass

    """
    plmn_identity: PlmnIdentity
    gnb_id: GnbId


@dataclass
class PlmnIdentity:
    """
    PlmnIdentity dataclass

    """
    data: list[int]


@dataclass
class GnbId:
    """
    GnbId dataclass

    """
    value: int
    length: int

d = Data.from_json("""
{
    "GlobalGnbId": {
        "PlmnIdentity": {
            "Data": [
                 19,
                241,
                132
            ]
        },
        "GnbId": {
            "Value":   1,
            "Length": 22
        }
    },
    "OptionalGnbDuId": 1
}
""")

print(repr(d))

Result:

Data(global_gnb_id=GlobalGnbId(plmn_identity=PlmnIdentity(data=[19, 241, 132]), gnb_id=GnbId(value=1, length=22)), optional_gnb_du_id=1)

As you can see, one of the main benefits is the automatic letter-case transformation. For example, TitleCase in your JSON input is seamlessly mapped to a snake_case field name, following Python's convention.

Hope this helps!

user26979062 · Accepted Answer · 2024-11-28 13:01:13Z

1

You can use Python’s dataclasses module alongside libraries like pydantic or dataclasses-json to map JSON to nested Python classes automatically.

Example:

from dataclasses import dataclass
from dataclasses_json import dataclass_json

@dataclass_json
@dataclass
class Entry:
    key: str
    value: str

# Small JSON
json_data = '{"key": "example_key", "value": "example_value"}'

# Deserialize JSON to Python object
entry = Entry.from_json(json_data)

# Access attributes
print(entry.key)   # Output: example_key
print(entry.value) # Output: example_value

# Serialize back to JSON
print(entry.to_json())  # Output: {"key": "example_key", "value": "example_value"}

edited Nov 28, 2024 at 13:01

answered Nov 27, 2024 at 19:07

user26979062

314 bronze badges

2 Comments

mabalenk Nov 27, 2024 at 22:22

Can you please provide a minimal example of how to use it?

user26979062 Nov 28, 2024 at 12:59

Please see this link can help you - lidatong.github.io/dataclasses-json and pypi.org/project/dataclasses-json/0.0.19

Collectives™ on Stack Overflow

Convert JSON to set of nested Python classes

2 Answers 2

Generating Dataclass Schema

Load JSON Data to Dataclass

Comments

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Generating Dataclass Schema

Load JSON Data to Dataclass

Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related