How to extract information from multiple json files using python

Question

I have multiple json files in a folder. I had implemented a way to catch only .json files in this folder. My problem is: I need to extract some information contained in each of those files but it didn't work the way I expected. I need to find a way to get this information and convert all into a pandas dataframe.

The variable jsons_data contains all .json files

jsons_data = json_import(path_to_json)

for index, js in enumerate(jsons_data):
    with open(os.path.join(path_to_json, js)) as json_file:
        data = json.loads(json_file)

print(data)

Tom Ron · Accepted Answer · 2021-10-01 05:06:28Z

3

The problem in your code is that on every iteration you override the content of data.

I assume you want to create on big dataframe from all the files in that case you can do -

dataframes = []
for js in jsons_data:
    dataframes.append(pd.read_json(os.path.join(path_to_json, js)))
df = pd.concat(dataframes)

See documentation about read_json - https://pandas.pydata.org/pandas-docs/version/1.3.0rc1/reference/api/pandas.read_json.html

answered Oct 1, 2021 at 5:06

Tom Ron

6,2413 gold badges28 silver badges43 bronze badges

Sign up to request clarification or add additional context in comments.

10 Comments

Felipe Roque Over a year ago

Yes, that is what i want to do but i tried you code and didn't worked, instead of this had returned an error "all arrays must be of the same length". Here it's a image showing the list o .json files i have imgur.com/a/mNpallS

Tom Ron Over a year ago

Do all the json files have the same format?

Felipe Roque Over a year ago

Yes, all have the same format

Tom Ron Over a year ago

It is hard for me to answer without having a look in the data itself but I tend to think that one of the files might have an extra column or something like that

Felipe Roque Over a year ago

Here goes one of those .json files gist.github.com/feliperoque/2d708597a5b62feb8570a6c8582717c8

|

Collectives™ on Stack Overflow

How to extract information from multiple json files using python

1 Answer 1

10 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

10 Comments

Your Answer

Sign up or log in

Post as a guest

Related