How to extract specific data from JSON object using python?

Question

I'm trying to scrape a website and get items list from it using python. I parsed the html using BeaufitulSoup and made a JSON file using json.loads(data). The JSON object looks like this:

{ ".1768j8gv7e8__0":{ 
    "context":{ 
       //some info
    },
    "pathname":"abc",
    "showPhoneLoginDialog":false,
    "showLoginDialog":false,
    "showForgotPasswordDialog":false,
    "isMobileMenuExpanded":false,
    "showFbLoginEmailDialog":false,
    "showRequestProductDialog":false,
    "isContinueWithSite":true,
    "hideCoreHeader":false,
    "hideVerticalMenu":false,
    "sequenceSeed":"web-157215950176521",
    "theme":"default",
    "offerCount":null
 },
 ".1768j8gv7e8.6.2.0.0__6":{ 
    "categories":[ 

    ],
    "products":{ 
       "count":12,
       "items":[ 
          { 
             //item info
          },
          { 
            //item info
          },
          { 
            //item info
          }
       ],
       "pageSize":50,
       "nextSkip":100,
       "hasMore":false
    },
    "featuredProductsForCategory":{ 

    },
    "currentCategory":null,
    "currentManufacturer":null,
    "type":"Search",
    "showProductDetail":false,
    "updating":false,
    "notFound":false
 }
}

I need the items list from product section. How can I extract that?

possibly duplicate stackoverflow.com/questions/47302018/…

SeleniumUser
– SeleniumUser

2019-10-27 10:53:44 +00:00
Commented Oct 27, 2019 at 10:53 — SeleniumUser
– SeleniumUser, Commented Oct 27, 2019 at 10:53
Possible duplicate of Extract data from JSON in python

Mišo
– Mišo

2019-10-27 11:36:04 +00:00
Commented Oct 27, 2019 at 11:36 — Mišo
– Mišo, Commented Oct 27, 2019 at 11:36

ByteFlowr · Accepted Answer · 2019-10-27 11:20:22Z

1

Just do:

products = jsonObject[list(jsonObject.keys())[1]]["products"]["items"]

edited Oct 27, 2019 at 11:20

answered Oct 27, 2019 at 10:52

ByteFlowr

5007 silver badges16 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Araf Over a year ago

To get the items list, i tried this way: ``` items = products[list(products.keys())[1]]["items"] ``` But it's not working. I'm new in python. A little explanation would help a lot

ByteFlowr Over a year ago

I edited my solution, now you'll get the items list. It's a a list inside the dict products so you have to access it first.

Jacek Rojek · Accepted Answer · 2019-10-27 11:30:23Z

1

import json packagee and map every entry to a list of items if it has any:

This solution is more universal, it will check all items in your json and find all the items without hardcoding the index of an element

import json

data = '{"p1": { "pathname":"abc" },  "p2": { "pathname":"abcd", "products": { "items" : [1,2,3]} }}'

# use json package to convert json string to dictionary
jsonData = json.loads(data)
type(jsonData) # dictionary

# use "list comprehension" to iterate over all the items in json file
# itemData['products']["items"] - select items from data
# if "products" in itemData.keys() - check if given item has products 
[itemData['products']["items"] for itemId, itemData in jsonData.items() if "products" in itemData.keys()]

Edit: added comments to code

edited Oct 27, 2019 at 11:30

answered Oct 27, 2019 at 11:11

Jacek Rojek

1,1328 silver badges16 bronze badges

2 Comments

Araf Over a year ago

can you kindly explain your code a bit? I'm new in python

Jacek Rojek Over a year ago

added explanation comments

Max Voisard · Accepted Answer · 2019-10-27 12:15:59Z

1

I'll just call the URL of the JSON file you got from BeautifulSoup "response" and then put in a sample key in the items array, like itemId:

import json
json_obj = json.load(response)
array = []
for i in json_obj['items']:
   array[i] = i['itemId']
print(array)

answered Oct 27, 2019 at 12:15

Max Voisard

1,9721 gold badge11 silver badges20 bronze badges

Collectives™ on Stack Overflow

How to extract specific data from JSON object using python?

3 Answers 3

2 Comments

2 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

2 Comments

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related