Parse JSON from CURL output using python

Question

I am trying to get value of id i.e. "id": 59 which is in the curl output in form of json. Below is the curl output in json:

[{"id":59,"description":"This is a demo project","name":"Demo_Project","name_with_namespace":"sam / Demo_Project","path":"demo_project","path_with_namespace":"sam/demo_project","created_at":"2020-03-02T08:43:13.664Z","default_branch":"master","tag_list":[],"ssh_url_to_repo":"ssh://[email protected]:2222/sam/demo_project.git","http_url_to_repo":"https://od-test.od.com/gitlab/sam/demo_project.git","web_url":"https://od-test.od.com/gitlab/sam/demo_project","readme_url":"https://od-test.od.com/gitlab/sam/demo_project/blob/master/README.md","avatar_url":null,"star_count":0,"forks_count":0,"last_activity_at":"2020-04-09T09:28:09.860Z","namespace":{"id":2259,"name":"sam","path":"sam","kind":"user","full_path":"sam","parent_id":null,"avatar_url":"https://secure.gravatar.com/avatar/755db8ssqaq50dcc9d189c53523b?s=80\u0026d=identicon","web_url":"https://od-test.od.com/gitlab/sam"}}]

I am using python to parse the json and get the value of id. I have tried the following command to do the same but got an error.

curl --header "PRIVATE-TOKEN: 9999ayayayar66" "https://od-test.od.com/gitlab/api/v4/search?scope=projects&search=demo_project" | python -c 'import sys, json; print(json.load(sys.stdin)["id"])'

Error:

Can anyone help me the correct python command to get the value of id. Thanks in advance.

Brilliant questions and you almost solved it yourself. I never thought of piping sys.stdin before. Thank you for that. — Prayson W. Daniel
– Prayson W. Daniel, Commented Apr 12, 2020 at 9:58

Jim · Accepted Answer · 2020-04-12 09:49:57Z

7

The JSON contains an array of objects but you are treating it like it is a single object:

import sys, json; print(json.load(sys.stdin)["id"])

You're basically saying "here is a collection of objects, give me the ID of the object". It doesn't make sense.

If you assume you only ever want the ID of the first object in the array, you can use this:

import sys, json; print(json.load(sys.stdin)[0]["id"])

answered Apr 12, 2020 at 9:49

Jim

74.1k15 gold badges105 silver badges114 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

BeneSim · Accepted Answer · 2020-04-12 09:51:18Z

4

The root of you JSON object is actually an array, so you should get the first item out of it first:

curl --header "PRIVATE-TOKEN: 9999ayayayar66" "https://od-test.od.com/gitlab/api/v4/search?scope=projects&search=demo_project" | python -c 'import sys, json; print(json.load(sys.stdin)[0]["id"])'

answered Apr 12, 2020 at 9:51

BeneSim

905 bronze badges

Comments

Ambitions · Accepted Answer · 2020-04-12 09:51:06Z

3

You have to write print(json.load(sys.stdin)[0]["id"]) because json response is a list of dictionaries, not a dictionary.

answered Apr 12, 2020 at 9:51

Ambitions

2,6433 gold badges16 silver badges26 bronze badges

Comments

RoadRunner · Accepted Answer · 2020-04-12 10:15:00Z

1

Since you also tagged Python Requests, your CURL command could be translated to:

import requests

headers = {
    'PRIVATE-TOKEN': '9999ayayayar66',
}

params = (
    ('scope', 'projects'),
    ('search', 'demo_project'),
)

response = requests.get('https://od-test.od.com/gitlab/api/v4/search', headers=headers, params=params)

Then you can get the id value from the JSON Response Content with response.json()[0]["id"]. This uses the built-in JSON decoder to convert the JSON response to a list of dictionaries.

edited Apr 12, 2020 at 10:15

answered Apr 12, 2020 at 9:58

RoadRunner

26.4k6 gold badges46 silver badges77 bronze badges

3 Comments

Phil Over a year ago

Requests is not part of python standard library and you may not always control/install it on-demand. Question here was about curl.

RoadRunner Over a year ago

@Phil this question was tagged with python requests, you seemed to have missed that.

Phil Over a year ago

That's correct. I have not mentioned that and indeed, python Requests makes our life easier with json.

刘孟德 · Accepted Answer · 2024-06-18 03:32:30Z

0

curl xxx | python -m json.tool

python provide a good json tool and can chained by pipeline

edited Jun 18, 2024 at 3:32

answered Jun 17, 2024 at 5:14

刘孟德

2512 silver badges9 bronze badges

1 Comment

Friedrich Over a year ago

Please edit and add a little bit of explanation how this solves the problem stated in the question.

Collectives™ on Stack Overflow

Parse JSON from CURL output using python

5 Answers 5

Comments

Comments

Comments

3 Comments

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

5 Answers 5

Comments

Comments

Comments

3 Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related