Shell Script CURL JSON value to variable

Question

I was wondering how to parse the CURL JSON output from the server into variables.

Currently, I have -

curl -X POST -H "Content: agent-type: application/x-www-form-urlencoded" https://www.toontownrewritten.com/api/login?format=json -d username="$USERNAME" -d password="$PASSWORD" | python -m json.tool

But it only outputs the JSON from the server and then have it parsed, like so:

{
    "eta": "0",
    "position": "0",
    "queueToken": "6bee9e85-343f-41c7-a4d3-156f901da615",
    "success": "delayed"
}

But how do I put - for example the success value above returned from the server into a variable $SUCCESS and have the value as delayed & have queueToken as a variable $queueToken and 6bee9e85-343f-41c7-a4d3-156f901da615 as a value?

Then when I use-

echo "$SUCCESS"

it shows this as the output -

delayed

And when I use

echo "$queueToken"

and the output as

6bee9e85-343f-41c7-a4d3-156f901da615

Thanks!

I didnt really try anything, for now, I just manually have echo to ask the user questions of what is the output of the JSON and have user manually type in it and have it stored inside the variable with "read SUCCESS" command. — EndenDragon
– EndenDragon, Commented May 3, 2015 at 21:08

glenn jackman · Accepted Answer · 2015-05-04 01:47:15Z

4

Find and install jq (https://stedolan.github.io/jq/). jq is a JSON parser. JSON is not reliably parsed by line-oriented tools like sed because, like XML, JSON is not a line-oriented data format.

In terms of your question:

source <(
    curl -X POST -H "$content_type" "$url" -d username="$USERNAME" -d password="$PASSWORD" | 
    jq -r '. as $h | keys | map(. + "=\"" + $h[.] + "\"") | .[]'
)

The jq syntax is a bit weird, I'm still working on it. It's basically a series of filters, each pipe taking the previous input and transforming it. In this case, the end result is some lines that look like variable="value"

This answer uses bash's "process substitution" to take the results of the jq command, treat it like a file, and source it into the current shell. The variables will then be available to use.

answered May 4, 2015 at 1:47

glenn jackman

249k42 gold badges233 silver badges362 bronze badges

Sign up to request clarification or add additional context in comments.

14 Comments

EndenDragon Over a year ago

Tried it, but How do I call the variables? Like, echo "$queueToken"?

EndenDragon Over a year ago

I tried

source <( curl -X POST -H "Content: apent-type: application/x-www-form-urlencoded" https://www.toontownrewritten.com/api/login?format=json -d username="$USERNAME" -d password="$PASSWORD" | jq -r '. as $h | keys | map(. + "=\"" + $h[.] + "\"") | .[]' | jq -r '.queueToken as $queueToken' )  echo "$queueToken"

, but I never seem to make this work.

glenn jackman Over a year ago

you need to remove the pipe to the 2nd jq command: it's eating the output of the first jq, so the source doesn't have any effect. The $queueToken in that jq command is a jq variable, not a shell variable. Do set -x and then run the command I showed.

EndenDragon Over a year ago

How do I pipe the jq variables to shell then?

glenn jackman Over a year ago

You don't. The <(,,,) process substitution syntax executes the command and holds the output in something the shell can handle like a file. Then, the source command evaluates the contents of that "file" in your current shell. It is the source command that actually creates the variables given the jq output.

|

Pedro Lobito · Accepted Answer · 2015-05-03 21:58:03Z

1

Here's an example of Extract a JSON value from a BASH script

#!/bin/bash
function jsonval {
    temp=`echo $json | sed 's/\\\\\//\//g' | sed 's/[{}]//g' | awk -v k="text" '{n=split($0,a,","); for (i=1; i<=n; i++) print a[i]}' | sed 's/\"\:\"/\|/g' | sed 's/[\,]/ /g' | sed 's/\"//g' | grep -w $prop`
    echo ${temp##*|}
}

json=`curl -s -X GET http://twitter.com/users/show/$1.json`
prop='profile_image_url'
picurl=`jsonval`

`curl -s -X GET $picurl -o $1.png`

A bash script which demonstrates parsing a JSON string to extract a property value. The script contains a jsonval function which operates on two variables, json and prop. When the script is passed the name of a twitter user it attempts to download the user's profile picture.

answered May 3, 2015 at 21:58

Pedro Lobito

99.8k36 gold badges274 silver badges278 bronze badges

1 Comment

EndenDragon Over a year ago

I do not know how to use this in terms of my script, mind demonstrating how it works and how I use it? Thanks!

F. Hauri - Give Up GitHub · Accepted Answer · 2015-05-14 17:24:14Z

0

You could use perl module on command line:

1st, ensure they is installed, under debian based, you could

sudo apt-get install libjson-xs-perl

But for other OS, you could install perl modules via CPAN (the Comprehensive Perl Archive Network):

cpan App::cpanminus
cpan JSON::XS

Note: You may have to run this with superuser privileges.

then:

curlopts=(-X POST -H
    "Content: apent-type: application/x-www-form-urlencoded"
    -d username="$USERNAME" -d password="$PASSWORD")
curlurl=https://www.toontownrewritten.com/api/login?format=json

. <(
    perl -MJSON::XS -e '
        $/=undef;my $a=JSON::XS::decode_json <> ;
        printf "declare -A Json=\047(%s)\047\n", join " ",map {
            "[".$_."]=\"".$a->{$_}."\""
        } qw|queueToken success eta position|;
    ' < <(
        curl "${curlopts[@]}" $curlurl
    )
)

The line qw|...| let you precise which variables you want to be driven... This could be replaced by keys $a, but could have to be debugged as some characters is forbiden is associative arrays values names.

echo ${Json[queueToken]}
6bee9e85-343f-41c7-a4d3-156f901da615

echo ${Json[eta]}
0

edited May 14, 2015 at 17:24

answered May 5, 2015 at 16:47

F. Hauri - Give Up GitHub

72.9k19 gold badges136 silver badges154 bronze badges

7 Comments

EndenDragon Over a year ago

This doesnt work as I had hoped... I cant seem to use echo ${Json[queueToken]} when I used the code.

F. Hauri - Give Up GitHub Over a year ago

@JeremyZh answer edited: added some double quotes and tested with real url (my 1st test used some json of mine)...

F. Hauri - Give Up GitHub Over a year ago

but as I don't have an account, I could only: echo ${!Json[@]} give: banner success and echo ${Json[success]} print false

EndenDragon Over a year ago

I am on a mac intel environment, that use shell scripting. So I do not know where I can install JSON::XS perl libs... Also after using the code along with echo ${Json[queueToken]}, it outputs nothing, i suppose I need to install some dependencies so it works. Where/how can I install JSON::XS perl libs? sudo apt-get install libjson-xs-perl doesnt work on mac. You can always make a disposable account in https://www.toontownrewritten.com/register.

F. Hauri - Give Up GitHub Over a year ago

You may install perl modules from CPAN.

|

Collectives™ on Stack Overflow

Shell Script CURL JSON value to variable

3 Answers 3

14 Comments

1 Comment

7 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

14 Comments

1 Comment

7 Comments

Your Answer

Sign up or log in

Post as a guest

Related