How can I run a Bash command for every JSON object in a JSON array using jq? So far I have this:
cat credentials.json | jq -r '.[] | .user, .date, .email' | mycommand -u {user} -d {date} -e {email}
This doesn't seem to work. How can I take the parameters out of the JSON array into my command?
My JSON file looks something like this:
[
"user": "danielrvt",
"date": "11/10/1988",
"email": "[email protected]",
...
]
Your best bet is probably to output each record in something like TSV format, then read that from a shell loop.
jq -r '.[]|[.user, .date, .email] | @tsv' |
while IFS=$'\t' read -r user date email; do
mycommand -u "$user" -d "$date" -e "$email"
done
jq itself doesn't have anything like a system call to run an external command from within a filter, although it seems that they are working on it.
IFS=$'\t' is not working when I use this in Makefile. Can you suggest any other way?
make is probably using /bin/sh, not bash, and it doesn't support $'...'-style quoting.With xargs:
curl localhost:8082/connectors | jq .[] | xargs -L1 -I'{}' curl -XDELETE 'localhost:8082/connectors/{}'
Or equivalently, to show the output of that first curl:
echo '["quickstart-file-sink4","quickstart-file-source","quickstart-file-sink","quickstart-file-sink2","quickstart-file-sink3","quickstart-file-source2"]' | jq .[] | xargs -L1 -I'{}' curl -XDELETE 'localhost:8082/connectors/{}'
jq .[] strips off one level of containment, so that a list becomes output as one line per item.
xargs -L1 processes one line at a time
xargs -I'{}' specifies that the string {} be replaced with the input line when invoking the following command.
xargs is essentially a map operator for the shell.
xargs is a great command.
xargs command as shown in the answer, but this answer definitely got me headed down the right path! For those interested, I ended up replacing the arguments with the more verbose config flag versions, with the equivalent for this question being curl localhost:8082/connectors | jq .[] | xargs --max-lines=1 --replace={} curl -XDELETE 'localhost:8082/connectors/{}' (I also used jq's -r flag before chaining).
jq .[] should actually be jq '.[]'You could have jq output the commands to execute, something like
.[] | "mycommand \(.user|@sh) \(.date|@sh) \(.email|@sh)"
Then execute it. Something like
bash <(jq -r '.[] | "mycommand \(.user|@sh) \(.date|@sh) \(.email|@sh)"' foo)
{"email": "daniel;rm -rf *", ...}Here is another variation which I based on the answer from @chepner.
echo "$config" | jq -c '.[]' |
while IFS=$"\n" read -r c; do
echo "start"
host=$(echo "$c" | jq -r '.host')
echo $host
echo "end"
done
I used jq's -c option to output "compact" jsons, so they are all on one line.
In combination with IFS=$"\n", I was able to loop over each item in the input json's array and do what I wanted to do.
So, with an input of
[
{
"host": "host1",
"settings": {}
},
{
"host": "host2",
"settings": {}
}
]
the output is
start
host1
end
start
host2
end
I came across the same problem recently where xargs doesn't help that much due to the relatively complicated set of arguments I wanted to pass around. Thus I implemented an sh filter (and its friends) to jq. I haven't yet had enough time to write documentation and tests for it so not creating a PR for it to become a part of the official codebase yet. So now it's only for the ones who are willing to compile this version themselves:
src/builtin.c is repetitive and needs to be simplified, but I think you should open a pull request against stedolan/jq.