Run python commands in parallel in Linux shell scripting

Question

I have a script which gets input from the user through command line arguments. It processes the arguments and starts running python commands. For Example:

./run.sh p1 p2 p3 p4
python abc.py p1 p4
python xyz.py p2 p3

where p1, p2, p3 and p4 can be of any type.

I need to run both of these python commands in parallel and in two different terminals. How can I do it so that I don't need to wait for 1 command to finish in order to start the next command?

I tried GNU parallel but it doesn't seem to work.

I worked on Python project that used parallelism for their hydrodynamic modeling and they utilized OpenMPI. If you want to see how they implemented it you might find something on their github: github.com/GeoscienceAustralia/anuga_core — Usagi
– Usagi, Commented Sep 30, 2016 at 20:01
There is a wonderful answer here: stackoverflow.com/a/10909842/5381704 — Bishwas Mishra
– Bishwas Mishra, Commented Sep 30, 2016 at 20:05

Alex Lamson · Accepted Answer · 2019-09-11 14:21:44Z

3

Try running each process in the background by adding & to the end of the command.

python script1.py arg1 arg2 &
python script2.py arg1 arg2 &

echo "Running scripts in parallel"
wait # This will wait until both scripts finish
echo "Script done running"

More Info

edited Sep 11, 2019 at 14:21

Alex Lamson

5656 silver badges16 bronze badges

answered Sep 30, 2016 at 19:52

pferate

2,1271 gold badge16 silver badges18 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Ole Tange Over a year ago

Remember a final "wait"

Ole Tange · Accepted Answer · 2016-10-01 08:02:32Z

0

Your task is not where GNU Parallel excels, but it can be done:

parallel ::: "python abc.py p1 p4" "python xyz.py p2 p3"

answered Oct 1, 2016 at 8:02

Ole Tange

34.1k9 gold badges93 silver badges111 bronze badges

Collectives™ on Stack Overflow

Run python commands in parallel in Linux shell scripting

2 Answers 2

1 Comment

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related