0

I have a csv file which includes a bunch of commands in the command line. I can just bash this csv file, but the commands will be executed sequentially in the same terminal. Is there a way to execute each one of these commands in a separate terminal, so that they are executed in parallel?

Example:

It's like:

python args1
python args2
...
python argsn
10
  • You can put each command in the background? There's also this stackoverflow.com/questions/4404242/… Commented Oct 16, 2017 at 16:27
  • Here is another question to check: stackoverflow.com/questions/989349/… Commented Oct 16, 2017 at 16:28
  • What exactly does your CSV file look like? In general, you should not be able to simply execute it, as the field delimiters shouldn't be considered part of the arguments. Commented Oct 16, 2017 at 16:50
  • 1
    How is that a CSV file? Commented Oct 16, 2017 at 18:05
  • Huh? Why would you have bash commands mixed in with a csv? Commented Oct 16, 2017 at 18:05

3 Answers 3

2

If you want to do any serious software development on a Mac, I would suggest you install homebrew because Apple doesn't ship a package manager, but does ship ancient versions of all the tools most people use... Python, sed, PHP, Perl, awk, find, grep, make, git...

Once you have homebrew, I would recommend GNU Parallel which you can install with:

brew install parallel

Once you have GNU Parallel, you can run your commands in parallel with:

parallel --dry-run -a commands.csv

Or, maybe you would like the lines tagged with their names:

parallel --tag -a commands.csv

If you would like 8 to run at a time, add -j 8. If you want an Estimated Time of Arrival (when they should complete) add --eta and so on.


By the way, you can look for other tools, like Intel TBB, with:

brew search tbb

or pango, with:

brew search pango
Sign up to request clarification or add additional context in comments.

4 Comments

My pleasure. Good luck with your project and remember questions, and answers, are free on StackOverflow, so come back if you get stuck.
It should show you the commands it would execute. If they look correct, you should run again but remove the --dry-run
The output may be buffered. Do the jobs take long?
You can try adding --line-buffer or --ungroup to get output sooner. You can also add --job-log YourLog.txt to get a log file.
2

You basically just have a shell script already. The only difference is that instead of running it as one script, you want to run each line as a separate command. How you open a new terminal is somewhat OS-dependent, but let's assume you would just use the xterm command. The following will treat your script commands.csv (or whatever it is named) as a data file.

while IFS= read -r cmdline; do
    xterm -e sh -c "$cmdline" &
done < commands.csv

3 Comments

(Just noticed you tagged the question osx; checking how you would use the standard Terminal app instead of xterm.)
Yup. Depending on the line in the file, though, there are going to be some hairy quoting issues to work out.
Thank you for the answer, I'll try to implement it.
-1

You can run commands in the background like

run-command &
run-some-other-command &
etc &

is that something you can use?

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.