Running sequential commands through ssh on multiple servers

Question

I have a list of servers in a file that I pass to the following script:

#!/bin/bash

while IFS='' read -r line || [[ -n "$line" ]]; do

    ssh_cmd="user@$line"
    ssh $ssh_cmd 'hostname && ls -l /path/to/folder | grep some_token' &

done < "$1"

What I want is output like this:

  output from hostname (1)
  output from ls -l (1)
  output from hostname (2)
  output from ls -l (2)
  output from hostname (3)
  output from ls -l (3)

But these seem to be mixed up:

  output from ls -l (1)
  output from hostname (1)      
  output from hostname (2)
  output from ls -l (2)
  output from ls -l (3)
  output from hostname (3)

Is there some way to ensure that these commands happen sequentially?

Thanks in adv!

Maximillian Laumeister · Accepted Answer · 2015-08-05 04:56:19Z

1

The & at the end of this line makes it so the process runs in the background:

ssh $ssh_cmd 'hostname && ls -l /path/to/folder | grep some_token' &

If you remove the & from that line, the script will wait until the ssh session finishes to continue on to the next iteration of the loop. Also, you will need to add the -n flag to the ssh command so that it doesn't read from stdin. This will give you the sequential output you're looking for.

Final command:

ssh -n $ssh_cmd 'hostname && ls -l /path/to/folder | grep some_token'

edited Aug 5, 2015 at 4:56

answered Aug 5, 2015 at 3:32

Maximillian Laumeister

20.5k10 gold badges62 silver badges84 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Pat Mustard Over a year ago

Thanks. When I remove the trailing & the script only executed for the 1st item in the list. Is ssh somehow eating the rest of the input?

Pat Mustard Over a year ago

The reason my loop wasn't completing was that ssh was eating up the rest of the stdin stream. This was fixes by adding the -n flag to my ssh command: ssh -n user@server command1 && command2

Maximillian Laumeister Over a year ago

@PatMustard Great! Sorry I didn't think of that, glad you got it fixed! I'll edit it into my answer for future readers.

Mark Setchell · Accepted Answer · 2015-08-05 08:12:46Z

Use GNU Parallel to do them all nicely in parallel and use the -k option to keep the output in order:

parallel -k -a hosts.txt 'echo ssh user@{} "hostname; ls"'

So, if hosts.txt contains the following:

host1
host27
host32
host100
hostfreddyfrog
hostmichaelfish

you will get this:

ssh user@host1 hostname; ls
ssh user@host27 hostname; ls
ssh user@host32 hostname; ls
ssh user@host100 hostname; ls
ssh user@hostfreddyfrog hostname; ls
ssh user@hostmichaelfish hostname; ls

Obvioulsy you need to remove the echo and add in your quoting and gripping, but the concept should be clear enough.

Another advantage, other than it now being a one-liner, is that if you have 32,000 hosts to work on, GNU Parallel can be told, using the -j flag, how many processes to run at a time without overloading your server. By default, if you have 8 cores in your CPU, it will run 8 jobs in parallel, but you could set it to 64 for example with

parallel -k -j 64 ...

You can also set up GNU Parallel to be able to ssh into hosts itself, normally in order to distribute processing jobs amongst servers, then the job becomes even easier.

parallel -S node1,node4,node56 'hostname; grep ...'

See example here.

ALternative syntaxes for passing hosts.txt are

parallel -k 'echo ssh user@{} "hostname; ls"' < hosts.txt

or

parallel -k 'echo ssh user@{} "hostname; ls"' :::: hosts.txt

Collectives™ on Stack Overflow

Running sequential commands through ssh on multiple servers

2 Answers 2

3 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

3 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related