242

I need to transfer a log file to a remote host using sftp from a Linux host. I have been provided credentials for the same from my operations group. However, since I don't have control over other host, I cannot generate and share RSA keys with the other host.

So is there a way to run the sftp command (with the username/password provided) from inside the Bash script through a cron job?

I found a similar Stack Overflow question, Specify password to sftp in a Bash script, but there was no satisfactory answer to my problem.

0

16 Answers 16

241

You have a few options other than using public key authentication:

  1. Use keychain
  2. Use sshpass (less secured but probably that meets your requirement)
  3. Use expect (least secured and more coding needed)

If you decide to give sshpass a chance here is a working script snippet to do so:

export SSHPASS=your-password-here
sshpass -e sftp -oBatchMode=no -b - sftp-user@remote-host << !
   cd incoming
   put your-log-file.log
   bye
!

Update: However do understand that using environment variables is also insecure as using command line option -p for passing password.

It is better to store and read password from a file like this using -f option:

echo 'your-password-here' > ~/.passwd
chmod 0400 ~/.passwd

sshpass -f ~/.passwd -e sftp -oBatchMode=no -b - sftp-user@remote-host << !
   cd incoming
   put your-log-file.log
   bye
!
Sign up to request clarification or add additional context in comments.

28 Comments

No need to export, I think. SSHPASS=password sshpass -e ... should do.
I was able to do a one-liner to dowload a log file: sshpass -p "my_password" sftp -oPort=9999 user@host:dir/file.log
@gorus: One-liner is very cool however I didn't want to use password on command line because of enhanced snooping risks.
-oBatchMode=no was the crucial missing piece for me.
|
141

Another way would be to use lftp:

lftp sftp://user:password@host  -e "put local-file.name; bye"

The disadvantage of this method is that other users on the computer can read the password from tools like ps and that the password can become part of your shell history.

A more secure alternative which is available since LFTP 4.5.0 is setting the LFTP_PASSWORD environment variable and executing lftp with --env-password. Here's a full example:

export LFTP_PASSWORD="just_an_example"
lftp --env-password sftp://user@host  -e "put local-file.name; bye"

# Destroy password after use
export LFTP_PASSWORD=""

LFTP also includes a cool mirroring feature (can include delete after confirmed transfer --Remove-source-files):

lftp -e 'mirror -R /local/log/path/ /remote/path/' --env-password -u user sftp.foo.com

10 Comments

+1 for suggesting an alternative but can we avoid supplying password on command line?
Downvoted for showing the password to anyone on the same computer
You can create the script file for lftp, this way you will not have to provide the password in the command line. Create a file put-script: open sftp://user:password@host; put local-file.name; exit Than run lftp -f put-script This way you do not have to have the username and password in a command line and can set up restrictive permissions to your script file.
Far easier to do lftp than mess around with sftp and sshpass. Good answer.
@AaronDigulla Upvoted because it is exactly what the OP asked for - "a way to run the sftp command (with the username / password provided) from inside the Bash script".
|
70

EXPECT is a great program to use.

On Ubuntu install it with:

sudo apt-get install expect

On a CentOS Machine install it with:

yum install expect

Lets say you want to make a connection to a sftp server and then upload a local file from your local machine to the remote sftp server

#!/usr/bin/expect

spawn sftp [email protected]
expect "password:"
send "yourpasswordhere\n"
expect "sftp>"
send "cd logdirectory\n"
expect "sftp>"
send "put /var/log/file.log\n"
expect "sftp>"
send "exit\n"
interact

This opens a sftp connection with your password to the server.

Then it goes to the directory where you want to upload your file, in this case "logdirectory"

This uploads a log file from the local directory found at /var/log/ with the files name being file.log to the "logdirectory" on the remote server

6 Comments

Thanks for your answer, I had expect as one of my options for achieving this task.
Thank you. I found your answer through a search, used it and built a script successfully with expect. Just posted the script for poterity
Thank you so much for you answer, I have already upvoted it :)
I'm using this to enter the passphrase for ssh private key. It is handier to put same public key on all servers and just rotate the private key passphrase periodically.
I get - spawn: command not found
|
30

You can use lftp interactively in a shell script so the password not saved in .bash_history or similar by doing the following:

vi test_script.sh

Add the following to your file:

#!/bin/sh
HOST=<yourhostname>
USER=<someusername>
PASSWD=<yourpasswd>

cd <base directory for your put file>

lftp<<END_SCRIPT
open sftp://$HOST
user $USER $PASSWD
put local-file.name
bye
END_SCRIPT

And write/quit the vi editor after you edit the host, user, pass, and directory for your put file typing :wq .Then make your script executable chmod +x test_script.sh and execute it ./test_script.sh.

1 Comment

To overwrite existing file, add "set xfer:clobber on" after lftp<<END_SCRIPT
19

The easiest way I found to accomplish this, without installing any third-party library like Expect, SSHPASS...etc, is by using a combination of CURL, and SFTP. Those two are almost in every Linux machine.

This is the command you should execute, after changing the values.

curl  -k "sftp://SERVER_IP:SERVER_PORT/FULL_PATH_OF_THE_FILE" --user "SERVER_USER:SERVER_PASSOWRD" -o "THE_NAME_OF_THE_FILE_AFTER_DOWNLOADING_IT"

Example:

curl  -k "sftp://10.10.10.10:77/home/admin/test.txt" --user "admin:123456" -o "test.txt"

Explanation:

We are connecting to the server 10.10.10.10:77 using the username admin and password 123456, to move the file /home/admin/test.txt from that server to the server you are using currently to execute the above command.

4 Comments

isn't this the opposite direction? this shows how to copy a file from the remote server to the local, when the question is to upload from local to remote.
Very clever solution that I've never seen before... no dependencies.
Showing error: "sftp" not supported or disabled in libcurl
This makes me think of how simple it would be to add to my .bashrc
18

I was recently asked to switch over from ftp to sftp, in order to secure the file transmission between servers. We are using Tectia SSH package, which has an option --password to pass the password on the command line.

example : sftp --password="password" "userid"@"servername"

Batch example :

(
  echo "
  ascii
  cd pub
  lcd dir_name
  put filename
  close
  quit
    "
) | sftp --password="password" "userid"@"servername"

I thought I should share this information, since I was looking at various websites, before running the help command (sftp -h), and was i surprised to see the password option.

4 Comments

+1 for you good suggestion. However this will require you to pass the password on command line and anybody on the system doing ps command would be able to see your password.
I tried that, I got unknown option -- - usage: sftp [-1246aCfpqrv] [-B buffer_size] [-b batchfile] [-c cipher] [-D sftp_server_path] [-F ssh_config] [-i identity_file] [-l limit] [-o ssh_option] [-P port] [-R num_requests] [-S program] [-s subsystem | sftp_server] host sftp [user@]host[:file ...] sftp [user@]host[:dir[/]] sftp -b batchfile [user@]host
@ihue probably because you need Techia SSH, not OpenSSH
Yeah this doesn't work with openssh, not a good solution.
18

You can override by enabling Password less authentication. But you should install keys (pub, priv) before going for that.

Execute the following commands at local server.

Local $> ssh-keygen -t rsa 

Press ENTER for all options prompted. No values need to be typed.

Local $> cd .ssh
Local $> scp .ssh/id_rsa.pub user@targetmachine:
Prompts for pwd$>  ENTERPASSWORD

Connect to remote server using the following command

Local $> ssh user@targetmachine
Prompts for pwd$> ENTERPASSWORD

Execute the following commands at remote server

Remote $> mkdir .ssh
Remote $> chmod 700 .ssh
Remote $> cat id_rsa.pub >> .ssh/authorized_keys
Remote $> chmod 600 .ssh/authorized_keys
Remote $> exit

Execute the following command at local server to test password-less authentication. It should be connected without password.

$> ssh user@targetmachine

5 Comments

+1 for your answer but question was about doing it without public/private keys.
oops.. Just saw the restrictions on key sharing :)
Out of curiosity will this command lftp -p ${port} -u ${login_id},${password} ${ip_number} when invoked from a shell script print them some where? How did you solve apart from sshpass?
I don't have lftp and rely on sshpass from Mac and Linux both.
Tip: If you are setting this up for additional remote servers, copy the same id_rsa.pub to the remote servers. The public key id_rsa.pub is generated once on the local server and shared with all remote hosts with which password less authentication needs to be set up.
8

Combine sshpass with a locked-down credentials file and, in practice, it's as secure as anything - if you've got root on the box to read the credentials file, all bets are off anyway.

Comments

7

Bash program to wait for sftp to ask for a password then send it along:

#!/bin/bash
expect -c "
spawn sftp username@your_host
expect \"Password\"
send \"your_password_here\r\"
interact "

You may need to install expect, change the wording of 'Password' to lowercase 'p' to match what your prompt receives. The problems here is that it exposes your password in plain text in the file as well as in the command history. Which nearly defeats the purpose of having a password in the first place.

1 Comment

+1 but I think you state it clearly what the problems are with this approach.
5

You can use sshpass for it. Below are the steps

  1. Install sshpass For Ubuntu - sudo apt-get install sshpass
  2. Add the Remote IP to your known-host file if it is first time For Ubuntu -> ssh user@IP -> enter 'yes'
  3. give a combined command of scp and sshpass for it. Below is a sample code for war coping to remote tomcat sshpass -p '#Password_For_remote_machine' scp /home/ubuntu/latest_build/abc.war #user@#RemoteIP:/var/lib/tomcat7/webapps

Comments

2

A few people have mentioned sshpass but not many clear coding examples...

This is how we are doing it with bash scripts for rsync backups:

sshpass -p "${RSYNC_PASSWORD}" sftp "${RSYNC_USER}"@"${RSYNC_REMOTE_HOST}"

Keep in mind you will have to sudo apt install sshpass before this works properly.

3 Comments

Clear (and better) example is given by the very first (most-upvoted) answer by @anubhava.
@MartinPrikryl His answer uses export and several other flags that are not required and in fact not even a good idea to use...
If you believe that using -p is better than SSHPASS (I do not agree), please explain why in your answer.
2
curl --netrc-file NETRC --upload-file LOCALPATH sftp://HOST:PORT/REMOTEPATH

LOCALPATH may be - for stdin. The NETRC file has lines with format:

machine HOST login USERNAME password PASSWORD

(It's also possible to use --user USER:PASSWORD instead of --netrc-file but that reveals the password to anyone running ps locally at the same time, so don't do that.)

If the NETRC file does not exist permanently, your script can create a temporary file (possibly with the help of mktemp).

Thanks to @ASammour for pointing out that curl speaks SFTP.

Comments

1

or without prior installing of sshpass and the advantage of encoding the password (e.g. with base64 or gpg) (i use this in CICD).

export SSH_ASKPASS_REQUIRE=force
export SSH_ASKPASS=/usr/local/bin/ssh_askpass_helper_env

sudo tee <<'EOF' /usr/local/bin/ssh_askpass_helper_env
#!/bin/bash

base64 -d <<<"${SSH_PW}"
EOF

sudo chmod +x /usr/local/bin/ssh_askpass_helper_env

sftp \
  -C \
  -o BatchMode=no \
  -o PasswordAuthentication=yes \
  -o PubkeyAuthentication=no \
  -o PreferredAuthentications=password \
  -o StrictHostKeyChecking=accept-new \
  -b - \
  -- \
  kitchen@your_mum \
  <<<$'ls\nbye'

Comments

0

You can use a Python script with scp and os library to make a system call.

  1. ssh-keygen -t rsa -b 2048 (local machine)
  2. ssh-copy-id user@remote_server_address
  3. create a Python script like:
    import os
    cmd = 'scp user@remote_server_address:remote_file_path local_file_path'
    os.system(cmd)
  1. create a rule in crontab to automate your script
  2. done

2 Comments

The question is about using a password authentication, not a public key authentication.
Yes, you are right, but can be useful for other ones looking for alternatives for their problems. Because the issue is related.
0

@ASamour inspired me to put this in my .bashrc:

sshpass()
{
        server=$1
        file=$2
        userpass=$3
        outfile=${4:-$file}

        curl \
                --insecure sftp://${server}/${file} \
                --user "${userpass}" \
                --output ${outfile}
}

2 Comments

Why do you need —insecure option?
I have no idea. I just expanded the -k switch that @ASamour used.
-1

I came here only to get the auth trick, and then run the commands manually in the terminal. This solution is ONLY ABOUT the authentication part:

I had to install sshpass.

sshpass -p thepassword sftp [email protected]

Then you will be logged on sftp to run commands manually, without having to type the password.

Comments

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.