Bash Variable Contains '\r' - Carriage Return Causing Problems in my Script

Question

I have a bash script (rsync.sh) that works fine and has this line in it:

python /path/to/rsync_script.py $EMAIL "$RSYNC $PATH1 $PATH1_BACKUP"

I want to break the command (it's actually much longer than shown here because my variables have longer names) in two and use something like this:

python /path/to/rsync_script.py \
$EMAIL "$RSYNC $PATH1 $PATH1_BACKUP"

But when I do this I get the error:

scripts/rsync.sh: line 32: $'[email protected]\r': command not found

It puts the carriage return, \r in there.

How can I break this line up and not include the carriage return?

Well, no, I'm on a Windows 7 machine but I am ssh-ed (using putty) into a Linux machine and I made the script using vim. — nicorellius
– nicorellius, Commented Dec 8, 2011 at 0:58
Hrm, that \r still feels out of place. If you don't have dos2unix or dtox already installed, re-edit the file with vim, and run :set ff=unix immediately before :wq. — sarnold
– sarnold, Commented Dec 8, 2011 at 1:01
also, can your double check that the previous line doesn't have space or other junk trailing the ` \ ` . We all seem to focus on the ` \r ` but not notice that bash complains that its a command that it cannot find. — nhed
– nhed, Commented Dec 8, 2011 at 1:14

Brigand · Accepted Answer · 2011-12-08 02:57:05Z

2

The problem looks like Windows line endings.

Here's how you can check in Python.

repr(open('rsync.sh', 'rb').read())
# If you see any \\r\\n, it's windows

Here's how you can fix it:

text = open('rsync.sh', 'r').read().replace('\r\n', '\n')
open('rsync.sh', 'wb').write(text)

Edit

Here's some code that shows the problem.

# Python:
open('abc-n.sh', 'wb').write('echo abc \\' + '\n' + 'def')
open('abc-r-n.sh', 'wb').write('echo abc \\' + '\r\n' + 'def')

And then run the files we made...

$ sh abc-n.sh 
abc def
$ sh abc-r-n.sh 
abc 
abc-r-n.sh: 2: def: not found

edited Dec 8, 2011 at 2:57

answered Dec 8, 2011 at 1:08

Brigand

86.4k20 gold badges167 silver badges174 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

Brigand Over a year ago

I think the source code has incorrect line endings. I'm providing a way to correctly format the line endings. The code above is meant to be run one time on the existing file. I'll change the file names in the example to 'rsync.sh'.

Brigand Over a year ago

I added an example that makes a 'good' file and one that makes a 'bad' file. The 'bad' gives a similar error as the question. My original code converts the 'bad' to 'good'.

Brigand Over a year ago

I apologize for the many edits. I believe everything is correct now.

nicorellius Over a year ago

I think I may have, at some point, copied and/or pasted to and from the script in vim, from Windows. This could have very well added incorrect line endings. I will try an experiment with creating the bash script in vim with a new name, typing all of it in Linux to see if that's the case. Thanks for all the tips!

jsbueno · Accepted Answer · 2011-12-09 11:41:48Z

1

If you can chnage the python script, maybe it will be easier to pass it the variable names thenselves, instead of their content.

From within the Python code you w=have better and more consistent tools to deal with whitespace characters (like \r) than from within bash.

To do that, just change your .sh line to

python /path/to/rsync_script.py EMAIL "RSYNC PATH1 PATH1_BACKUP"

And on your rsync_script.py, use os.environ to read the contents of the shell variables (and clear the \r's in them) - something like:

import os, sys
paths = []
for var_name in sys.argv(2).split(" "):
     paths.append(os.environ[var_name].strip())

edited Dec 9, 2011 at 11:41

answered Dec 8, 2011 at 2:28

jsbueno

113k11 gold badges159 silver badges239 bronze badges

2 Comments

nicorellius Over a year ago

The programs run fine unless I break the line in the shell script. Using the line you mention, python /path/to/rsync_script.py $EMAIL "$RSYNC $PATH1 $PATH1_BACKUP" is already what I had (see original post)... This is the way I make it work now.

jsbueno Over a year ago

Sorry - I typed it incorrectly here - I meant passing the variable names, without the $, not their contents. Fixing now.

nicorellius · Accepted Answer · 2011-12-10 00:27:24Z

0

So I figured it out... I made a mistake in this question and I got so much awesome help but it was me doing a dumb thing that caused the problem. As I mentioned above, I may have copied and pasted from Windows at some point (I had forgotten since I did most of the edits in vim). I went back and wrote a short script with the essentials of the original in vim and then added in the '\' for line break and the script worked just fine. I feel bad accepting my own answer since it was so stupid. I made sure to up-vote everyone who helped me. Thanks again.

answered Dec 10, 2011 at 0:27

nicorellius

4,1635 gold badges56 silver badges81 bronze badges

Collectives™ on Stack Overflow

Bash Variable Contains '\r' - Carriage Return Causing Problems in my Script

3 Answers 3

Edit

4 Comments

2 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Edit

4 Comments

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related