Trouble using sed/regex to remove last character in string

Question

I'm trying to remove a trailing " from all lines in a CSV. I am using RedHat Linux.

I'm able to do it successfully in VIM doing %s/"$//g

However, doing sed -e 's/"$//g' does not work (both in the terminal, and when I have it as part of a bash script). A similar call that I make to remove LEADING " from all the lines, sed -e 's/^"//g' DOES work, so I am extremely confused! Especially since it works in VIM.

Any suggestions?

Thanks!

@AvinashRaj Yup, but I don't have trailing whitespace. I also checked in VIM to see if I have the carriage return right after the ". Definitely weird behavior... — Chris
– Chris, Commented Mar 20, 2015 at 1:24
did you check the contents of the file before running the sed command? — Avinash Raj
– Avinash Raj, Commented Mar 20, 2015 at 1:25

pynexj · Accepted Answer · 2015-03-20 01:34:12Z

3

I suspect your file is in DOS format. Try like this:

$ cat file
line1"
line2"
line3"
$ sed -e 's/"$//' file
line1"
line2"
line3"
$ sed -e $'s/"[\r]\\?$//' file
line1
line2
line3
$

To check if the file is in DOS format, you can:

$ file file
file: ASCII text, with CRLF line terminators
$ head -n 1 file | hexdump -C
00000000  6c 69 6e 65 31 22 0d 0a                   |line1"..|
00000008
$

There are tools like dos2unix (and unix2dos) which you can use to convert it if necessary.

Vim is much smarter in dealing with different format of files so it's not surprised vim can handle it as expected.

edited Mar 20, 2015 at 1:34

answered Mar 20, 2015 at 1:27

pynexj

21.3k6 gold badges45 silver badges63 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Chris Over a year ago

Wow! That did the trick! And normally VIM displays end of lines as ^M, hinting at the \r, so that is interesting.. Thanks :)

Collectives™ on Stack Overflow

Trouble using sed/regex to remove last character in string

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related