90

I'm curious as to why the backspace is necessary when setting IFS to split on newlines like this:

IFS=$(echo -en "\n\b")

Why can I not just use this (which doesn't work) instead?

IFS=$(echo -en "\n")

I'm on a Linux system that is saving files with Unix line endings. I've converted my file with newlines to hex and it definitely only uses "0a" as the newline character.

I've googled a lot and although many pages document the newline followed by backspace solution, none that I have found explain why the backspace is required.

-David.

Improve this question
4
  • Could somebody please say where do they see such code? Thank you. Commented May 31, 2016 at 12:42
  • @spbnick You need to reset the IFS whenever you want to iterate over files, e.g. in for file in $(ls) ; do echo "$f" ; done. Without setting the IFS to only newlines, the for loop will echo each space-separated hunk in each file name individually instead of the whole filename. Commented May 17, 2019 at 17:11
  • Thanks @bleistift2, I understand why somebody would want to change IFS and I understand how the code works (my answer is the most popular here). I'm just curious where this particular use of backspace character in IFS comes from, and who came up with it, historically. Hence I'm asking where people have seen this kind of code. Commented May 20, 2019 at 12:12
  • @spbnick Darn good question, and I'd love to know, too. It is a rather mind-bogglingly obscure and misleading invocation. And pointless! If you want to set, say, shell variable X to "a", you just say X="a"; you don't say X=$(echo "a"). And that's before we get to the part that here we're actually setting IFS to something that will additionally split on a backspace, if the input happens to contain one! If the question is "Why?", yours is the right answer, but if the question is "What's a good way to set IFS to a newline?", I have to commend moddie's answer. Commented Mar 26, 2022 at 11:38

6 Answers 6

Reset to default
185

Because as bash manual says regarding command substitution:

Bash performs the expansion by executing command and replacing the command substitution with the standard output of the command, with any trailing newlines deleted.

So, by adding \b you prevent removal of \n.

A cleaner way to do this could be to use $'' quoting, like this:

IFS=$'\n'
Improve this answer
Sign up to request clarification or add additional context in comments.

3 Comments

$'...' is targeted for inclusion in POSIX. [mywiki.wooledge.org/Bashism] [austingroupbugs.net/view.php?id=249]
While these are supposed to be included in Posix at some point, they currently aren't and this won't work with /bin/sh in many places which links against a posix shell, not bash.
But when I want to split the array using newline and read command. Only the first item is getting added to the hostArray. But the host has 4 items. host=$(mysql -S $socket $catalog --skip-column-names -e "select host from mysql.user); echo "Host: $host"; IFS=$'\n';read -ra hostArray <<< "$host"; echo "HostArray:$hostArray" #This prints only 1 item. But there are 4 items; What is wrong here?
66

I just remembered the easiest way. Tested with bash on debian wheezy.

IFS="
"

no kidding :)

Improve this answer

4 Comments

but it's not a one liner 😛 -- specially if you need to restore the old $IFS, as in: stackoverflow.com/questions/1406966/…
Man, you made my day! No kidding and works in posix also.
@juanmirocks Why does this solution make it any harder to restore the old IFS afterwards?
@SteveSummit I've not tested it right now. But I assume you will have to quote the "oldIFS" well to avoid any pitfalls.
14

It's a hack because of the use of echo and command substitution.

prompt> x=$(echo -en "\n")
prompt> echo ${#x}
0
prompt> x=$(echo -en "\n\b")
prompt> echo ${#x}
2

The $() strips trailing newlines and \b prevents \n from being a trailing newline while being highly unlikely to appear in any text. IFS=$'\n' is the better way to set IFS to split on newlines.

Improve this answer

1 Comment

This demonstrates that the \n\b formation also includes backspace as an IFS delimiter, so I'd consider that broken in some circumstances.
11

The \b char as a suffix of newline \n is added due to removal of the tailing \n in the command substitution $(...). So the \b is used as a suffix to \n, and by that \n is no longer trailing, so it is returned from the the command substitution.

The side effect is, that IFS also will include the \b char as a separator as well, instead of just \n, which really is our sole interest.

If you expect \b may someday appear in the string (why not?), then you may use:

IFS="$(printf '\nx')" && IFS="${IFS%x}";

that returns \n suffixed with x && removes x

now IFS contains only the \n char.

IFS="$(printf '\nx')" && IFS="${IFS%x}";
echo ${#IFS}; # 1

and nothing will break in case of \b, test:

#!/bin/sh

sentence=$(printf "Foo\nBar\tBaz Maz\bTaz");
IFS="$(printf '\nx')" && IFS="${IFS%x}";

for entry in $sentence
do
    printf "Entry: ${entry}.\n";
done

gives two lines (due to one \n):

Entry: Foo.
Entry: Bar      Baz Maz Taz.

as expected.

IFS="$(printf '\nx')" && IFS="${IFS%x}"; using:

IFS="
"

gives the same result, but these two lines must not be indented, and if you accidentally put space or tab or any other white char between " and ", you'll no longer have only the \n char there, but some "bonuses" as well. This bug will be hard to spot, unless you use the option "show all characters" in your editor.

Improve this answer

4 Comments

In POSIX compliant scripts, using printf instead of echo is mandatory, and using IFS=$(printf '\nx'); IFS=${IFS%?} instead of IFS="<literal newline>" is more clear IMHO
Mine keeps the newline in the second var isn't it supposed to strip it out?
@JPM Which example exactly do you mean?
Only answers that works for me!
0

I always avoid anything that interrupts indentation, such as the literal newline highlighted by @moddie. @moddie's answer has universal portability and the least overhead, whereas this answer can be considered equally as portable in a practical sense, and has a negligible increase in overhead.

This should work in almost any unix-like shell that supports $() command substitution.

IFS="$(printf "%s" "$(printf "\nx")" | sed 's/^\(.\).*/\1/')"

This should work in every Bourne Shell, including the first release of Bourne Shell on Unix 7.

IFS="`printf "\nx"`"
IFS="`printf "%s" "${IFS}" | sed 's/^\(.\).*/\1/'`"
Improve this answer

Comments

-2

Just pressing enter without assigning anything also works. Although, it looks like someone made a mistake and difficult to understand.

IFS=
#This is a line
Improve this answer

3 Comments

Doesn't this set $IFS to an empty string?
Well, you press enter after the '=' sign for it to work. It worked when i tried. @moddie's answer looks more clear.
This is incorrect. It is functionally equivalent to IFS="", which is not what was asked for. IFS= creates a zero-length string, whereas IFS=$'\n' creates a string of length 1 character. You can verify with echo ${#IFS}.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.