bash trouble assigning to an array index in a loop

Question

I can get this to work in ksh but not in bash which is really driving me nuts. Hopefully it is something obvious that I'm overlooking.

I need to run an external command where each line of the output will be stored at an array index.

This simplified example looks like it is setting the array in the loop correctly however after the loop has completed those array assignments are gone? It's as though the loop is treated completely as an external shell?

junk.txt

this is a
test to see
if this works ok

testa.sh

#!/bin/bash

declare -i i=0
declare -a array

echo "Simple Test:"
array[0]="hello"
echo "array[0] = ${array[0]}"

echo -e "\nLoop through junk.txt:"
cat junk.txt | while read line
do
    array[i]="$line"
    echo "array[$i] = ${array[i]}"
    let i++
done

echo -e "\nResults:"
echo "       array[0] = ${array[0]}"
echo " Total in array = ${#array[*]}"
echo "The whole array:"
echo ${array[@]}

Output

Simple Test:
array[0] = hello

Loop through junk.txt:
array[0] = this is a
array[1] = test to see
array[2] = if this works ok

Results:
      array[0] = hello
Total in array = 1
The whole array:
hello

So while in the loop, we assign array[i] and the echo verifies it. But after the loop I'm back at array[0] containing "hello" with no other elements.

Same results across bash 3, 4 and different platforms.

chepner · Accepted Answer · 2012-08-13 22:27:41Z

9

Because your while loop is in a pipeline, all variable assignments in the loop body are local to the subshell in which the loop is executed. (I believe ksh does not run the command in a subshell, which is why you have the problem in bash.) Do this instead:

while read line
do
    array[i]="$line"
    echo "array[$i] = ${array[i]}"
    let i++
done < junk.txt

Rarely, if ever, do you want to use cat to pipe a single file to another command; use input redirection instead.

UPDATE: since you need to run from a command and not a file, another option (if available) is process substitution:

while read line; do
...
done < <( command args ... )

If process substitution is not available, you'll need to output to a temporary file and redirect input from that file.

If you are using bash 4.2 or later, you can execute these two commands before your loop, and the original pipe-into-the-loop will work, since the while loop is the last command in the pipeline.

set +m    # Turn off job control; it's probably already off in a non-interactive script
shopt -s lastpipe
cat junk.txt | while read line; do ...; done

UPDATE 2: Here is a loop-less solution based on user1596414's comment

array[0]=hello
IFS=$'\n' array+=( $(command) )

The output of your command is split into words based solely on newlines (so that each line is a separate word), and appends the resulting line-per-slot array to the original. This is very nice if you are only using the loop to build the array. It can also probably be modified to accomodate a small amount of per-line processing, vaguely similar to a Python list comprehension.

edited Aug 13, 2012 at 22:27

answered Aug 13, 2012 at 21:02

chepner

538k77 gold badges594 silver badges746 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

user1596414 Over a year ago

"I need to run an external command where each line of the output will be stored at an array index." The "cat" was a simplified example. I need to run a command and get it's output into the loop and a redirect like < junk.txt doesn't work for that. Other options?

chepner Over a year ago

Updated with a few options. The last one (if your version of bash is new enough) is probably the one you want.

user1596414 Over a year ago

+1 a temporary file or the bash 4.2 (or later) options do work. There is a third option using IFS to handle white spaces and can be combined with a simple array assign if there is no need to iterate through the output as well.

chepner Over a year ago

@user1596414 Very nice observation.

Gordon Davisson Over a year ago

There's some more discussion and options in BashFAQ #24: "I set variables in a loop that's in a pipeline. Why do they disappear after the loop terminates? Or, why can't I pipe data to read?"

Collectives™ on Stack Overflow

bash trouble assigning to an array index in a loop

1 Answer 1

5 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

5 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related