Passing individual lines from files into a python script using a bash script

Question

This might be a simple question, but I am new to bash scripting and have spent quite a bit of time on this with no luck; I hope I can get an answer here.

I am trying to write a bash script that reads individual lines from a text file and passes them along as argument for a python script. I have a list of files (which I have saved into a single text file, all on individual lines) that I need to be used as arguments in my python script, and I would like to use a bash script to send them all through. Of course I can take the tedious way and copy/paste the rest of the python command to individual lines in the script, but I would think there is a way to do this with the "read line" command. I have tried all sorts of combinations of commands, but here is the most recent one I have:

#!/bin/bash
# Command Output Test

cat infile.txt << EOF
while read line
do
    VALUE = $line
    python fits_edit_head.py $line $line NEW_PARA 5
    echo VALUE+"huh"
done

EOF

When I do this, all I get returned is the individual lines from the input file. I have the extra VALUE there to see if it will print that, but it does not. Clearly there is something simple about the "read line" command that I do not understand but after messing with it for quite a long time, I do not know what it is. I admit I am still a rookie to this bash scripting game, and not a very good one at that. Any help would certainly be appreciated.

konsolebox · Accepted Answer · 2014-07-11 04:48:39Z

1

You probably meant:

while read line; do
    VALUE=$line   ## No spaces allowed
    python fits_edit_head.py "$line" "$line" NEW_PARA 5  ## Quote properly to isolate arguments well
    echo "$VALUE+huh"  ## You don't expand without $
done < infile.txt

Python may also read STDIN so that it could accidentally read input from infile.txt so you can use another file descriptor:

while read -u 4 line; do
     ...
done 4< infile.txt

Better yet if you're using Bash 4.0, it's safer and cleaner to use readarray:

readarray -t lines < infile.txt
for line in "${lines[@]}; do
    ...
done

answered Jul 11, 2014 at 4:48

konsolebox

76.3k13 gold badges110 silver badges114 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

JSLifeform Over a year ago

The first version you put up worked like a charm; I also tried the last one and got some issues, but that is something I can work out on my own after learning more about arrays. Out of curiosity, what do you think hosed me more: the spaces between VALUE and $line or the lack of a semi-colon on the first line? Or was it a conjunctive issue?

konsolebox Over a year ago

The real problem was that you did cat infile.txt << EOF and placed the commands within that here document block. VALUE = $line was just a syntax error problem.

JSLifeform Over a year ago

Also, after removing the section that prints the value of each line, it still prints the name of each file before processing them. Not a big issue, but I would still like to know how to stop it from doing that. Does that have to do with the "while read line" command? Is there a way to input each line's data into the script without printing it onto the screen as well?

konsolebox Over a year ago

@user3827679 No other part of the script prints a message besides echo "$VALUE+huh". If it's the one you removed then it's likely that the python script fits_edit_head.py is already the one that is printing the names and should be examined.

JSLifeform Over a year ago

Answers much appreciated, sir.

Collectives™ on Stack Overflow

Passing individual lines from files into a python script using a bash script

1 Answer 1

5 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

5 Comments

Your Answer

Sign up or log in

Post as a guest

Related