0

I'm writing a script to parse a larger file into usable data. I will attach the script below. The script was not running as I wanted and I thought it might have been the if/then statement I used, but when I took that off it revealed the real problem. This is now the error I am getting which makes it seem like the real problem is a quote mark of some kind.

FirstName,LastName,Company,Address,City,County,State,ZIP,Phone,Fax,Email,Web
FinalProject.sh: 36: FinalProject.sh: Syntax error: Unterminated quoted string

While I technically understand what the syntax error means, I cannot find where the problem is, and I have been trying to fix it for about half an hour. Help would be greatly appreciated!

#!/bin/bash

OLDIFS=${IFS}
IFS=$'\n'

head -1 50000_a.csv | sed 's/"//g'
for LINE in 'grep WV 50000_a.csv | sed 's/"//g' | awk -F, '{print $(NF-11)"," \ 
    $(NF-10)"," \ 
    $(NF-9)"," \ 
    $(NF-8)"," \ 
    $(NF-7)"," \ 
    $(NF-6)"," \ 
    $(NF-5)"," \ 
    $(NF-4)"," \ 
    $(NF-3)"," \ 
    $(NF-2)"," \ 
    $(NF-1)"," \
    $NF \ 
}''
 do
    FNAME='echo ${LINE} | awk -F, '{print $1}''
    LNAME='echo ${LINE} | awk -F, '{print $2}''
    COMPANY='echo ${LINE} | awk -F, '{print $3}''
    ADDRESS='echo ${LINE} | awk -F, '{print $4}''
    CITY='echo ${LINE} | awk -F, '{print $5}''
    COUNTY='echo ${LINE} | awk -F, '{print $6}''
    STATE='echo ${LINE} | awk -F, '{print $7}''
    ZIP='echo ${LINE} | awk -F, '{print $8}''
    PHONE='echo ${LINE} | awk -F, '{print $9}''
    FAX='echo ${LINE} | awk -F, '{print $10}''
    EMAIL='echo ${LINE} | awk -F, '{print $11}''
    WEB='echo ${LINE} | awk -F, '{print $12}''
 done
Improve this question
2
  • 1
    I think you need to review your syntax for command substitution Commented Dec 13, 2015 at 1:10
  • the grep pipeline should be in back quotes not single quotes, as should each of the echo pipelines in the body of the loop Commented Dec 13, 2015 at 1:13

1 Answer 1

Reset to default
1

In general, you are using single quotes '...' to wrap "command execution", which should be wrapped with back-quotes `...`, or (much better) $(...).

That makes the line that start with: for LINE in 'grep and ends with }'' fail.
Making the minimum possible changes, the code becomes:

#!/bin/bash

OLDIFS=${IFS}
IFS=$'\n'

head -1 50000_a.csv | sed 's/"//g'
for LINE in `grep WV 50000_a.csv | sed 's/"//g' | awk -F, '{print $(NF-11) "," \
    $(NF-10)"," \
    $(NF-9)"," \
    $(NF-8)"," \
    $(NF-7)"," \
    $(NF-6)"," \
    $(NF-5)"," \
    $(NF-4)"," \
    $(NF-3)"," \
    $(NF-2)"," \
    $(NF-1)"," \
    $NF \
}'`
do
    FNAME=`echo ${LINE} | awk -F, '{print $1}'`
    LNAME=`echo ${LINE} | awk -F, '{print $2}'`
    COMPANY=`echo ${LINE} | awk -F, '{print $3}'`
    ADDRESS=`echo ${LINE} | awk -F, '{print $4}'`
    CITY=`echo ${LINE} | awk -F, '{print $5}'`
    COUNTY=`echo ${LINE} | awk -F, '{print $6}'`
    STATE=`echo ${LINE} | awk -F, '{print $7}'`
    ZIP=`echo ${LINE} | awk -F, '{print $8}'`
    PHONE=`echo ${LINE} | awk -F, '{print $9}'`
    FAX=`echo ${LINE} | awk -F, '{print $10}'`
    EMAIL=`echo ${LINE} | awk -F, '{print $11}'`
    WEB=`echo ${LINE} | awk -F, '{print $12}'`
done

However, it is a mistake to try to process a file with a for loop, you should use read to get the output of grep:

#!/bin/bash

OLDIFS=${IFS}
IFS=$'\n'

head -1 50000_a.csv | sed 's/"//g'
while IFS=$'\n' read -r LINE
do
    FNAME="$(echo ${LINE} | awk -F, '{print $1}')"
    LNAME="$(echo ${LINE} | awk -F, '{print $2}')"
    COMPANY="$(echo ${LINE} | awk -F, '{print $3}')"
    ADDRESS="$(echo ${LINE} | awk -F, '{print $4}')"
    CITY="$(echo ${LINE} | awk -F, '{print $5}')"
    COUNTY="$(echo ${LINE} | awk -F, '{print $6}')"
    STATE="$(echo ${LINE} | awk -F, '{print $7}')"
    ZIP="$(echo ${LINE} | awk -F, '{print $8}')"
    PHONE="$(echo ${LINE} | awk -F, '{print $9}')"
    FAX="$(echo ${LINE} | awk -F, '{print $10}')"
    EMAIL="$(echo ${LINE} | awk -F, '{print $11}')"
    WEB="$(echo ${LINE} | awk -F, '{print $12}')"
done < <(
    grep WV 50000_a.csv | sed 's/"//g' | awk -F, '{print $(NF-11) "," \
    $(NF-10)"," \
    $(NF-9)"," \
    $(NF-8)"," \
    $(NF-7)"," \
    $(NF-6)"," \
    $(NF-5)"," \
    $(NF-4)"," \
    $(NF-3)"," \
    $(NF-2)"," \
    $(NF-1)"," \
    $NF \
    }'
)
Improve this answer
6
  • why is $() "much better" than back quotes? Commented Dec 13, 2015 at 1:15
  • @MurrayJensen Because it becomes very clear where a "Command Execution starts and ends. Commented Dec 13, 2015 at 1:22
  • I think it confuses things between variable expansions and command substitutions ... back quotes is very clear IMO Commented Dec 13, 2015 at 1:28
  • @MurrayJensen Exparience has shown that backticks are more dificult to use:The second form COMMAND` is more or less obsolete for Bash, since it has some trouble with nesting ("inner" backticks need to be escaped) and escaping characters. Use $(COMMAND), it's also POSIX! ` Commented Dec 13, 2015 at 1:37
  • echo ${LINE} | awk {print $n}'` squashes any consecutive whitespace and can delete some leading tokens from the first field. Assuming you actually want all fields as-is, it's much simpler (and faster) to just do while IFS=, read -r FNAME LNAME COMPANY etc etc. Or if all input lines contain at least 12 fields, skip the awk to select NF-11 etc, use read -r LINE; IFS=,; set -- $LINE; shift $#-12 then just use FNAME=$1; LNAME=$2; etc. Commented Dec 15, 2015 at 6:35

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.