Feed CSV file into variable in Bash for parsing

Question

For each row in a CSV file, I'd like to extract a field and reposition it in the row using Bash. The row is a URL and I am using / as a delimiter.

This is the starting file (start.csv):

https://docs.website.com/12-3/articles/guide-1/article-1.html
https://docs.website.com/12-2/articles/guide-2/article-5.html
https://docs.website.com/12-1/articles/guide-3/article-6.html

For later reference, the URL is https://{url}/{version}/irrelevant/{guide}/irrelevant.html.

The desired output is (end.csv):

url,name,tag,version,guide,views
https://docs.website.com/12-3/articles/guide-1/article-1.html,,,12-3,guide-1,0
https://docs.website.com/12-2/articles/guide-2/article-5.html,,,12-2,guide-2,0
https://docs.website.com/12-1/articles/guide-3/article-6.html,,,12-1,guide-3,0

I've unsuccessfully tried many variations of:

file="start.csv"
var="$(<<<"${file}" cut -d'/' -f4)"

sed -e "s|$|,$var,,,,0|g" < start.csv > end.csv

However this successfully produces a column with the version:

cut -d'/' -f4 < start.csv

Somewhere my logic is seriously flawed. Is anyone able to help me spot my problem? Thank you.

There are two problems here, one trivial, one serious. The trivial one is that <<<"${file}" cut -d'/' -f4 passes the string "start.csv" as input to cut, rather than reading from the file by that name. The serious one is that (if you fix the first one) $var will contain the version for all lines (separated by newlines), and then using that with sed will try to attach all of that to each and every line in the file. It doesn't apply line one of $var to line one of the file etc, it applies all lines of $var to line one of the file, then again to line two, etc. — Gordon Davisson
– Gordon Davisson, Commented Jun 10, 2020 at 23:36
@GordonDavisson Thank you for this explanation - now it is much clearer to me — hcdocs
– hcdocs, Commented Jun 11, 2020 at 16:37

anubhava · Accepted Answer · 2020-06-10 21:34:02Z

2

It is easier, using awk:

awk -F/ -v OFS=, '{print $0, "", "", $4, $6, 0}' file

https://docs.website.com/12-3/articles/guide-1/article-1.html,,,12-3,guide-1,0
https://docs.website.com/12-2/articles/guide-2/article-5.html,,,12-2,guide-2,0
https://docs.website.com/12-1/articles/guide-3/article-6.html,,,12-1,guide-3,0

answered Jun 10, 2020 at 21:34

anubhava

790k67 gold badges603 silver badges671 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

hcdocs Over a year ago

Way better than what I was trying to figure out. Thank you.

Collectives™ on Stack Overflow

Feed CSV file into variable in Bash for parsing

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related