Looking for some help turning a CSV into variables. I tried using IFS, but seems you need to define the number of fields. I need something that can handle varying number of fields.
*I am modifying my original question with the current code I'm using (taken from the answer provided by hschou) which includes updated variable names using type instead of row, section etc.
I'm sure you can tell by my code, but I am pretty green with scripting, so I am looking for help to determine if and how I should add another loop or take a different approach to parsing the typeC data because although they follow the same format, there is only one entry for each of the typeA and typeB data, and there can be between 1-15 entries for the typeC data. The goal being only 3 files, one for each of the data types.
Data format:
Container: PL[1-100]
TypeA: [1-20].[1-100].[1-1000].[1-100]-[1-100]
TypeB: [1-20].[1-100].[1-1000].[1-100]-[1-100]
TypeC (1 to 15 entries): [1-20].[1-100].[1-1000].[1-100]-[1-100]
*There is no header in the CSV, but if there were it would look like this (Container, typeA, and typeB data always being in position 1,2,3, and typeC data being all that follow): Container,typeA,typeB,typeC,tycpeC,typeC,typeC,typeC,..
CSV:
PL3,12.1.4.5-77,13.6.4.5-20,17.3.577.9-29,17.3.779.12-33,17.3.802.12-60,17.3.917.12-45,17.3.956.12-63,17.3.993.12-42
PL4,12.1.4.5-78,13.6.4.5-21,17.3.577.9-30,17.3.779.12-34
PL5,12.1.4.5-79,13.6.4.5-22,17.3.577.9-31,17.3.779.12-35,17.3.802.12-62,17.3.917.12-47
PL6,12.1.4.5-80,13.6.4.5-23,17.3.577.9-32,17.3.779.12-36,17.3.802.12-63,17.3.917.12-48,17.3.956.12-66
PL7,12.1.4.5-81,13.6.4.5-24,17.3.577.9-33,17.3.779.12-37,17.3.802.12-64,17.3.917.12-49,17.3.956.12-67,17.3.993.12-46
PL8,12.1.4.5-82,13.6.4.5-25,17.3.577.9-34
Code:
#!/bin/bash
#Set input file
_input="input.csv"
# Pull variables in from csv
# read file using while loop
while read; do
declare -a COL=( ${REPLY//,/ } )
echo -e "containerID=${COL[0]}\ntypeA=${COL[1]}\ntypeB=${COL[2]}" >/tmp/typelist.txt
idx=1
while [ $idx -lt 10 ]; do
echo "typeC$idx=${COL[$((idx+2))]}" >>/tmp/typelist.txt
let idx=idx+1
#whack off empty variables
sed '/\=$/d' /tmp/typelist.txt > /tmp/typelist2.txt && mv /tmp/typelist2.txt /tmp/typelist.txt
#set variables from temp file
. /tmp/typelist.txt
done
sleep 1
#Parse data in this loop.#
echo -e "\n"
echo "Begin Processing for $container"
#echo $typeA
#echo $typeB
#echo $typeC
#echo -e "\n"
#Strip - from sub data for extra parsing
typeAsub="$(echo "$typeA" | sed 's/\-.*$//')"
typeBsub="$(echo "$typeB" | sed 's/\-.*$//')"
typeCsub1="$(echo "$typeC1" | sed 's/\-.*$//')"
#strip out first two decimils for extra parsing
typeAprefix="$(echo "$typeA" | cut -d "." -f1-2)"
typeBprefix="$(echo "$typeB" | cut -d "." -f1-2)"
typeCprefix1="$(echo "$typeC1" | cut -d "." -f1-2)"
#echo $typeAsub
#echo $typeBsub
#echo $typeCsub1
#echo -e "\n"
#echo $typeAprefix
#echo $typeBprefix
#echo $typeCprefix1
#echo -e "\n"
echo "Getting typeA dataset for $typeA"
#call api script to pull data ; echo out for test
echo "API-gather -option -b "$typeAsub" -g all > "$container"typeA-dataset"
sleep 1
echo "Getting typeB dataset for $typeB"
#call api script to pull data ; echo out for test
echo "API-gather -option -b "$typeBsub" -g all > "$container"typeB-dataset"
sleep 1
echo "Getting typeC dataset for $typeC1"
#call api script to pull data ; echo out for test
echo "API-gather -option -b "$typeCsub" -g all > "$container"typeC-dataset"
sleep 1
echo "Getting additional typeC datasets for $typeC2-15"
#call api script to pull data ; echo out for test
echo "API-gather -option -b "$typeCsub2-15" -g all >> "$container"typeC-dataset"
sleep 1
echo -e "\n"
done < "$_input"
exit 0
Speed isnt a concern, but if I've done anything really stupid up there, feel free to slap me in the right direction. :)
perl, rather thanbashfor this.perlhas thesplitfunction (perldoc split), and handles variable fields easily.