PowerShell - Get unique rows based on multiple columns from multiple tab/CSV delimited files by passing arguments to Select-Object

Question

I am trying to club multiple tab delimited files from a folder & get a unique result of distinct columns.

I have been able to make it work by passing the list of columns for the unique rows directly to Select-Object command.

However, I want to parameterise the unique columns list & pass it as a command argument because I am trying to create a function for this & expect the consumer to pass these values.

I have tried the below but it is not working & does not return any values.

$FileDate="20200420"
$TempFilePath="C:\Abdul\Work\Work From Home\TempFiles"
$DistinctColumnList="'ColA','ColB','ColC'"
   
$FilePattern =  $FileDate + "*.SDE"

$FinalFileNamePath = "C:\Abdul\Work\Work From Home\TempOut" + "\" + $FileDate + "_Combined.csv" 
$data = Import-Csv -Delimiter "`t"  -Path  (Get-ChildItem -Path $TempFilePath -Filter $FilePattern).FullName   

#This Works!!
#$dataUnique =$data | Select-Object 'ColA','ColB','ColC' -Unique

#How do I make this work?
$dataUnique =$data | Select-Object $DistinctColumnList -Unique


$dataUnique | export-csv $FinalFileNamePath -notypeinformation

$Waits = Read-Host -Prompt 'Finished'

$DistinctColumnList="'ColA','ColB','ColC'" --> $DistinctColumnList = 'ColA','ColB','ColC' and $dataUnique =$dataUnique | Select-Object ... --> $dataUnique = $data | Select-Object ... — Theo
– Theo, Commented Jun 25, 2020 at 10:33

Abdul Rehman Sayed · Accepted Answer · 2020-06-25 10:43:11Z

1

The solution was very simple. Just convert that string to an Array & use it.

$DistinctColumnList="ColA,ColB,ColC"
$ColumnsListArray=$DistinctColumnList -Split ",";

$dataUnique =$data | Select-Object $ColumnsListArray -Unique

edited Jun 25, 2020 at 10:43

answered Jun 25, 2020 at 10:35

Abdul Rehman Sayed

6,7129 gold badges51 silver badges79 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Theo Over a year ago

Why make it a single string in the first place? Just take off the double quotes and use $DistinctColumnList = 'ColA','ColB','ColC', so it will be a string array. Also, you are still selecting from the wrong variable, since the data is in $data, not $dataUnique

Abdul Rehman Sayed Over a year ago

@Theo $data was a typo.corrected it even in qn. Thanks for identifying. I cannot directly create $DistinctColumnList as array because I am expecting that to be passed via command arguments.

Theo Over a year ago

Arrays can be passed as parameter just fine.. Param ( [string[]]$DistinctColumnList) for instance

Collectives™ on Stack Overflow

PowerShell - Get unique rows based on multiple columns from multiple tab/CSV delimited files by passing arguments to Select-Object

1 Answer 1

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related