I have to build an SSIS package where it will load the data based on column names. I have 7 flat files where I have to load to 2 tables in SQL server based on column names. the flat file having column names (first name, last name, phone number) should go to Table A. the flat file having column names (first name, last name, cell number) should go to Table B. The split should be done based on Column names but not column values. I am not a .net person.
-
So... what's the problem? Have you started working with the DTS? Any problems you found? Create your database and flat file connections and then create a Data flow for each of your files.EzLo– EzLo2019-07-03 15:19:28 +00:00Commented Jul 3, 2019 at 15:19
-
Why dont you load all flat files into one staging table if there aren't many columns, and sort out from thereVen– Ven2019-07-03 15:21:42 +00:00Commented Jul 3, 2019 at 15:21
-
I'd recommend 1 table with a 4th column called PhoneType. you can create views off of that to meet your needsKeithL– KeithL2019-07-03 15:46:24 +00:00Commented Jul 3, 2019 at 15:46
-
I am guessing at this, but is your problem you don't know what columns are inside the file? the easiest solution is to not read header rows and use a derived column based on file name to add phone type (if possible).KeithL– KeithL2019-07-03 15:49:18 +00:00Commented Jul 3, 2019 at 15:49
-
1what are the names of your files? can you determine content from that? are the files in the same folder? is this a repeatable process? Please complete your question and you will get help.KeithL– KeithL2019-07-03 15:53:18 +00:00Commented Jul 3, 2019 at 15:53
|
Show 2 more comments
1 Answer
Use a foreach loop to loop through the files saving full file path with name to a variable.
Use a script task to determine if the file is for cell phones or regular phones. Note: filepath is likely a variable as well from foreach loop
string readingLine; Using(System.IO.StreamReader readingFile = new System.IO.StreamReader(filePath)) { readingLine = readingFile.ReadLine(); } if(readingLine.Contains("Cell") //It would be best to use the full column name with a comma to avoid names with Cell in it to false trigger a true result { Dts.Variables("PhoneType").Value = "Cell"; { else { Dts.Variables("PhoneType").Value = "Phone"; }Add a dataflow
- Read the flat file skipping header row.
- Add a derived column to add phone type from variable
- Load to a table
After this is done you can create views for your desired output.
1 Comment
KeithL
You can conditionally split as well based on variable to load to different tables, but I don't think that is the best solution