How to export a flat file with different rows using SSIS?

Question

I have tree tables, Customer, Invoice and InvoiceRow with the standard relations.

These I have to export in one fixed field length file with the first two characters of each row identifying the row type. The row types have different specifications.

I could probably do it with a nested loop in a script block, but this is my first ever SSIS package and that solution feels wrong.

edit:

The output has to have:
Customer
Invoice
Rows
Customer
Invoice
Rows
and so on

/johan/

Answer 1

Process your three tables so that the outputs are all appropriate for your output file (including the row type designator). You'll have to do this in three separate flow paths in your data flow, then bring the rows together in a Union All data flow element. From there, process them as needed to create your output file.

Answer 2

Your gut feeling on doing this using a Script Destination component is correct. Unfortunately, this scenario doesn't jive with SSIS well. I don't consider this a beginner package. If you must use SSIS then I'd start by inner joining all the data so there is one row for each InvoiceRow, containing the data needed from all three tables.

CustomerCols, InvoiceCols, RowCols

Then, in the script destination component you'll need to keep track of the customer and invoice values, as they change you'll need to write extra rows to the output.

See Creating a Destination with the Script Component for more information on script destination.

My experience shows that script destinations can have good performance.

Answer 3

I would avoid writing Script Destination, and use just Script Transform + Flat File Destination. This way, you concentrate on the logical output (strings of data), while allowing SSIS to do actual writing to the file (it might be a bit more efficient, plus you concentrate on your business, not on writing to files).

First, you'll need to get denormalized data. You can do joins and sorts in the DBMS, but if you don't want to put too much pressure on DBMS - just get sorted data out of it and merge it using two SSIS Merge Join transforms.

Then do the script: keep running values of current Customer and Invoice, output them when they change, output InvoiceRow on every input. Something like this:

if (this.CustomerID != InputBuffer.CustomerID) {
  this.CustomerID = InputBuffer.CustomerID;
  OutputBuffer.AddRow();
  OutputBuffer.OutputColumn = "Customer: " + InputBuffer.CustomerID + " " + InputBuffer.CustomerName;
}
// repeat the same code for Invoice

OutputBuffer.AddRow();
OutputBuffer.OutputColumn = "InvoiceRow: " + InputBuffer.InvoiceRowPrice;

Finally, add a Flat File Destination with a single column (OutputColumn created by the script) to write this to the file.