tags:

views:

670

answers:

2

H guys. I'm developing a custom component for SSIS. I'm having a problem when processing inputs. The problem is that the "ProcessInput" method gets executed more than once. Two times in this case.

This is the process input snippet:

public override void ProcessInput(int inputID, PipelineBuffer buffer)
{
    IDTSInput90 input = ComponentMetaData.InputCollection.GetObjectByID(inputID);
    if (input.InputColumnCollection.Count > 0)
    {
     while (buffer.NextRow())
     {
      try
      {
       for (int columnIndex = 0; columnIndex < input.InputColumnCollection.Count; columnIndex++)
       {

        ColumnInfo columnInfo = _columnInfos[input.InputColumnCollection[columnIndex].ID];
        IDTSInputColumn90 inputColumn = input.InputColumnCollection[columnIndex];
        try
        {
         //write to destination
        }
        catch (Exception writeEx)
        {
         throw new Exception("Couldn't write to destination");
        }
       }
      }
      catch (Exception ex)
      {
       throw ex;
      }
     }
    }
    else
    {
     throw new Exception("There is no columns in the input collection");
    }

}

I have no idea why its being called twice. This is the dataflow:

And this is the mapping window:

A: 

This is by design. SSIS sends data in chunks (buffers in SSIS terminology), to optimize memory usage. The buffer size is limited, so SSIS does not have to read all the data into memory (otherwise SSIS would not be able to process terabytes of data). So you can get multiple ProcessInput calls - one ProcessInput call per buffer.

In addition, you'll get one empty buffer with EndOfRowset flag set to true at the very end. But don't rely on this - this is more of an implementation detail (the last buffer is documented to have EndOfRowset = true, but it is not documented to be empty).

Michael
A: 

So i can safely ignore a buffer with EndOfRowSet == true?.

I will try that! and play with it a little.

Thanks for the tip.

Ironicnet
I would avoid checking EndOfRowset, unless you need to do something special at the end.The pattern I recommend iswhile (buffer.NextRow()){ // do something with the row}only if you need a special handling, add to this:if (buffer.EndOfRowset){ // do something at the end}
Michael
The problem is that is a destination task...I'm writing the data to an output. Before processing and after processing i need to enclose the info (like an XML).But i don't want to do it in the pre-execute and post-execute/cleanup...
Ironicnet
Makes sense, but don't ignore buffer with EOR (it is undocumented it is empty). Just do the processing as usual with while() loop, then add if (buffer.EOR) write-closing-xml-tag.And don't forget to mark answers as answers ;)
Michael