SSIS - ProcessInput in PipelineComponent gets called more than once

Question

H guys. I'm developing a custom component for SSIS. I'm having a problem when processing inputs. The problem is that the "ProcessInput" method gets executed more than once. Two times in this case.

This is the process input snippet:

public override void ProcessInput(int inputID, PipelineBuffer buffer)
{
    IDTSInput90 input = ComponentMetaData.InputCollection.GetObjectByID(inputID);
    if (input.InputColumnCollection.Count > 0)
    {
     while (buffer.NextRow())
     {
      try
      {
       for (int columnIndex = 0; columnIndex < input.InputColumnCollection.Count; columnIndex++)
       {

        ColumnInfo columnInfo = _columnInfos[input.InputColumnCollection[columnIndex].ID];
        IDTSInputColumn90 inputColumn = input.InputColumnCollection[columnIndex];
        try
        {
         //write to destination
        }
        catch (Exception writeEx)
        {
         throw new Exception("Couldn't write to destination");
        }
       }
      }
      catch (Exception ex)
      {
       throw ex;
      }
     }
    }
    else
    {
     throw new Exception("There is no columns in the input collection");
    }

}

I have no idea why its being called twice. This is the dataflow:

And this is the mapping window:

Answer 1

This is by design. SSIS sends data in chunks (buffers in SSIS terminology), to optimize memory usage. The buffer size is limited, so SSIS does not have to read all the data into memory (otherwise SSIS would not be able to process terabytes of data). So you can get multiple ProcessInput calls - one ProcessInput call per buffer.

In addition, you'll get one empty buffer with EndOfRowset flag set to true at the very end. But don't rely on this - this is more of an implementation detail (the last buffer is documented to have EndOfRowset = true, but it is not documented to be empty).

Answer 2

So i can safely ignore a buffer with EndOfRowSet == true?.

I will try that! and play with it a little.

Thanks for the tip.