Basically, i have a series of steps that are processed using the pipes-filter pattern using the typical form:
public interface IOperation<IN, OUT>
{
Iterator<OUT> execute(Iterator<IN> input);
}
So if i have a pipeline with steps X->Y->Z. I want to be able to batch the outputs of filter X and execute all subsequent steps Y and Z (It may even be the case that i want to batch the outputs of Y). Something similar to what is described here i.e. utilizing the yield functionality in C#, but using java.
Subsequently i will want to have a more multithreaded approach, where i could execute each batch on a seperate thread using a queue.
Any ideas what be the best way to approach this issue?