views:

111

answers:

2

A PipedInputStream/PipedOutputStream connection works great when the data only needs to be piped to one output, but if multiple output streams are connected to one input stream, the data becomes fragmented across the different outputs. My current solution involves having a threaded "reader" that reads data from an InputStream and then writes the data to OutputStream objects that have been associated with the reader. This seems to work ok, but it seems messy and inefficient by comparison to the native PipedIO classes.

Is there a better way to handle this or is the implementation that I'm working with about as good as I'm going to get?

A: 

If one input stream has to be read by multiple consumers, and the input stream is ephemeral (i.e. is not a resource that can be 'rewound', or supports multiple input pointers) you will generally have to provide a buffering scheme that behaves as if it retains each data item until all consumers have read it.

You have several choices on implementation. The simplest is what you suggest, with the overhead being primarily storage space for multiple copies of the data in the output buffers. If storage is an issue you could provide a single buffer that maintains separate read pointers, one for each consumer, and keeps in memory only the data between the lowest and highest read pointers. If the consumers read data at greatly different speeds you could still end up with most or all of the input data in memory, at which point some kind of input throttling, or intermediate disk buffering scheme, would become necessary.

I assume the single input stream is not persistent (i.e. a file on disk)... in that case the solution is trivial.

Jim Garrison
Thank you for the response. You are correct in assuming that the data is not persistent. Basically I am reading data from an SSH connection for verification purposes. I buffer the command responses so that they can be verified against expected behavior, write data to a debug log as required and write all data to a GUI for monitoring purposes. Since I'm really only concerned about buffering small amounts data for the command response, I think that I'm either going to stick with my current method, or the multiplex method above unless there are other suggestions.
WeeTodd
A: 

Old style code, this will give you a good starting place.

Romain Hippeau
Thanks for the response. Do you have any additional input on how well this would work when having to talk across threads as compared to my current solution? I tried playing with piping the input data to a multiplex output stream but the performance seem to be laggy and not as reliable.
WeeTodd
@WeeTodd You are probably going to need to benchmark the different solutions. I would recommend moving forward and then looking at different options if you run into perf issues. Make it so that you can easily replace implemntations
Romain Hippeau