Most efficient way to create InputStream from OutputStream

Question

This page: http://ostermiller.org/convert_java_outputstream_inputstream.html

describes how to create an InputStream from OutputStream:

new ByteArrayInputStream(out.toByteArray())

Other alternatives are to use PipedStreams and new threads which is cumbersome.

I do not like the idea of copying many megabytes to new in memory Bytes array. Is there a library that allows to do it more efficiently ?

EDIT:

By advise from Laurence Gonsalves, i tried PipedStreams and it turned out they are not that hard to deal with. Here's the sample code in clojure:

(defn #^PipedInputStream create-pdf-stream [pdf-info]
  (let [in-stream (new PipedInputStream)
        out-stream (PipedOutputStream. in-stream)]
    (.start (Thread. #(;Here you write into out-stream)))
    in-stream))

Answer 1

If you don't want to copy all of the data into an in-memory buffer all at once then you're going to have to have your code that uses the OutputStream (the producer) and the code that uses the InputStream (the consumer) either alternate in the same thread, or operate concurrently in two separate threads. Having them operate in the same thread is probably much more complicated that using two separate threads, is much more error prone (you'll need to make sure that the consumer never blocks waiting for input, or you'll effectively deadlock) and would necessitate having the producer and consumer running in the same loop which seems way too tightly coupled.

So use a second thread. It really isn't that complicated. The page you linked to had a perfect example:

  PipedInputStream in = new PipedInputStream();
  PipedOUtputStream out = new PipedOutputStream(in);
  new Thread(
    new Runnable(){
      public void run(){
        class1.putDataOnOutputStream(out);
      }
    }
  ).start();
  class2.processDataFromInputStream(in);

Answer 2

The Apache Commons IOUtils class has methods for copying streams and readers/writers. You would need to look at the source code to see how efficient they are.

EDIT: I checked ... the IOUtil Stream copy methods all use the Stream read() and write() methods and a 4K byte buffer. That doesn't strike me as particularly efficient. You may be able to implement a more efficient copy using NIO FileChannels, etcetera.