How can you pipe an OutputStream to a StreamingDataHandler?

Question

I've got a Java web service in JAX-WS that returns an OutputStream from another method. I can't seem to figure out how to stream the OutputStream into the returned DataHandler any other way than to create a temporary file, write to it, then open it back up again as an InputStream. Here's an example:

@MTOM
@WebService
class Example {
    @WebMethod
    public @XmlMimeType("application/octet-stream") DataHandler service() {
        // Create a temporary file to write to
        File fTemp = File.createTempFile("my", "tmp");
        OutputStream out = new FileOutputStream(fTemp);

        // Method takes an output stream and writes to it
        writeToOut(out);
        out.close();

        // Create a data source and data handler based on that temporary file
        DataSource ds = new FileDataSource(fTemp);
        DataHandler dh = new DataHandler(ds);
        return dh;
    }
}

The main issue is that the writeToOut() method can return data that are far larger than the computer's memory. That's why the method is using MTOM in the first place - to stream the data. I can't seem to wrap my head around how to stream the data directly from the OutputStream that I need to provide to the returned DataHandler (and ultimately the client, who receives the StreamingDataHandler).

I've tried playing around with PipedInputStream and PipedOutputStream, but those don't seem to be quite what I need, because the DataHandler would need to be returned after the PipedOutputStream is written to.

Any ideas?

Answer 1

Wrapper pattern ? :-).

Custom javax.activation.DataSource implementation (only 4 methods) to be able to do this ?

return new DataHandler(new DataSource() { 
  // implement getOutputStream to return the stream used inside writeToOut() 
  ... 
});

I don't have the IDE available to test this so i'm only doing a suggestion. I would also need the writeToOut general layout :-).

Answer 2

Sorry, I only did this for C# and not java, but I think your method should launch a thread to run "writeToOut(out);" in parralel. You need to create a special stream and pass it to the new thread which gives that stream to writeToOut. After starting the thread you return that stream-object to your caller.

If you only have a method that writes to a stream and returns afterwards and another method that consumes a stream and returns afterwards, there is no other way.

Of coure the tricky part is to get hold of such a -multithreading safe- stream: It shall block each side if an internal buffer is too full.

Don't know if a Java-pipe-stream works for that.

Answer 3

I figured out the answer, along the lines that Christian was talking about (creating a new thread to execute writeToOut()):

@MTOM
@WebService
class Example {
    @WebMethod
    public @XmlMimeType("application/octet-stream") DataHandler service() {
        // Create piped output stream, wrap it in a final array so that the
        // OutputStream doesn't need to be finalized before sending to new Thread.
        PipedOutputStream out = new PipedOutputStream();
        InputStream in = new PipedInputStream(out);
        final Object[] args = { out };

        // Create a new thread which writes to out.
        new Thread(
            new Runnable(){
                public void run() {
                    writeToOut(args);
                    ((OutputStream)args[0]).close();
                }
            }
        ).start();

        // Return the InputStream to the client.
        DataSource ds = new ByteArrayDataSource(in, "application/octet-stream");
        DataHandler dh = new DataHandler(ds);
        return dh;
    }
}

It is a tad more complex due to final variables, but as far as I can tell this is correct. When the thread is started, it blocks when it first tries to call out.write(); at the same time, the input stream is returned to the client, who unblocks the write by reading the data. (The problem with my previous implementations of this solution was that I wasn't properly closing the stream, and thus running into errors.)