views:

384

answers:

2

I am running my application under a profiler. The 'class' that has the most memory consumption is the 'char[]' which is about 10kB in my application.

I then created an InputStream (PipedInputStream to be exact) which holds a byte array data of 300MB.

Then I took a look at my profiler, and I don't see any significant change ( don't see anywhere that something eats up 300MB).

The question is, if that 300MB of byte array is not in memory, where is java keeping it?

[Update] Additional info on how I got the 300MB to my PipedInputStream:

I am developing a web app that has a file upload mechanism. And in one of the processes in my file upload, I create an input stream (PipedInputStream). Basically,

  1. I read the multipartfile's input stream (a few KB of byte[] at a time),
  2. Created a PipedOutputStream
  3. Created a PipedInputStream ( passing the recently created output stream to the constructor )
  4. Wrote the multipart's input stream to my PipedOutputStream (running on a separated thread; which flushes and closes that output stream before exiting the thread). At this point, I now have a copy of the multipart's bytes in my own input stream
  5. Then (accidentally) stored that input stream in my http session (discussion/debate on whether that is a good idea would be on a different question)

So the question then again is, where is Java keeping my InputStream's content (I don't see it anywhere in my profiler)?

[Update#2]

I have a FileOutputStream which reads from the PipedInputStream and writes to a file.

+2  A: 

A PipedInputStream just makes data available when it's written by the output stream that it's connected to. So long as you keep reading from your input stream as fast as it receives data from the output stream, there won't be much data to buffer.

If that doesn't help, you'll need to give more information about what you're doing with the piped input stream - what output stream is it connected to, and what's reading from it?

EDIT: You still haven't said what's reading from your PipedInputStream. Something must be, as otherwise the PipedOutputStream would block - PipedInputStream only have a fairly small buffer (by default).

Jon Skeet
Additional information has been provided in the question. Thanks
Franz See
sorry about that. just added update #2. ..And I'm now starting to understand what you're trying to say. Thanks.
Franz See
A: 

A PipedInputStream does not store any data at all. Also, where do you get that 300 MB byte array from?

Bombe
Additional information has been provided in the question. Thanks
Franz See
A PipedInputStream *does* store data - it stores whatever has been written to the attached output stream, but not read from the input stream yet. However, it shouldn't buffer *much* data...
Jon Skeet
This answer is wrong. PipedInputStream has a 1k buffer by default. This size can be changed by overriding the class. When the buffer is full, waiting for a thread to read from the the stream, the connected PipedOutputStream will block on write operations. Otherwise, both threads are runnable.
erickson