generic FIFO class to use for storing bytes and doing InputStream.read() [Java 5 and up]

Question

I'm thinking about using as a base some FIFO (for example, BoundedFifoBuffer from Apache Commons) to organize IO tasks in my application (I was looking for something easy to use as a temporary buffer for reading from a stream and FIFO simultaneously: so I will read data from an InputStream to it and then consume the data later; in my case consuming code will eat data mostly in fixed-size chunks but I can't guarantee that I will get those data in the same sized chunks from a stream - so to adapt this parts I want to use a FIFO that is also easily usable to InputStream.read() in its internal array to lower amount of data copying).

But I stumbled on the generic implementation of the FIFO collection. To me it looks like Java generics are just a facade that hides Object references. What about having a generic collection class (that has T[] array as its data container) with a primitive type parameter? Is it possible or do I have to use a wrapper type instead?

The most interesting question for me: is it a good idea to use generic collection like BoundedFifoBuffer, use it with byte generic type parameter and then try to use its array for reading data from a Stream: InputStream.read(byte[] b, int off, int len)? In case if generic collection here means a bunch of references to boxed bytes I doubt I want it this way... Any better suggestion than creating a special collection bound to byte type then?

PS: I was using Java 2 but then switched to C# 2 more than 4 years ago. Now I want to get my Java knowledge up-to-date. So I may get things really wrong at least around Java generics :)

UPDATE:

I still like Apache Commons *FifoBuffer classes. I'm considering to rewrite one of them to use the primitive byte type instead of generic type parameter and then add some logic to use its array for reading from an InputStream.

GNU Trove shows an interesting approach with simple code generation for primitive type friendly collections but it lacks FIFO out of the box. As a very general solution would be fun to get some *FifoBuffer and make a template for GNU Trove from it. Additional logic most likely will fit into two additional methods E[] getBufferArray() and int getBufferLength() (to know how to invoke InputStream.read() on its array directly). For my purpose I will skip Trove step for simplicity :) I checked Mahout 0.2 and so far didn't find any FIFO collections too. The example provided by McDowell (based on ByteArray*Stream) and java.nio buffers are very close to what I want. But ByteArray*Stream can be only reset to 0, you can't conveniently get some of its data off (not all buffer's data) and I wanted to remove only those parts I'm sure are ready to be processed and discarded (e.g. full packet or garbage before next valid packet in incoming data). java.nio.ByteBuffer has no way to discard already read data AFAIK.

To summarize: for now circular FIFO looks like the best option and I may get one from Apache Commons sources to adapt for my needs (primitive byte type + making it suitable for read() calls).

Any comments on the update?..

Answer 1

To begin with, you can't say BoundedFifoBuffer<byte>. The compiler won't permit it.

If you use the raw type, you get autoboxing, in which byte silently promotes itself to Byte, and you don't want that, either.

Answer 2

You can't use a primitive type as a type argument at all:

found   : int
required: reference
    List<int> l = new ArrayList<int>();
         ^

So you'll need to write a class that's specifically for bytes.

Answer 3

If you want to buffer all data, you can do it with a method like this:

  private static InputStream bufferAll(InputStream source) throws IOException {
    class OutputStore extends ByteArrayOutputStream {
      public byte[] buf() {
        return buf;
      }

      public int count() {
        return count;
      }
    }
    OutputStore store = new OutputStore();
    byte[] buffer = new byte[1024]; // arbitrary size
    int bytesRead;
    while ((bytesRead = source.read(buffer)) != -1) {
      store.write(buffer, 0, bytesRead);
    }
    return new ByteArrayInputStream(store.buf(), 0, store.count());
  }

(That OutputStore type is there so you don't copy its array contents via toByteArray.)

If you know the data sizes involved, have a look at the java.nio buffers. If you want to block until a certain amount of data is ready, use the DataInputStream decorator class' readFully methods. If you're reading from one thread while writing from another, have a look at the pipe classes.