tags:

views:

24

answers:

1

I could use some hints or tips for a decent interface for reading file of special characteristics.

The files in question has a header (~120 bytes), a body (1 byte - 3gb) and a footer (4 bytes).

The header contains information about the body and the footer is only a simple CRC32-value of the body.

I use Java so my idea was to extend the "InputStream" class and add a constructor such as "public MyInStream( InputStream in)" where I immediately read the header and the direct the overridden read()'s the body.

Problem is, I can't give the user of the class the CRC32-value until the whole body has been read.

Because the file can be 3gb large, putting it all in memory is a be an idea.

Reading it all in to a temporary file is going to be a performance hit if there are many small files.

I don't know how large the file is because the InputStream doesn't have to be a file, it could be a socket.

Looking at it again, maybe extending InputStream is a bad idea.

Thank you for reading the confused thoughts of a tired programmer. :)

A: 

Looking at it again, maybe extending InputStream is a bad idea.

If users of the class need to access the body as a stream, it's IMO not a bad choice. Java's ObjectOutput/InputStream works like this.

I don't know how large the file is because the InputStream doesn't have to be a file, it could be a socket.

Um, then your problem is not with the choice of Java class, but with the design of the file format. If you can't change the format, there isn't really anything you can do to make the data at the end of the file available before all of it is read.

But perhaps you could encapsulate the processing of the checksum completely? Presumably it's a checksum for the body, so your class could always "read ahead" 4 bytes to see when the file ends and not return the last 4 bytes to the client as part of the body and instead compare them with a CRC computed while reading the body, throwing an exception when it does not match.

Michael Borgwardt