Fastest way to read long[] from file?

Question

I have a file that contains about 200,000 long values that I want to read as fast as possible into a long[]. It's for an Android app; function calls are slow (so anything involving reading a long at a time with a "for" loop will be super slow) and I need loading to be fast. What can I use? Everything I look at seems to read only bytes fast.

I've used ByteBuffer and FileChannel from the NIO package before and this seems a really fast way to load arrays of values from files. However, I cannot work out how to use this to read data into a long[]. I've tried wrapping a long[] as a LongBuffer, but I cannot see any way I can feed data from a file into a LongBuffer.

Edit: Whatever method I use, I need to be able to use Arrays.binarySearch on the long[] array at the end.

Answer 1

There is no way to cast a byte[] into a long[]. However, you could try to use a FileChannel to read the content into a ByteBuffer and then get hold of a LongBuffer through ByteBuffer.asLongBuffer from which you could get a long[] through LongBuffer.array().

You could also try FileChannel.map to get a MappedByteBuffer of the file. This may be faster than going through a FileChannel.read.

If this doesn't work you could try to use a FileChannel to read the content into a ByteBuffer and then access the longs inside it, using ByteBuffer.getLong(index).

---

An alternative solution. (No method-calls in the loop :-)

byte[] byteArray = new byte[longCount * 8];
FileInputStream fis = new FileInputStream("lotsoflongs");
fis.read(byteArray);
fis.close();
for (int i = 0; i < longCount; i += 8)
    longArray[i >> 3] = ((long) byteArray[0+i]        << 56) +
                        ((long)(byteArray[1+i] & 255) << 48) +
                        ((long)(byteArray[2+i] & 255) << 40) +
                        ((long)(byteArray[3+i] & 255) << 32) +
                        ((long)(byteArray[4+i] & 255) << 24) +
                              ((byteArray[5+i] & 255) << 16) +
                              ((byteArray[6+i] & 255) <<  8) +
                              ((byteArray[7+i] & 255) <<  0);

I've benchmarked a few solutions now, and this one seems to be the fastest way of doing it. Also, note that the actual bytes read in fis.read(byteArray) may be less than the actual size of byteArray. Thus, if this should be done properly, you need to put it in a loop that iterates until all bytes have been read.

Answer 2

Try using a DataInputStream. Since you can easily find out the length of the file, you also know how many elements it contains (filesize / 8 byte).

 DataInputStream dataStream = new DataInputStream(inputStream);

 long count = filesize/8;
 long[] longArray = new long[count];

 for(int i=0;i<count;i++) 
     longArray[i] = dataStream.getLong();

Guess that should be enough to give you an idea.