Read a segment of a file in Java / Android

Question

I'm sure this might be a simple question, but unfortunately this is my first time using Java and working the Android SDK.

I am uploading files on Android using the Apache HTTP libraries, in particular using the MultipartEntity.

I'm uploading to a service that allows me to send them chunks of the file, and once complete, they'll reassemble the chunks. I'd like to take advantage of this feature.

Here's the scenario.

File FOO.BAR is 20 MB. I'd split it into some arbitrary chunk size, let's say 1 MB, which means 20 chunks. Chunks #3 and #14 fail (maybe the cellular/WiFi connection was bad). I can now re-upload just these two chunks and everything will be good.

What I'd like to know is how can I read only part of a file (like the data between 3MB and 4MB)?

The file piece should be an InputStream or File object.

Thanks, Makoto

Answer 1

Use skip() method of FileInputStream stream to seek to the fragment you need.

Answer 2

You can either use the skip(long) method to skip the number of bytes in the InputStream or you can create a RandomAccessFile on the File object and call its seek(long) method to set the pointer to that position so you can start reading from there.

The quick test below reads in a 4mb+ file (between 3m and 4mb) and writes the read data to an ".out" file.

import java.io.*;
import java.util.*;

public class Test {

    public static void main(String[] args) throws Throwable {
       long threeMb = 1024 * 1024 * 3;
       File assembled =  new File(args[0]); // your downloaded and assembled file
       RandomAccessFile raf = new RandomAccessFile(assembled, "r"); // read
       raf.seek(threeMb); // set the file pointer to 3mb
       int bytesRead = 0;
       int totalRead = 0;
       int bytesToRead = 1024 * 1024; // 1MB (between 3M and 4M

       File f = new File(args[0] + ".out");
       FileOutputStream out = new FileOutputStream(f);

       byte[] buffer = new byte[1024 * 128]; // 128k buffer 
       while(totalRead < bytesToRead) { // go on reading while total bytes read is
                                        // less than 1mb
         bytesRead = raf.read(buffer);
         totalRead += bytesRead;
         out.write(buffer, 0, bytesRead);
         System.out.println((totalRead / 1024));
       }
    }
}

Answer 3

I was able to figure it out... just had to discover that there's a ByteArrayInputStream that would allow me to convert my byte[] buffer to an InputStream. From here on, I can now track which chunks failed and handle it. Thanks Konstantin for Here's my implementation:

    final int chunkSize = 512 * 1024; // 512 kB
    final long pieces = file.length() / chunkSize;
    int chunkId = 0;

    HttpPost request = new HttpPost(endpoint);

    BufferedInputStream stream = new BufferedInputStream(new FileInputStream(file));

    for (chunkId = 0; chunkId < pieces; chunkId++) {
        byte[] buffer = new byte[chunkSize];

        stream.skip(chunkId * chunkSize);
        stream.read(buffer);

        MultipartEntity entity = new MultipartEntity();
        entity.addPart("chunk_id", new StringBody(String.valueOf(chunkId)));
        request.setEntity(entity);
        ByteArrayInputStream arrayStream = new ByteArrayInputStream(buffer);

        entity.addPart("file_data", new InputStreamBody(arrayStream, filename));

        HttpClient client = app.getHttpClient();
        client.execute(request);
    }