Java multiple connection downloading

Question

Hi all,

I wanted to get some advice, I have started on a new project to create a java download accelerator that will use multiple connections. I wanted to know how best to go about this.

So far I have figured out that i can use HttpUrlConnection and use the range property, but wanted to know an efficient way of doing this. Once i have download the parts from the multiple connections i will then have to join the parts so that we end up with a fully downloaded file.

Thanks in advance :)

Answer 1

JDownloader is the best downloader I've seen. If you are interested, it's open source and surely you can learn a lot from their code.

Answer 2

1. Get the Length of the content.
2. Divide it according to any criteria on size etc.
3. Multi thread read the file from different ints. and Write them as maybe myfile.part1 .part2 etc
4. Read the same files once all downloaded into one single stream and write them to a new file myfile.file

I tried the following code to get the content length.

public Downloader(String path) throws IOException {
    int len = 0;
    URL url = new URL(path);
    URLConnection connectUrl = url.openConnection();
    System.out.println(len = connectUrl.getContentLength());
    System.out.println(connectUrl.getContentType());

    InputStream input = connectUrl.getInputStream();
    int i = len;
    int c = 0;
    System.out.println("=== Content ==="); 
    while (((c = input.read()) != -1) && (--i > 0)) {
        System.out.print((char) c);
    }
    input.close(); 
}

heres a sample to join the files. public void join(String FilePath)

{

    long leninfile=0,leng=0;

    int count=1,data=0;

    try

    {

        File filename=new File(FilePath);

        RandomAccessFile outfile = new RandomAccessFile(filename,"rw");

        while(true)

        {

            filename=new File(FilePath + count + ".sp");

            if (filename.exists())

            {

                RandomAccessFile infile = new RandomAccessFile(filename,"r");

                data=infile.read();

                while(data!=-1)

                {

                    outfile.write(data);

                    data=infile.read();

                }

                leng++;

                infile.close();

                count++;

            }

            else

            {

                break;

            }

        }

        outfile.close();

    }

    catch(Exception e)

    {

        e.printStackTrace();

    }

}

Hope it works out for you.

Answer 3

If you want to avoid joining segments after downloading you could use a FileChannel. With a FileChannel you can write to any position in a file (even with multiple threads).

So you could allocate the whole file at the beginning and then write the segments where they belong as they come in.