Multi threading in java

Question

if i have a file contains 4000 bytes, can i have 4 threads read from the file at same time? and each thread access a different section of the file.

thread 1 read 0-999, thread 2 read 1000 - 2999, etc.

please give an example in java.

Answer 1

RandomAccessFile or FileChannel will let you access bytes within a file. For waiting until your threads finish, look at CyclicBarrier or CountDownLatch.

Answer 2

You must somehow synchronize the read access to the file. I suggest to use the ExecutorService:

Your main thread reads the IDs from the file and passed them to the executor service one at a time. The executor will run N threads to process N IDs concurrently.

Answer 3

The file is very very small, and will be very fast to load. What I would do is create a thread-safe data class that loads the data. Each processing thread can then request an ID from the data class and receive a unique one with a guarantee of no other thread sending the same ID to your remote service.

In this manner, you remove the need to have all the threads accessing the file, and trying to figure out who has read and sent what ID.

Answer 4

Given this comment by the question's author:

I want to run a batch file , in which it contains thousands of unique id's. this each unique id will send as a request to the remote system. So i want to send requests parallely using threads to speed up the process. But if use multi threads then all the threads reading complete data and duplicate requests are sending. So i want to avoid this duplicate requests.

I would suggest that you load the file into memory as some kind of data structure - an array of ids perhaps. Have the threads consume ids from the array. Be sure to access the array in a synchronized manner.

If the file is larger than you'd like to load in memory or the file is constantly being appended to then create a single producer thread that watches and reads from the file and inserts ids into a queue type structure.

Answer 5

Sorry, Here is the working code. Now i've test it self :-)

package readfilemultithreading;

import java.io.BufferedInputStream;
import java.io.File;
import java.io.FileInputStream;
import java.io.IOException;
import java.util.ArrayList;
import java.util.List;

public class MultiThreadFileReader
{

    public MultiThreadFileReader(File fileToRead, int numThreads, int numBytesForEachThread)
    {
        this.file = fileToRead;
        this.numThreads = numThreads;
        this.bytesForEachThread = numBytesForEachThread;
        this.bytes = new byte[(int) file.length()];
    }
    private File file;
    private int numThreads;
    private byte[] bytes;
    int bytesForEachThread;

    public byte[] getResult()
    {
        return bytes;
    }

    public void startReading()
    {
        List<ReaderThread> readers = new ArrayList<ReaderThread>();
        for (int i = 0; i < numThreads; i ++) {
            ReaderThread rt = new ReaderThread(i * bytesForEachThread, bytesForEachThread, file);
            readers.add(rt);
            rt.start();
        }
        // Each Thread is Reading....
        int resultIndex = 0;
        for (int i = 0; i < numThreads; i++) {
            ReaderThread thread = readers.get(i);
            while (!thread.done) {
                try {
                    Thread.sleep(1);
                } catch (Exception e) {
                }
            }
            for (int b = 0; b < thread.len; b++, resultIndex++)
            {
                bytes[resultIndex] = thread.rb[b];
            }
        }
    }

    private class ReaderThread extends Thread
    {

        public ReaderThread(int off, int len, File f)
        {
            this.off = off;
            this.len = len;
            this.f = f;
        }
        public int off,  len;
        private File f;
        public byte[] rb;
        public boolean done = false;

        @Override
        public void run()
        {
            done = false;
            rb = readPiece();
            done = true;
        }

        private byte[] readPiece()
        {
            try {
                BufferedInputStream reader = new BufferedInputStream(new FileInputStream(f));
                if (off + len > f.length()) {
                    len = (int) (f.length() - off);
                    if (len < 0)
                    {
                        len = 0;
                    }
                    System.out.println("Correct Length to: " + len);
                }
                if (len == 0)
                {
                    System.out.println("No bytes to read");
                    return new byte[0];
                }
                byte[] b = new byte[len];
                System.out.println("Length: " + len);
                setName("Thread for " + len + " bytes");
                reader.skip(off);
                for (int i = off, index = 0; i < len + off; i++, index++)
                {
                    b[index] = (byte) reader.read();
                }
                reader.close();
                return b;
            } catch (IOException e) {
                e.printStackTrace();
            }
            return null;
        }
    }
}

Here is usage code:

package readfilemultithreading;

import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;

public class Main
{

    public static void main(String[] args)
    {
        new Main().start(args);
    }

    public void start(String[] args)
    {
        try {
            MultiThreadFileReader reader = new MultiThreadFileReader(new File("C:\\Users\\Martijn\\Documents\\Test.txt"), 4, 2500);
            reader.startReading();
            byte[] result = reader.getResult();
            FileOutputStream stream = new FileOutputStream(new File("C:\\Users\\Martijn\\Documents\\Test_cop.txt"));
            for (byte b : result) {
                System.out.println(b);
                stream.write((int) b);
            }
            stream.close();
        } catch (IOException ex) {
            System.err.println("Reading failed");
        }
    }
}

Can I get now my +1 back ;-)