How to prevent the other threads from accessing a method when one thread is accessing a method?

Question

I want to search for a string in 10 files and write the matching lines to a single file. I wrote the matching lines from each file to 10 output files(o/p file1,o/p file2...) and then copied those to a single file using 10 threads.

But the output single file has mixed output(one line from o/p file1,another line from o/p file 2 etc...) because its accessed simultaneously by many threads. If I wait for all threads to complete and then write the single file it will be much slower. I want the output file to be written by one thread at a time. What should i do?

My source code:(only writing to single file method)

public void WriteSingle(File output_file,File final_output) throws IOException {
   synchronized(output_file){
       System.out.println("Writing Single file");
       FileOutputStream fo = new FileOutputStream(final_output,true);
       FileChannel fi = fo.getChannel();
       FileInputStream fs = new FileInputStream(output_file);
       FileChannel fc = fs.getChannel();
       int maxCount = (64 * 1024 * 1024) - (32 * 1024);
       long size = fc.size();
       long position = 0;
       while (position < size) {
           position += fc.transferTo(position, maxCount, fi);
       }
    }
}

Answer 1

public synchronized void method() {

}

And be sure to flush() / close() all streams before you exist the method.

And, as noted in the comments by Xavier, make sure you are invoking the method on the same instance. Because synchronization happens per-instance (in this case)

Answer 2

Source code:

public void WriteSingle(File output_file,File final_output)
      throws IOException
  {
     synchronized(output_file){
        try{ 
            wait();
        }
        catch (InterruptedException e) { 
            e.printStackTrace(); 
        } 

     System.out.println("Writing Single file");
 FileOutputStream fo = new FileOutputStream(final_output,true);
     FileChannel fi = fo.getChannel();
     FileInputStream fs = new FileInputStream(output_file);
     FileChannel fc = fs.getChannel();
     int maxCount = (64 * 1024 * 1024) - (32 * 1024);
     long size = fc.size();
     long position = 0;
     while (position < size) {
       position += fc.transferTo(position, maxCount, fi);
     }
   }

Answer 3

What about having a file writer class with lock (i.e. obtain lock on the instance, perform whatever write is necessary, then release the lock; somewhat like database transactions)? Consider passing the instance of the file writer to the objects/methods that need to execute the writes (with additional benefit of easier unit testing/mocking).

Answer 4

If I understand this, you want to prevent two threads from writing to the same file?

The easiest way to do that is to lock the file itself, not the File object:

public void WriteSingle(File output_file, File final_output) throws IOException {
    System.out.println("Writing Single file");
    FileOutputStream fo = new FileOutputStream(final_output,true);
    FileChannel fi = fo.getChannel();
    FileInputStream fs = new FileInputStream(output_file);
    FileChannel fc = fs.getChannel();

    FileLock lock = fi.lock(); // Get lock here, blocks until file is closed

    fc.transferTo(0, fc.size(), fi);

    fc.close();
    fi.close();
    fo.close();
    fs.close();
}

Answer 5

public synchronized void WriteSingle(File output_file,File final_output) throws IOException {
       System.out.println("Writing Single file");
       FileOutputStream fo = new FileOutputStream(final_output,true);
       FileChannel fi = fo.getChannel();
       FileInputStream fs = new FileInputStream(output_file);
       FileChannel fc = fs.getChannel();
       int maxCount = (64 * 1024 * 1024) - (32 * 1024);
       long size = fc.size();
       long position = 0;
       while (position < size) {
           position += fc.transferTo(position, maxCount, fi);
       }
}

If that doesn't work, then you may be using multiple instances to write to the file. In this case, you could try a class lock like this:

public void WriteSingle(File output_file,File final_output) throws IOException {
    synchronized(getClass()) {
       System.out.println("Writing Single file");
       FileOutputStream fo = new FileOutputStream(final_output,true);
       FileChannel fi = fo.getChannel();
       FileInputStream fs = new FileInputStream(output_file);
       FileChannel fc = fs.getChannel();
       int maxCount = (64 * 1024 * 1024) - (32 * 1024);
       long size = fc.size();
       long position = 0;
       while (position < size) {
           position += fc.transferTo(position, maxCount, fi);
       }
    }
}

This isn't ideal, but it should give us a hint about what your code is doing.

If neither of these work, then your general programming logic is probably wrong and the problem may have little to do with synchronization.