views:

563

answers:

10

Java is not my main programming language so I might be asking the obvious.

But is there a simple file-handling library in Java, like in python?

For example I just want to say:

File f = Open('file.txt', 'w')
for(String line:f){
      //do something with the line from file
}

Thanks!

UPDATE: Well, the stackoverflow auto-accepted a weird answer. It has to do with bounty that I placed - so if you want to see other answers, just scroll down!

+11  A: 

Reading a file line by line in Java:

BufferedReader in = new BufferedReader(new FileReader("myfile.txt"));

String line;
while ((line = in.readLine()) != null) {
    // Do something with this line
    System.out.println(line);
}

in.close();

Most of the classes for I/O are in the package java.io. See the API documentation for that package. Have a look at Sun's Java I/O tutorial for more detailed information.

addition: The example above will use the default character encoding of your system to read the text file. If you want to explicitly specify the character encoding, for example UTF-8, change the first line to this:

BufferedReader in = new BufferedReader(
    new InputStreamReader(new FileInputStream("myfile.txt"), "UTF-8"));
Jesper
Hm.. that still looks clunky :-(
drozzy
Yes, but that's how it works with Java's standard API, Java isn't the most concise language...
Jesper
So, outside of the toy example of reading a file line by line, what other things do you want to do? It isn't that clunky IMHO.
Justin
Well, basically same thing that python's File object does. It also has binary mode. I never had problems with it. http://docs.python.org/py3k/library/functions.html#open
drozzy
this version doesnt work with utf-8 text files
Chris
It would be cool to have a code-template for this in eclipse, http://stackoverflow.com/questions/1028858/useful-eclipse-java-code-templates/1051926#1051926
questzen
@Chris: added an example to show how you can specify the character encoding.
Jesper
How could anyone actually like to use code templates? Doesn't it just feel wrong and dirty? I mean if you are really banging out SO many of these things that you need to have IDE help you type it out - there is something fundamentally wrong in the "programming" approach.
drozzy
+3  A: 

If you already have dependencies to Apache commons lang and commons io this could be an alternative:

String[] lines = StringUtils.split(FileUtils.readFileToString(new File("myfile.txt")), '\n');
for(String line: lines){
      //do something with the line from file
}

(I would prefer Jesper's answer)

stacker
That reads the entire file into a `String`, and then splits the whole thing in memory... that could use a lot of memory if the file is large.
Jesper
I second @Jesper's consideration. It's just too many String's in memory and can literally kill your application (OutOfMemoryError)
MasterGaurav
Just added an alternative before the bounty was added, and I also recommended Jespers answer. In cases of smaller files it works fine.
stacker
FileUtils has a method readLines, which returns a list of strings.
Krab
I try to stay away from anything in Apache commons, as it usually means including all 100 Zillion other dependencies in apache commons.
drozzy
+14  A: 

I was thinking something more along the lines of:

File f = File.open("C:/Users/File.txt");

for(String s : f){
   System.out.println(s);
}

Here is my source code for it:

import java.io.BufferedReader;
import java.io.BufferedWriter;
import java.io.FileReader;
import java.io.FileWriter;
import java.io.IOException;
import java.io.Writer;
import java.util.Iterator;

public abstract class File implements Iterable<String>{
    public final static String READ = "r";
    public final static String WRITE = "w";

    public static File open(String filepath) throws IOException{
        return open(filepath, READ);
    }   

    public static File open(String filepath, String mode) throws IOException{
    if(mode == READ){
        return new ReadableFile(filepath);
    }else if(mode == WRITE){
        return new WritableFile(filepath);
    }
    throw new IllegalArgumentException("Invalid File Write mode '" + mode + "'");
    }

    //common methods
    public abstract void close() throws IOException;

    // writer specific
    public abstract void write(String s) throws IOException;

}

class WritableFile extends File{
    String filepath;
    Writer writer;

    public WritableFile(String filepath){
        this.filepath = filepath;
    }

    private Writer writer() throws IOException{
        if(this.writer == null){
            writer = new BufferedWriter(new FileWriter(this.filepath));
        }
        return writer;
    }

    public void write(String chars) throws IOException{
        writer().write(chars);
    }

    public void close() throws IOException{
        writer().close();
    }

    @Override
    public Iterator<String> iterator() {        
        return null;
    }
}

class ReadableFile extends File implements Iterator<String>{
    private BufferedReader reader;
    private String line;    
    private String read_ahead;

    public ReadableFile(String filepath) throws IOException{        
        this.reader = new BufferedReader(new FileReader(filepath)); 
        this.read_ahead = this.reader.readLine();
    }

    private Reader reader() throws IOException{
         if(reader == null){
               reader = new BufferedReader(new FileReader(filepath));   
         }
         return reader;
    }

    @Override
    public Iterator<String> iterator() {
        return this;
    }

    @Override
    public void close() throws IOException {
        reader().close();
    }

    @Override
    public void write(String s) throws IOException {
        throw new IOException("Cannot write to a read-only file.");
    }

    @Override
    public boolean hasNext() {      
        return this.read_ahead != null;
    }

    @Override
    public String next() {
        if(read_ahead == null)
            line = null;
        else
            line = new String(this.read_ahead);

        try {
            read_ahead = this.reader.readLine();
        } catch (IOException e) {
            read_ahead = null;
            reader.close()
        }
        return line;
    }

    @Override
    public void remove() {
        // do nothing       
    }
}

and here is the unit-test for it:

import java.io.IOException;
import org.junit.Test;

public class FileTest {
    @Test
    public void testFile(){
        File f;
        try {
            f = File.open("File.java");
            for(String s : f){
                System.out.println(s);
            }
        } catch (IOException e) {
            e.printStackTrace();
        }
    }

    @Test
    public void testReadAndWriteFile(){
        File from;
        File to;
        try {
            from = File.open("File.java");
            to = File.open("Out.txt", "w");
            for(String s : from){           
                to.write(s + System.getProperty("line.separator"));
            }
            to.close();
        } catch (IOException e1) {
            e1.printStackTrace();
        }   
    }
}
drozzy
That's interesting, you can make it look almost like in Python. I would call the class `File` something else, to avoid confusion with Java's own `java.io.File`. Make the `remove()` method in your `FileIterator` throw `UnsupportedOperationException` instead of leaving it empty.
Jesper
Thanks! I was not sure what to do with remove there ;-) I think I WILL keep the file name however, as with namespaces I don't see any problems arise. I made more changes to this - and I would really like someone with java background to take a look at it. I will post my version on pastebin tomorrow.
drozzy
Ok, i've added my code.
drozzy
Drozzy, I think it's actually preferable to leave your code here regardless of the length. I always assume when answering that every other website on the planet will disappear (yes, even MSDN, Wikipedia and the like). Answers here on SO should be able to stand on their own merits. I have no problems with linking to other sites for more detail but the "meat" should be here. Let's just think of what happens to this answer if pastebin goes under (or worse, starts yo monetise their investment). This answer then becomes absolutely useless since it's basically a quote from the question.
paxdiablo
paxdiablo
Wow ok thanks :-)
drozzy
+1 for cleverness, but your iterator needs to make sure that the file is closed if an IOException occurs in next()
Justin
Thanks! I've added the close. Yours - is precisely the types of comments that I wanted to get, because I am not sure I am handling the resources correctly.
drozzy
Your `next` method should not create a new `String`; just use `line = read_ahead`. Also, it should check for `read_ahead == null` and throw a `NoSuchElementException` (see the JavaDoc for `Iterator<E>`).
Kevin Brock
Also, the `close()` call in `next()` probably is a compile error since it is declared to throw `IOException`.
Kevin Brock
Actually, I would recommend throwing some unchecked exception with the IOException as a cause so that the iterator doesn't silently fail from the read exception (in `next()`).
Kevin Brock
There is no need for `ReadableFile` to have the `reader()` private method. Just make `reader` variable final. Your constructor assigns this.
Kevin Brock
`WriteableFile` should probably follow the same design as `ReadableFile` in the constructor, then eliminate the `writer()` private method. It's best to open the file during construction in this case so that open failures (e.g. no such file) would occur when the caller opens the file (i.e. at `File.open(...)` instead of the first call to `write(...)`).
Kevin Brock
Thanks for comments. Any chance you can post up an answer with the changes you are suggesting?
drozzy
A: 

You could use jython which lets you run Python syntax in Java.

Nathan Voxland
I didn't downvote you. :-(
drozzy
A: 

Nice example here: Line by line iteration

Rulmeq
This is indeed what I was thinking. Except the code is not reviewed and tested by anyone. It would be great if there was like a community of utilities project. Also - the problem is that code eats up the newline character. (Which I must admit my answer does too - but it should leave the character in).
drozzy
+2  A: 

If you want to iterate through a file by strings, a class you might find useful is the Scanner class.

import java.io.*;
import java.util.Scanner;

    public class ScanXan {
        public static void main(String[] args) throws IOException {
            Scanner s = null;
            try {
                s = new Scanner(new BufferedReader(new FileReader("myFile.txt")));

                while (s.hasNextLine()) {
                    System.out.println(s.nextLine());
                }
            } finally {
                if (s != null) {
                    s.close();
                }
            }
        }
    }

The API is pretty useful: http://java.sun.com/javase/7/docs/api/java/util/Scanner.html You can also parse the file using regular expressions.

Grantismo
Thanks, but I really don't want to have anything to do with BufferedReaders or anything of that sort. Just give it a filepath, and done! Hm... but if I think about it more - i might wrap this Scanner in my own wrapper that will allow for some convenience.
drozzy
+2  A: 

I never get tired of pimping Google's guava-libraries, which takes a lot of the pain out of... well, most things in Java.

How about:

for (String line : Files.readLines(new File("file.txt"), Charsets.UTF_8)) {
   // Do something
}

In the case where you have a large file, and want a line-by-line callback (rather than reading the whole thing into memory) you can use a LineProcessor, which adds a bit of boilerplate (due to the lack of closures... sigh) but still shields you from dealing with the reading itself, and all associated Exceptions:

int matching = Files.readLines(new File("file.txt"), Charsets.UTF_8, new LineProcessor<Integer>(){
  int count;

  Integer getResult() {return count;}

  boolean processLine(String line) {
     if (line.equals("foo")
         count++;
     return true;
  }
});

If you don't actually want a result back out of the processor, and you never abort early (the reason for the boolean return from processLine) you could then do something like:

class SimpleLineCallback extends LineProcessor<Void> {
    Void getResult{ return null; }

    boolean processLine(String line) {
       doProcess(line);
       return true;
    }

    abstract void doProcess(String line);
}

and then your code might be:

Files.readLines(new File("file.txt"), Charsets.UTF_8, new SimpleLineProcessor(){
  void doProcess(String line) {
     if (line.equals("foo");
         throw new FooException("File shouldn't contain 'foo'!");
  }
});

which is correspondingly cleaner.

Cowan
If you post the code to use the callback in a nice way I'll accept :-)
drozzy
@drozzy: done, along with a little tip I'd use to reduce the boilerplate back again.
Cowan
Sorry now that I look at it - it is kind of cumbersome. It is not as simple as (for line in file): do stuff. I hope you know what I mean. But I did upvote your answer. Maybe I can use your piece of code as the guts of my implementation - which should take care of a lot of exception handling and proper file/buffer closing.
drozzy
It's hard. Google could have a version of the simple readLines which returns an Iterable<String>, not a List<String>, and doesn't keep in memory, but that's awkward -- fact is, all IO operations in Java can throw IOException, and it's not easy to have a clean API which completely hides that. What should the iterator do when it gets such an exception? Pretend there's no next item? Rethrow an unchecked exception? etc. Checked exceptions unfortunately make it hard to do in a 'least surprise' way.
Cowan
@Cowan: I'd say throw an unchecked exception. Silently failing is not really an option. Another option would be to log the failure at that point (but then your `Iterator` would probably just be a private API).
Kevin Brock
+1  A: 

Simple example using Files.readLines() from guava-io with a LineProcessor callback:

import java.io.File;
import java.io.IOException;

import com.google.common.base.Charsets;
import com.google.common.io.Files;
import com.google.common.io.LineProcessor;

public class GuavaIoDemo {

    public static void main(String[] args) throws Exception {
        int result = Files.readLines(new File("/home/pascal/.vimrc"), //
            Charsets.UTF_8, // 
            new LineProcessor<Integer>() {
                int counter;

                public Integer getResult() {
                    return counter;
                }

                public boolean processLine(String line) throws IOException {
                    counter++;
                    System.out.println(line);
                    return true;
                }
            });
    }
}
Pascal Thivent
Thanks for this, but now that I see it - it does look kind of like a lot of work :-)
drozzy
+1  A: 
  public static void main(String[] args) throws FileNotFoundException {
    Scanner scanner = new Scanner(new File("scan.txt"));
    try {
      while (scanner.hasNextLine()) {
        System.out.println(scanner.nextLine());
      }
    } finally {
      scanner.close();
    }
  }

Some caveats:

  • That uses the default system encoding, but you should specify the file encoding
  • Scanner swallows I/O exceptions, so you may want to check ioException() at the end for proper error handling
McDowell
A: 

Try looking at groovy!

Its a superset of Java that runs in hte JVM. Most valid Java code is also valid Groovy so you have access any of the million java APIs directly.

In addition it has many of the higher level contructs familiar to Pythonists, plus a number of extensions to take the pain out of Maps, Lists, sql, xml and you guessed it -- file IO.

James Anderson
Yeah, but I really don't like the fact that Groovy can contain java source code. I mean it's like taking really cluttered language syntax and duct-taping some "nice" syntax on top of it.
drozzy