Java file not written to stream with new line characters

Question

We're streaming a CSV file from a web service. It appears that we're losing the new line characters when streaming - the client gets the file all on a single line. Any idea what we're doing wrong?

Code:

 public static void writeFile(OutputStream out, File file) throws IOException {
    BufferedReader input = new BufferedReader(new FileReader(file)); //File input stream 
    String line;
    while ((line = input.readLine()) != null) { //Read file
        out.write(line.getBytes());  //Write to output stream 
        out.flush();
    }
    input.close();
} 

Answer 1

The readline method uses the newline chars to delimit what gets read, so the newlines themselves are not returned by readLine.

Don't use readline, you can use a BufferedInputStream and read the file one byte at a time if you want, or pass your own buffer into OutputStream.write.

Note that, like BalusC and Michael Borgwardt say, Readers and Writers are for text, if you just want to copy the file you should use InputStream and OutputStream, you are only concerned with bytes.

Answer 2

BufferedReader.ReadLine() does not preserve the newline. Thus you'll have to add it when writing it out

Answer 3

You can use a PrintWriter which offers a prinln() method. This will also save you from converting the string into an array of chars.

public static void writeFile(OutputStream o, File file) throws IOException {
   PrintWriter out = new PrintWriter(new OutputStreamWriter(o));
   BufferedReader input = new BufferedReader(new FileReader(file)); //File input stream 
   String line;
   while ((line = input.readLine()) != null) { //Read file
       out.println(line);  //Write to output stream 
       out.flush();
   }
   input.close();
}  

Answer 4

There are several things wrong with that code. It may also mutilate any NON-ASCII text since it converts via the platform default encoding twice - and for no good reason at all.

Don't use a Reader to read the file, use a FileInputStream and transfer bytes, avoiding the unnecessary and potentially destructive charset conversions. The line break problem will also be gone.

Answer 5

Don't use BufferedReader. You already have an OutputStream at hands, so just get an InputStream of the file and pipe the bytes from input to output it the usual Java IO way. This way you also don't need to worry about newlines being eaten by BufferedReader:

public static void writeFile(OutputStream output, File file) throws IOException {
    InputStream input = null;
    byte[] buffer = new byte[10240]; // 10KB.
    try {
        input = new FileInputStream(file);
        for (int length = 0; (length = input.read(buffer)) > 0;) {
            output.write(buffer, 0, length);
        }
    } finally {
        if (input != null) try { input.close(); } catch (IOException logOrIgnore) {}
    }
}

Using a Reader/Writer would involve character encoding problems if you don't know/specify the encoding beforehand. You actually also don't need to know about them here. So just leave it aside.

To improve performance a bit more, you can always wrap the InputStream and OutputStream in an BufferedInputStream and BufferedOutputStream respectively.

Answer 6

Any idea what we're doing wrong?

Yes. This line drops the "new line character"

while ((line = input.readLine()) != null) {

And then you write it without it:

 out.write(line.getBytes());

This this related question.

Answer 7

You can also use the New IO API (java.nio.*):

private static void copy(File source, File destination) throws IOException {
    long length = source.length();
    FileChannel input = new FileInputStream(source).getChannel();
    try {
        FileChannel output = new FileOutputStream(destination).getChannel();
        try {
            for (long position = 0; position < length; ) {
                position += input.transferTo(position, length-position, output);
            }
        } finally {
            output.close();
        }
    } finally {
        input.close();
    }
}

The loop is probably not needed, but the documentation of Channel.trasnferTo() says

may or may not transfer all of the requested bytes