Do line endings differ between Windows and Linux?

Question

Hi all,

I am trying to parse the linux /etc/passwd file in java. I'm currently reading each line through the scanner class in java and then using string.split() to delimit each line.

The problem is that the line "list:x:38:38:Mailing List Manager:/var/list:/bin/sh" is treated by the scanner as 3 different lines: 1) "list:x:38:38:Mailing" 2) "List" 3) "Manager..." When I type this out into a new file that I didn't get from linux, the Scanner parses it properly.

Is there something I'm not understanding about new lines in linux?

Obviously a work around is to parse it without using scanner, but it wouldn't be elegant. Does anyone know of an elegant way to do it?

Is there a way to convert the file into one that would work with Scanner?

Please let me know.

Thanks, jbu

---

Not even two days ago: http://stackoverflow.com/questions/419291/historical-reason-behind-different-line-ending-at-different-platforms

EDIT

Note from the original author:

"I figured out I have a different error that is causing the problem. Disregard question"

Answer 1

From Wikipedia:

• LF: Multics, Unix and Unix-like systems (GNU/Linux, AIX, Xenix, Mac OS X, FreeBSD, etc.), BeOS, Amiga, RISC OS, and others
• CR+LF: DEC RT-11 and most other early non-Unix, non-IBM OSes, CP/M, MP/M, DOS, OS/2, Microsoft Windows, Symbian OS
• CR: Commodore machines, Apple II family, Mac OS up to version 9 and OS-9

I translate this into these line endings in general:

• Windows: '\r\n'
• Mac (OS 9-): '\r'
• Mac (OS 10+): '\n'
• Unix/Linux: '\n'

You need to make your scanner/parser handle the unix version, too.

Answer 2

Have you tried to remove all hidden characters but '\n'. What is the regex your using to split the lines?

Answer 3

Why not use LineNumberReader?

If you can't do that, what does the code look like?

The only difference I can think of is that you are splitting on a bad regex and that when you edit the file yourself, you get dos newlines that somehow pass your regex.

Still, for reading things one line at a time, it seems like overkill to use Scanner.

Of course, why you are parsing /etc/passwd is a hole other discussion :)

Answer 4

I figured out that I was doing something wrong with Scanner.

Though your answers were still helpful in understanding linux vs. windows.

Thanks guys.

Answer 5

The scanner is breaking at the spaces.

EDIT: The 'Scanning' Java Tutorial states:

By default, a scanner uses white space to separate tokens. (White space characters include blanks, tabs, and line terminators. For the full list, refer to the documentation for Character.isWhitespace.)

You can use the useDelimiter() method to change these defaults.

Answer 6

This works for me on Ubuntu

import java.util.Scanner;
import java.io.File;

public class test {
  public static void main(String[] args) {
    try {
      Scanner sc = new Scanner(new File("/etc/passwd"));
      String l;
      while( ( l = sc.nextLine() ) != null ) {
        String[] p = l.split(":");
        for(String pi: p) System.out.print( pi + "\t:\t" );
        System.out.println();
      }
    } catch(Exception e) { e.printStackTrace(); }
  }
}

Answer 7

Now I remember why I use BufferedReader on these occasions... :-)

Answer 8

You can get the standard line ending for your current OS from:

System.getProperty("line.separator")