How to convert dos/windows newline to unix newline in bash script?

Question

I bash, how do I convert dos/windows newlines to unix? It must be non-interactive, not like starting vi and typing something in, but a command on the command line that can be put in a script file. The dos2unix and unix2dos commands are not available on the system.

How do I do this with commands like sed/awk/tr?

Answer 1

Maybe something like

cat FILE | tr '\n\r' '\n' > FILE

Answer 2

Using AWK you can do:

awk '{ sub("\r$", ""); print }' dos.txt > unix.txt

Using Perl you can do:

perl -pe 's/\r$//' < dos.txt > unix.txt

Answer 3

You can only use tr to convert from DOS to Unix and you can do this only if CR appears in your file only as the first byte of a CRLF byte pair. This is usually the case. You then use:

tr -d '\015' <DOS-file >UNIX-file

You can't do it the other way round (with standard 'tr').

If you know how to enter carriage return into a script (control-V, control-M to enter control-M), then:

sed 's/^M$//'     # DOS to Unix
sed 's/$/^M/'     # Unix to DOS

where the '^M' is the control-M character.

Question: why can't you get dos2unix and unix2dos installed (or dtou and utod)? Or is this a restriction imposed by the person setting homework?

Answer 4

tr -d "\r" < file

take a look here for examples using sed

Answer 5

The solutions posted so far only deal with part of the problem, converting DOS/Windows' CRLF into Unix's LF; the part they're missing is that DOS use CRLF as a line separator, while Unix uses LF as a line terminator. The difference is that a DOS file (usually) won't have anything after the last line in the file, while Unix will. To do the conversion properly, you need to add that final LF (unless the file is zero-length, i.e. has no lines in it at all). My favorite incantation for this (with a little added logic to handle Mac-style CR-separated files, and not molest files that're already in unix format) is a bit of perl:

perl -pe 'if ( s/\r\n?/\n/g ) { $f=1 }; if ( $f || ! $m ) { s/([^\n])\z/$1\n/ }; $m=1' PCfile.txt

Note that this sends the Unixified version of the file to stdout. If you want to replace the file with a Unixified version, add perl's -i flag.

Answer 6

This problem can be solved with standard tools, but there are sufficiently many traps for the unwary that I recommend you install the flip command, which was written over 20 years ago by Rahul Dhesi, the author of zoo. It does an excellent job converting file formats while, for example, avoiding the inadvertant destruction of binary files, which is a little too easy if you just race around altering every CRLF you see...

Answer 7

You can find few commands at Convert files from DOS line endings to UNIX line endings to match your needs.