Linux Tool To Parse CSV files

Question

Is anyone aware of a standalone command line program that can be used to parse CSV files?

To do things like:

csvparse -c 2,5,6 filename

to extract fields from columns 2 5 and 6 from all rows.

It should be able to handle the csv file format: http://tools.ietf.org/html/rfc4180 which means quoting fields and escaping inner quotes as appropriate, so for an example row with 3 fields:

field1,"field, number ""2"", has inner quotes and a comma",field3

so that if I request field 2 for the row above I get:

field, number "2", has inner quotes and a comma

I appreciate that there are numerous solutions, perl, awk (etc..) to this problem but I would like a native bash command line tool that does not require me to invoke some other scripting environment or write any additional code (!).

Answer 1

This sounds like a job for awk.

You will most likely need to write your own script for your specific needs, but this site has some dialogue about how to go about doing this.

You could also use the cut utility to strip the fields out.

Something like:

cut -f 2,5,6 -d , filename

where the -f argument is the field you want and -d is the delimeter you want. You could then sort these results, find the unique ones, or use any other bash utility. There is a cool video here about working with CSV files from the command line. Only about a minute, I'd take a look.

However, I guess you could group the cut utility with awk and not want to use it. I don't really know what exactly you mean by native bash command though, so I'll still suggest it.

Answer 2

A quick google reveals an awk script that seems to handle csv files.

Answer 3

My FOSS CSV stream editor CSVfix does exactly what you want. There is a binary installer for Windows, and a compilable version (via a makefile) for UNIX/Linux.

Answer 4

I would bet $100 that there is no such csv-specific tool -- at least, not that comes pre-installed on a standard linux distro. Why don't you write it in C for yourself? I just wrote a CSV parser last week in a couple of hours. Yes, it handles quoted strings.

Answer 5

My gut reaction would be to write a script wrapper around Python's csv module (if there isn't already such a thing).

Answer 6

Try crush-tools, they are great at manipulating delimited data. It sounds like exactly what you're looking for.