Proper way to parse this line with FasterCSV?

Question

I have the following line in a CSV file that's giving me issues when parsing:

312,'997639',' 2','John, Doe. "J.D." ',' ','2000 ',' ','Street ','City ','NY','99999','','2010-02-17 19:12:04','2010-02-17 19:12:04';

I'm parsing with the following parameters:

FasterCSV.foreach(file, {:headers => true, :quote_char => '"', :col_sep => "','"} ) do |row|

However, it's blowing up on rows like the one above due to the "J.D" inside a row column. How do I properly parse that line with FasterCSV?

Thanks!

Answer 1

It looks to me like your :quote_char should be ' and your :col_sep should be ,. In that case:

FasterCSV.foreach(file, {:headers => true, :quote_char => "'", :col_sep => ','} ) ...

Answer 2

You can't do that. FasterCSV only allows one choice of quote character, and your application needs two. There isn't a way to do cute stuff like pass in a regex instead of a character because FasterCSV precompiles matchers with the quote character escaped as follows:

# prebuild Regexps for faster parsing
esc_col_sep = Regexp.escape(@col_sep)
esc_row_sep = Regexp.escape(@row_sep)
esc_quote   = Regexp.escape(@quote_char)
@parsers = {
  :any_field      => Regexp.new( "[^#{esc_col_sep}]+",
                                 Regexp::MULTILINE,
                                 @encoding ),
  :quoted_field   => Regexp.new( "^#{esc_quote}(.*)#{esc_quote}$",
                                 Regexp::MULTILINE,
                                 @encoding ),
  ...
}

Answer 3

I haven't been able to bend FasterCSV to work the way I need it to with this data so the end result was simply requesting a new dump of the data with proper CSV output. Thanks for the attempts!