How can I parse downloaded CSV data with Perl?

Question

I am trying to download a .csv file into an array and then parse each column line-by-line with Text::CSV. I have the following:

my @file = get("http://www.someCSV.com/file.csv") or warn $!;

my $CSV = Text::CSV->new();
$CSV->sep_char (',');

for ( @file ) {

  $CSV->parse($_) or warn $!;

      my @columns = $CSV->fields();
      print $columns[0] . "\n";
 }

Rather than downloading the file, saving it and then slurping it into a filehandle, I thought it would be more efficient to just put the CSV file into an array and parse from there. However, the above code doesn't work and I don't understand why. I get "Warning: something's wrong at test.pl"; not very helpful, to say the least.

This is more for learning. I don't have to do it this way but it's just bugging me why I'm not able to use Text::CSV with an array.

Answer 1

I'm assuming you're using LWP::Simple's get function there. That doesn't return a list of lines of the response body, but a string containing the response body. So at first, you probably meant:

my $content = get($uri);

You could now go and read that line by line, passing each line to Text::CSVs parse method. That might appear to work in some cases, but as CSV files may contain embedded newlines, it won't be very reliable.

Instead, let Text::CSV figure out what exactly is a line in the input by passing it a filehandle it can read from by itself. To do that, there's no need to save the file locally. You can just open a handle to a string:

open my $fh, '<', \$content or die $!;

my $csv = Text::CSV->new({ sep_char => ',' });
while (my $row = $csv->getline($fh)) {
    my @fields = @{ $row };
    ...
}