FasterCSV: Read Remote CSV Files

Question

I can't seem to get this to work. I want to pull a CSV file from a different webserver to read in my application. This is how I'd like to call it:

url = 'http://www.testing.com/test.csv'
records = FasterCSV.read(url, :headers => true, :header_converters => :symbol)

But that doesn't work. I tried Googling, and all I came up with was this excerpt: Practical Ruby Gems

So, I tried modifying it as follows:

require 'open-uri'
url = 'http://www.testing.com/test.csv'
csv_url = open(url)
records = FasterCSV.read(csv_url, :headers => true, :header_converters => :symbol)

... and I get a can't convert Tempfile into String error (coming from the FasterCSV gem).

Can anyone tell me how to make this work?

Answer 1

I would retrieve the file with Net::HTTP for example and feed that to FasterCSV

Extracted from ri Net::HTTP

 require 'net/http'
 require 'uri'

 url = URI.parse('http://www.example.com/index.html')
 res = Net::HTTP.start(url.host, url.port) {|http|
   http.get('/index.html')
 }
 puts res.body

Answer 2

require 'open-uri'
url = 'http://www.testing.com/test.csv'
open(url) do |f|
  f.each_line do |line|
    FasterCSV.parse(line) do |row|
      # Your code here
    end
  end
end

http://www.ruby-doc.org/core/classes/OpenURI.html http://fastercsv.rubyforge.org/

Answer 3

I would download it with rio - as easy as:

require 'rio'
require 'fastercsv'

array_of_arrays = FasterCSV.parse(rio('http://www.example.com/index.html').read)

Answer 4

You just had a small typo. You should have used FasterCSV.parse instead of FasterCSV.read:

data = open('http://www.testing.com/test.csv')
records = FasterCSV.parse(data)

Answer 5

I upload CSV file with Paperclip and save it to Cloudfiles and then start file processing with Delayed_job.

This worked for me:

require 'open-uri'
url = 'http://www.testing.com/test.csv'
open(url) do |file|
  FasterCSV.parse(file.read) do |row|
    # Your code here
  end
end