Opening and decompressing an XML URL in Rails

Question

I'm building a rails app that takes information about products from an XML datafeed hosted on a 3rd party server. This XML is sent gzipped, and I'm having serious difficulty in getting anywhere with it.

I've spent a fair bit of time with Google on this, but the results of my searching seem to be more about Sending Gzipped output rather than receiving a Gzipped input.

The closed I've come to a solution came from StackOverflow, but I'm still getting errors.

What I'm trying to do in the first instance is print the XML data to the browser, then I can start with the processing of it. Here's my current code:

   def load_data
     url = "http://xml.domain.com/datafeed/"
     xml_input = Net::HTTP.get(URI.parse(url))
     zstream = Zlib::Inflate.new
     @xml_output = zstream.inflate(xml_input)
     zstream.finish
     zstream.close
   end

The error I'm getting from it is:

 Zlib::DataError in Cron/get datafeedController#load_data

incorrect header check

I guess this means that the data isn't in the format that is expected, but I can't find information about how to do this properly anywhere. Two things I've ruled out is that the URL is valid and the response is Gzipped, but I'm stuck with how to get past this.

Any help would be greatly appreciated :-)

Answer 1

Sorted!

file = Net::HTTP.get(URI.parse(url))
gz = Zlib::GzipReader.new(StringIO.new(file))
whole_xml = gz.read

Then to load into Hpricot to do the XML parsing:

hp = Hpricot(whole_xml)