open-uri

XML => HTML with Hpricot and Rails

I've never worked with web services and rails, and obviously this is something I need to learn. I've chosen to use hpricot because it looks great. Anyway, _why's been nice enough to provide the following example on the hpricot website: #!ruby require 'hpricot' require 'open-uri' # load the RedHanded home page doc = Hpricot(open("ht...

How do I make a POST request with open-uri?

Is it possible to make a POST request from Ruby with open-uri? ...

RSpec and Open-URI how do I mock raise a SocketError/TimeoutError

I want to be able to spec out that when Open-Uri open() calls either timeout or raise an exception such as SocketError I am handling things as expected, however I'm having trouble with this. Here is my spec (for SocketError): @obj.should_receive(:open).with("some_url").and_raise(SocketError) And the part of my object where I'm using ...

Ruby open-uri file locking

I had an issue where downloading to a local file was locking it until I killed the ruby script. On a hunch, I changed the form of the call and now it doesn't lock the file. Can someone tell me why the second form below doesn't lock the file but the first one does? This form locks the local file: open(ver_local_zip, "w+").write(open(r...

[Ruby] open-uri + hpricot & nokogiri don't parse html correctly

I'm trying to parse a webpage using open-uri + hpricot but it seems to be a problem in the parsing proccess as the gems don't bring me the things I want. Specifically I want to get this div (whose id is 'pasajes') in this url: http://www.despegar.com.ar I write this code: require 'nokogiri' require 'hpricot' require 'open-uri' docu...

Ruby's open-uri and cookies

I would like to store the cookies from one open-uri call and pass them to the next one. I can't seem to find the right docs for doing this. I'd appreciate it if you could tell me the right way to do this. NOTES: w3.org is not the actual url, but it's shorter; pretend cookies matter here. h1 = open("http://www.w3.org/") h2 = open("http:/...

Timeout Error with Hpricot in Rails Controller

Hey--I'm writing a basic Rails app that uses the digg API. I'm trying to parse the xml data that digg's api provides with hpricot, but when testing the page, the browser hangs until I eventually catch the Timeout::Error exception. Here's the code for the controller: require 'rubygems' require 'hpricot' require 'open-uri' appkey = 'htt...

Convert latin1 string to utf8?

Hello, how can I convert a string, that contains latin1 characters to utf8? The string is a document, that is opened by open-uri and that contains these special characters. Best regards ...

Recieving a 404 HTTPError on a working page in Ruby Script

This is my first time asking a question, please be gentle! I have a Rails application that handles content for a whole bunch of domains (over 100 so far). Each domain either points to where my app is hosted (Heroku, if you're interested), or the original place it was hosted. Every time a domain is ready, it needs to point to the herok...

Opening a WIKI URL with a comma using `open-uri`

I am running in to OpenURI::HTTPError: 403 Forbidden error when I try to open a URL with a comma (OR other special characters like .). I am able to open the same url in a browser. require 'open-uri' url = "http://en.wikipedia.org/wiki/Thor_Industries,_Inc." f = open(url) # throws OpenURI::HTTPError: 403 Forbidden error How do I escape...

Using open-uri how can I get the contents of a redirecting page?

I ultimately want to get data from this page: http://www.canadapost.ca/cpotools/apps/track/personal/findByTrackNumber?trackingNumber=0656887000494793 But that page forwards to: http://www.canadapost.ca/cpotools/apps/track/personal/findByTrackNumber?execution=eXs1 So when I use open (open-uri) to try and fetch the data, it throws a Ru...

Ruby HTML scraper written in Hpricot having trouble with escaped HTML

I am trying to scrape this page: http://www.udel.edu/dining/menus/russell.html. I have written a scraper in Ruby using the Hpricot library. problem: HTML page is escaped and I need to display it unescaped example: "M&M" should be "M&M" example: "Entrée" should be "Vegetarian Entrée" I have tried using the CGI library...

Ruby: retrieve contents of URL as string

For tedious reasons to do with hpricot, I need to write a function that is passed a URL, and returns the whole contents of the page as a single string. I'm close. I know I need to use open-uri, and it should look something like this: require 'open-uri' open(url) { # do something mysterious here to get page_string } puts page_string ...

Encoding error in content get from open-uri in ruby on rails

Hi, Im some cases when I use open to get a web page in ruby the content of the page has an encoding error. Exemple : open("http://www.google.com.br").read Chars like ç and ã are replaced by "?" How can I get the right chars ? thanks. ...