Ruby: retrieve contents of URL as string

Question

For tedious reasons to do with hpricot, I need to write a function that is passed a URL, and returns the whole contents of the page as a single string.

I'm close. I know I need to use open-uri, and it should look something like this:

require 'open-uri'
open(url) {
  # do something mysterious here to get page_string
}
puts page_string

Please could anyone suggest what I need to add? I am new to Ruby, so finding all the open-uri documentation hard to parse.

Answer 1

open(url) do |f|
  page_string = f.read
end

See also the documentation of IO class

Answer 2

You can do the same without "open-uri":

require 'net/http'
require 'uri'

def open(url)
  Net::HTTP.get(URI.parse(url))
end

page_content = open('http://www.google.com')
puts page_content

Answer 3

require 'open-uri'
open(url) {|f|  #url must specify the protocol
str = f.read()
}

Answer 4

The open method passes an IO representation of the resource to your block when it yields. You can read from it using the IO#read method

open([mode [, perm]] [, options]) [{|io| ... }] 
open(path) { |io| data = io.read }