Ruby's open-uri and cookies

Question

I would like to store the cookies from one open-uri call and pass them to the next one. I can't seem to find the right docs for doing this. I'd appreciate it if you could tell me the right way to do this.
NOTES: w3.org is not the actual url, but it's shorter; pretend cookies matter here.

h1 = open("http://www.w3.org/")
h2 = open("http://www.w3.org/People/Berners-Lee/", "Cookie" => h1.FixThisSpot)

Update after 2 nays: While this wasn't intended as rhetorical question I guarantee that it's possible. Update after tumbleweeds: See (the answer), it's possible. Took me a good while, but it works.

Answer 1

im pretty sure theres no support for cookies in open-uri so consider using httpclient http://raa.ruby-lang.org/project/httpclient/ or something like mechanize instead http://mechanize.rubyforge.org/mechanize/

Answer 2

You need to add a "Cookie" header.

I'm not sure if open-uri can do this or not, but it can be done using Net::HTTP.

# Create a new connection object.
conn = Net::HTTP.new(site, port)

# Get the response when we login, to set the cookie.
# body is the encoded arguments to log in.
resp, data = conn.post(login_path, body, {})
cookie = resp.response['set-cookie']

# Headers need to be in a hash.
headers = { "Cookie" => cookie }

# On a get, we don't need a body.
resp, data = conn.get(path, headers)

Answer 3

I thought someone would just know, but I guess it's not commonly done with open-uri. Here's the ugly version that neither checks for privacy, expiration, the correct domain, nor the correct path:

h1 = open("http://www.w3.org/")
h2 = open("http://www.w3.org/People/Berners-Lee/",
          "Cookie" => h1.meta['set-cookie'].split('; ',2)[0])

Yes, it works. No it's not pretty, nor fully compliant with recommendations, nor does it handle multiple cookies (as is).

Clearly, HTTP is a very straight-forward protocol, and open-uri lets you at most of it. I guess what I really needed to know was how to get the cookie from the h1 request so that it could be passed to the h2 request (that part I already knew and showed). The surprising thing here is how many people basically felt like answering by telling me not to use open-uri, and only one of those showed how to get a cookie set in one request passed to the next request.

Answer 4

Depending on what you are trying to accomplish, check out webrat. I know it is usually used for testing, but it can also hit live sites, and it does a lot of the stuff that your web browser would do for you, like store cookies between requests and follow redirects.