Hpricot error parsing special characters in URI

Question

I'm working on a ruby script to grab historical stock prices from Yahoo, using Hpricot to parse the pages. This is mostly straighforward: the url is "http://finance.yahoo.com/q/hp?s=TickerSymbol" For example, to look up Google, I would use "http://finance.yahoo.com/q/hp?s=GOOG"

Unfortunately, it breaks down when I'm looking up the price of an index. The indexes are prefixed with a caret, such as "http://finance.yahoo.com/q/hp?s=^DJI" for the Dow.

The line:

ticker_symbol = '^DJI'
doc = Hpricot(open("http://finance.yahoo.com/q/hp?s=#{ticker_symbol}"))

throws this exception:

bad URI(is not URI?): http://finance.yahoo.com/q/hp?s=^DJI

Hpricot chokes on the caret (I think because the underlying Ruby URI library does). Is there a way to escape that character or force the library to try it?

Answer 1

The escape for ^ is %5E; you could do a straight substitution on the URL.

http://finance.yahoo.com/q/hp?s=%5EDJI

Answer 2

Well, don't I feel dumb. Five more minutes and I got this working:

doc = Hpricot(open(URI.encode("http://finance.yahoo.com/q/hp?s=#{ticker_symbol}")))

So if anyone else is wondering, that's how you do it. facepalm