How can I Parse an HTML response without response headers using Ruby

Question

How can I grab an HTML response when there are no response headers?

I've got this:

require 'hpricot'
require 'open-uri'

doc = Hpricot(open('http://192.168.100.1/phy.htm'))

The server in this case is a cable modem that is not returning HTTP Response headers.

The above code is failing with:

C:/Ruby/lib/ruby/1.8/net/http.rb:2022:in `read_status_line': wrong status line: "" (Net::HTTPBadResponse)

from C:/Ruby/lib/ruby/1.8/net/http.rb:2009:in `read_new'

from C:/Ruby/lib/ruby/1.8/net/http.rb:1050:in `request'

from C:/Ruby/lib/ruby/1.8/open-uri.rb:248:in `open_http'

Answer 1

So if your server is not HTTP compliant, maybe you should drop HTTP idea altogether and work with Socket directly...

Then you can use hpricot with a string returned.

Answer 2

I have a similar problem and am reluctantly comming to the same conclusion. I'm talking to a device that returns raw XML without bothering to wrap it in an html response.

There is a work around:

resultxml = %x{wget -qO- "http://192.168.100.1/state.xml"}