Segmentation fault in hpricot

Question

I'm using hpricot to read HTML. I got a segmentation fault error, I googled and some say upgrade to latest version of Ruby. I am using rails 2.3.2 and ruby 1.8.7. How to resolve this error?

Answer 1

Well, based on your own question, I'd say "Upgrade to the latest version of Ruby". However, I've also had problems with hpricot segfaulting, which seemed to be related to my usage of threading.

Answer 2

This appears to be an outstanding issue on the bug list. I have experienced it to. My theory is has to do with the HTML structure or bad/corrupt character in the file but I have not found where exactly.

Here are the links to the issues:

• http://github.com/why/hpricot/issues/#issue/10
• http://github.com/why/hpricot/issues/#issue/4

Answer 3

If you're free to choose your HTML parsing library, switch it. Why, the creator of Hpricot, recently posted that you should better use Nokogiri instead of HPricot, nowadays.

You may also have a look at HTTParty.

Answer 4

I'm having the same segfault issue but sadly can't consult the issues Dave cited above, even via Google cache -- from what I've been googling the parse.rb segfaults have to do with encoded entities or alt character sets (accented characters perhaps)

The sanitize lib encountered the same issue and posted a monkeypatch here: http://github.com/rgrove/sanitize/blob/1e1dc9681de99e32dc166f591343dfa60fc1f648/lib/sanitize/monkeypatch/hpricot.rb

Answer 5

From memory, since I last used it about a year ago:

Hpricot stores attributes in a fixed-size buffer, and some frameworks generate outrageously long hashes in document attributes. There's some static field you can set before parsing that lets you set the size of this buffer.

I remember it being fairly prominent in the docs on the webpage, though of course it's gone now.

Answer 6

I was trying to parse html pages with many unicode characters in them and Hpricot kept crashing. Finally, I used the monkey patch from sanitize and put it in the environment.rb for my rails application. There hasn't been a single crash since I added this patch:

http://github.com/rgrove/sanitize/blob/1e1dc9681de99e32dc166f591343dfa60fc1f648/lib/sanitize/monkeypatch/hpricot.rb