ruby regex, parsing html

Question

Hello all, I'm trying to parse some returned html (from http://www.google.com/movies?near=37130 )to look for currently playing movies. The pattern I'm trying to match looks like:
<span dir=ltr>Clash of the Titans</span>

Of which there are several in the returned html.

I'm trying get an array of the movie titles with the following command:
titles = listings_html.split(/(<span dir=ltr>).*(<\/span>)/)

But I'm not getting the results I'm expecting. Can anyone see a problem with my approach or regex?

Answer 1

It is considered Verey Bad generally to parse HTML with RegExs since HTML does not have regular grammar. See the list of links to explanations (some from SO) here.

You should instead use a designated HTML library, such as this

Answer 2

To parse HTML with Ruby, use Nokogiri or hpricot.

Answer 3

I didn't read the whole code you posted since it burned my eyes.

<span>.*</span>

This regex matches <span>hello</span> correctly, but fails at <span>hello</span><span>there</span> and matches the whole string. Remember that the * operator is greedy, so it will match the longest string possible. You can make it non-greedy by using .*? should make it work.

However, it's not wise to use regular expressions to parse HTML code.

1- You can't always parse HTML with regex. HTML is not regular.

2- It's very hard to write or maintain regex.

3- It's easy to break the regex by using an input like <span><a href="</span>"></a></span>.

Answer 4

(it doesn't appear that the sample html you posted actually has any examples of the pattern you're trying to match.)

Alicia is correct that regex against html is generally a bad idea, and as your requirements become more complex it will break down.

That said, your example is pretty simple..

doc.scan(/<span dir=ltr>(.*)<\/span/) do |match|
   puts match               
end 

As mentioned, .* is usually greedy (and I expected to have to account for that), but it appears that when used within scan, you don't get greedy behavior. I was able to match several of these patterns in a single document.