mechanize html scraping problem

Answer 1

Mechanize puts full content of a page into history, this may cause problems when browsing through many pages. To limit the size of history, try

@mech = WWW::Mechanize.new do |agent|
  agent.history.max_size = 1
end