How to test a ruby application which uses mechanize

Question

Hi, I wrote a small program that uses Mechanize to traverse a site.

I want to write tests for it, but don't want it to actually go log onto the site every time I run the tests. I would like to mock the internet, so that when it goes to some site, it simply returns the stored results.

Here is a small example, pretend my code's purpose was to pull links off of the google homepage, so I write a test to ensure the first link my code finds has the text "Images". I might write something like this:

require 'rubygems'
require 'mechanize'
require 'test/unit'

def my_code_to_find_links
  google = WWW::Mechanize.new.get('http://www.google.com')
  # ...
  # some code to figure out which liks it wants
  # ...
  google.links
end

class TestGoogle < Test::Unit::TestCase
  def test_first_link_is_images
    assert_equal 'Images' , my_code_to_find_links.first.text
  end
end

How do I mock google.com so that I can test my_code_to_find_links without all the overhead of actually accessing the internet?

thanks -Josh

Answer 1

If you are writing functional tests and the server doesn't need to be anything fancy, you can write a little HTTP server that runs just for the test; the test code configures the code-under-test to contact the little HTTP server on localhost:.

I recommend WEBrick. Putting up an HTTP server is a snap.

Answer 2

If you are writing unit tests then you are looking at the wrong end of the stick, you don't need to mock google.com but you should mock/stub Mechanize object, ie. a mechanize object is created, get method is called on the object with parameter http://www.google.com. And yes, you do need to test the operation you are performing on the result you get by visiting the page which you can easily do by a stubbed response and then doing all the test. For example if you are using mocha, then

mechanize = stub('WWW::Mechanize')
WWW::Mechanize.stubs(:new).returns(mechanize)
mechanize.stubs(:get).with('http://www.google.com').returns('What ever out put you are expecting so that you can test')

I don't think that you really need to test whether your request goes to google.com or not because that has already been tested in Mechanize gem.

On another note if you are doing integration/cucumber tests then you can use sinatra to fake the site. Sinatra makes it very easy to write a single page script and run that script as a website.

Edit:

If you are not using mocha then you need to install it and require it in your tests

sudo gem install mocha

And in your tests

require 'rubygems'
require 'mocha'

Answer 3

Use Fakeweb to stub out the internet responses.

For the Google example, first go to the website and save the html of the page you desire. In this case, let's assume you saved www.google.com via your browser or curl. Add the Fakeweb gem to your test.rb file

Then your code is

stream = File.read("saved_google_page.html")
FakeWeb.register_uri(:get, "http://www.google.com", :body => stream, :content_type => "text/html")

When you do your standard Mechanize call of

agent = Mechanize.New
page = agent.get("http://www.google.com/")

Fakeweb will return the page you saved with the content_type headers set in a way that Mechanize will think it accessed the internet successfully. Make sure that content_type header is set since otherwise Mechanize treats the response as Mechanize::File instead of Mechanize::Page. You can test that it's fully working by running tests on your machine with the network connection unplugged.

p.s. I'm answering this 6 months after the question was asked since this is the top result in Google but it's unanswered. I just spent 30 minutes figuring this out myself and thought I'd share the solution.