Options for full stack testing in Ruby on Rails

Question

There are quite a number of options for Full-Stack testing of Rails applications. Some use real browsers, some are headless, some don't run javascript at all.

Which tools do you use or recommend and why?

List of browser simulators or automators:

• Rails built-in support for integration and functional tests (no JS)
• Webrat
• Webrat::selenium
• Selenium
• Celerity (through Culerity)
• Watir
• ...

List of testing DSLs and frameworks:

• Rails defaults (assertions, ...)
• Shoulda
• Cucumber
• Capybara (unified DSL for several browser simulators)
• ...

Answer 1

I have used a bunch of things during my Rails career over the last few years.

Currently working on a pretty large Rails app on JRuby with very solid test coverage and our stack looks like the following.

Unit Testing:

• RSpec coverage of models, helpers, libraries and controllers. Controller coverage tends to be very high-level
• JSpec coverage for a project that is using some pretty cutting edge JS and HTML 5 wizardry

Functional Testing:

• Cucumber using Capybara and Culerity (we just converted from WebRat in order to get coverage of JS-heavy front-ends from Cucumber)
• Selenium that is now "legacy" and slowly being migrated to cucumber/capybara

Answer 2

In the project I'm currently working on, for testing the full stack we're using Cucumber with capybara as the driver.

The front end is very javascript heavy, I've tried a couple of headless browser drivers (capybara-envjs and akephalos), but both were incorrectly failing tests that passed in a real browser.

Answer 3

I'm using Cucumber + Capybara with a combination of selenium-webdriver and the rack-test driver. Capybara does a good job of abstracting the difference between in-browser tests that get driven by selenium-webdriver for test that can actually test that the JavaScript works in a browser but take a while to run and test that just run against the HTML produced by the controller/action, which run much faster but don't test JavaScript. This means that the same set of scenarios and step definitions can be run in a browser or not without having to maintain duplicate sets of step definitions.

Answer 4

All rails applications benefit from a good coverage of unit and functional tests, whether you use rspec or shoulda etc, is mostly a matter of personal preference. I'm a fan of shoulda mostly for the context blocks it provides, making setting up test scenarios much easier and clearer to understand.

I don't think browser simulators/automators are needed unless your application is quite javascript heavy. I'd recommend only using them to test javascript, and for that purpose it's definitely better to go with driving a real browser than to simulate. The app I'm working on now is quite javascript heavy and we're using cucumber along with watir/firewatir to run our cucumber tests in firefox for the javascript driven functions on our site.