What resources are there for A/B split-testing in Rails?

Question

Some information on A/B split-testing:

• http://startuplessonslearned.blogspot.com/2008/09/one-line-split-test-or-how-to-ab-all.html
• http://www.37signals.com/svn/posts/1525-writing-decisions-headline-tests-on-the-highrise-signup-page

I could do this in a Rails app with a simple case or if statement in my views, but that gets to be a lot of repetition. Is there a gem to support this? Any design patterns that would help?

Answer 1

In ApplicationController:

options = ["option1", "option2", "option3"]
session[:option] ||= option.rand

In your views render a partial based on the one you want and the option chosen:

<%= render :partial => "foo#{session[:option]} %>

That way you guarantee that the user gets the same option for the entire session, across the whole site. Plus you can go back to the default partial by just setting the option to an empty string. You could even put an empty string in the array, or duplicate entries to change the weighting of how often each one is chosen.

You can use Google Website Optimizer to figure out which option did the best conversion. Check out their step-by-step walkthrough in their documentation, Quick Start Guide - Website Optmizer Help. That has the bits of JavaScript you'll need to add.

Answer 2

Have you seen 7 Minute ABs, an A/B testing plugin for rails?

http://github.com/paulmars/seven_minute_abs/tree/master

Answer 3

I just released A/Bingo, an OSS Rails plugin to do this.

You can see the comparison with Seven Minute Abs for details, but I think it is largely more easy to use.

• It supports tracking any event as a conversion. Seven Minute Abs only tracks clicks off the page you're currently viewing.
• It remembers what alternative a user saw, and only shows them that.
• It has lots of syntax sugar aimed at maximizing programmer productivity.
• It will do statistical significance tests for you.

Answer 4

Vanity is worth a mention. Some GA integration, custom metrics, clean code. Github here.