abtest

A/B Test statistics

I am trying to do some statistical analysis of different A/B tests to see which alternative is better and have found conflicting information about this. First, I am interested in a couple different things: Tests that measure success by counting events, such as conversions or emails sent Tests that measure success by counting revenue T...

Best practice for assigning A/B test variation based on IP address

I am starting to write some code for A/B testing in a Grails web application. I want to ensure that requests from the same IP address always see the same variation. Rather than store a map of IP->variant, is it OK to simply turn the IP address into an integer by removing the dots, then use that as the seed for a random number generator...

Is it possible to do A/B testing by page rather than by individual?

Lets say I have a simple ecommerce site that sells 100 different t-shirt designs. I want to do some a/b testing to optimise my sales. Let's say I want to test two different "buy" buttons. Normally, I would use AB testing to randomly assign each visitor to see button A or button B (and try to ensure that that the user experience is con...

A/B testing on App Engine?

What would be the simplest implementation of an A/B testing system running on App engine? I'm especially keen towards performance implications of using Datastore for back-end (with looong query times), and database design. ...

How do you do A/B switching in a web app? (for deploying updates)

I currently have a mysite.com (A) which points to /public_html/ and a dev.mysite.com (B) which I created a /dev_html/ for that. I cloned the site but now I need a way of switching between them without having to copy all the files from one dir to another and without bringing the site down. I just want to switch the live site to B and then...

What's a good way of managing A/B testing-related source?

The current method employed at our company is to have a long-lived release branch that we all develop on, with developers checking in any non-ab test code into both the branch as well as the trunk, while ab test code only gets checked into the branch. Then, once we release, winning test cells get merged into the trunk, while the majorit...

A/B framework for Rails 3?

Is there anything like ABingo or Vanity that works in Rails 3? I know there's Google Website Optimizer, and Visual Website optimizer, but I need more power w/ regard to determining what is displayed, and what is in turn tracked than they can provide. If not, I'll even take thoughts on frameworks that are relatively simple enough that I...

A/B test with Google Web optimizer; what cookie tells me visitor got A or B

When using Google Website Optimizer A/B split test, is there a way I can read a cookie to see which version A or B did the user get? I need to know this from a page other than the control page. ...