Please explain (with examples) what is meant by A/B testing of a website.
+8
A:
In the context of a web site it means that some users of the site/webapp get slightly modified version(s) and you verify how each modification affects the usability/user's behavior compared to baseline "control" version.
As always with A/B testing, which is a statistical technique, you need a large enough sample to properly assess impact of change.
This was done by both Amazon and Google among well known web companies.
A VERY good article (with marketing angle) on web A/B testing is : http://www.clickz.com/3349901
In the interest of completeness, here's the full A/B testing definition from Wikipedia:
A/B testing or bucket testing is a method of marketing testing by which a baseline control sample is compared to a variety of single-variable test samples in order to improve response rates. A classic direct mail tactic, this method has been recently adopted within the interactive space to test tactics such as banner ads, emails and landing pages.
Significant improvements can be seen through testing elements like copy text, layouts, images and colors. However, not all elements produce the same improvements, and by looking at the results from different tests, it is possible to identify those elements that consistently tend to produce the greatest improvements.
Employers of this A/B testing method will distribute multiple samples of a test, including the control, to see which single variable is most effective in increasing a response rate or other desired outcome. The test, in order to be effective, must reach an audience of a sufficient size that there is a reasonable chance of detecting a meaningful difference between the control and other tactics
DVK
2010-06-09 03:49:22
so it's not possible without user's input. it's not a job of alone person.
metal-gear-solid
2010-06-09 03:52:17
+1 - I've already read Wikipedia's page then asked question here to understand in simple words with example. Is A/B testing related to Designer and users input? not related to developer? Can you give any example to explain A/B testing? IS it mean we should release two version of website?- Thanks
metal-gear-solid
2010-06-09 03:55:51
@metal-gear-solid - I updated with an article on clickz.com that provides examples.
DVK
2010-06-09 03:59:52
It is a method of automatically testing the performance of your website versions with respect to each other. The whole point, is to evaluate the end user responses to changes in your site.
Scott P
2010-06-09 04:44:45
+2
A:
Disclaimer: I own an A/B testing startup called Visual Website Optimizer http://visualwebsiteoptimizer.com/
A/B testing is a method whereby you present two or more different versions of a website to visitors. For example, suppose you have a website and then your boss/partner suggests that moving sign up form from left to right may increase total number of signups because it reduces friction. Now either you can have faith in god and implement the suggestion right away or you can be curious and make a version of page where form is on the left. Then you redirect 50% of traffic to right-form and 50% to left-form. Then you observe which version actually gets more sign ups.
The issue becomes a bit complicated because you have to statistically prove that one version is better than the other. For that you can use online calculators such as the one available on my site http://visualwebsiteoptimizer.com/ab-split-significance-calculator/
If you are looking for real-world case studies, again there are lots of them on my site http://visualwebsiteoptimizer.com/case-studies.php
If you have any additional questions, please feel free to ask. I LOVE this topic :)
Paras Chopra
2010-06-09 15:00:25