What would be the simplest implementation of an A/B testing system running on App engine?
I'm especially keen towards performance implications of using Datastore for back-end (with looong query times), and database design.
+1
A:
Assuming you want to test different versions of your app, I would suggest using a simple bit of WSGI middleware. Build something that directs x% of users to one WSGI app, and the remainder to another, sharded by whatever suits - user ID, IP address, etcetera. This should be pretty straightforward to implement, and you can pile whatever you like on top of it.
Nick Johnson
2010-04-14 09:32:54
Doable, but kind of overly complicated. Read how easy A/B testing should be: http://www.bingocardcreator.com/abingo
Vladimir Dyuzhev
2010-04-14 13:43:25
Something like that is certainly possible in Python as well as ruby. I was going for a more general (though higher overhead) approach. :)
Nick Johnson
2010-04-15 09:43:17
+1
A:
A/B test requires to show page A to some users, while page B to some other users.
App Engine has nothing to do with it. App Engine is a way to deploy applications, not direct user along the pages.
It's the function of the web framework you use to serve one page or another based on user cookie/session.
In a simple way it could be done like this:
- Get user cookie
- Find it in datastore
- Found? Use the same set of pages (A or B) as the last time
- Not found? Choose A or B randomly, save the choice into datastore along with cookie
- (May be) Place the choice into session for fast access
Then, in specific controllers/views, based on selected A or B, serve/redirect user to page A or page B. Record the outcome (whatever your outcome is -- sale, registration, ...) into datastore.
That can be done for any web framework. You didn't even told which one you use ;)
Vladimir Dyuzhev
2010-04-14 13:40:21