A/B testing with ember.js

Question

I've found absolutely nothing on Google with regard to A/B testing with a client-side framework such as ember.js.

The goal is to serve up adjusted content (different nav items, header phrasing etc.) in order to A/B test our UI/UX. I should note that nothing significant (i.e. sitemap) is changing, just some minor presentational aspects.

There are several possible approaches, namely using different view templates / helper snippets, or serving up a different stylesheet. Both have advantages and challenges, and ideally the same visitor would always be served the same version. Results would be fed through a service like Mixpanel.

I fear I may have to roll my own solution here, but would love to hear any suggestions / pointers.

Republicanize

Answer 1

At their root, most A/B javascript testing frameworks cookie a user as being in the "A" or "B" group, give you a way to ask if a user is "A" or "B" and report "results" back to a service to measure. This can plug into Ember or other client-side frameworks in a way that is fairly orthogonal to the framework.

I would recommend exposing the "A"- or "B"-ness of the user as a property on your user (in Ember, probably your UserController). Then you can use your framework's standard branching or conditionals to render the "A" UI or the "B" UI.

Wheelwork

Answer 2

I have actually built a pretty robust A/B Testing tool using Ember for my startup. We are actually thinking of open sourcing it if there was a demand for it. I can let you know the basic idea of how it works for now though.

I have landingPage objects, that can then have a bunch of A/B tests associated with the, When a user comes to the landing page, they are assigned a cookie, and for each A/B test that are assigned either A or B.

I have used two different approaches within jade to handle A/B testing.

For styling type things, I use something like this

and set the .css property in the view to either test-a or test-b

or if it is for text I will do something like this

{{view view.landingPageText}}

and the landingPageText would be set to either the text for A or the test for B.

This thing also dynamically sets up mixpanel, mailchimp, and uses parse.com and node. You can see the code in action here.

http://golf.nextstudioapps.com/

Rhiamon

Hey, are you guys using this? Did you end up open-sourcing it / have you guys written any more about it? I'm thinking through approaches to AB testing with Ember and am definitely interested! – Lithopone

@SherwinYu - I have pu thte code up here github.com/bwship/ember-ab-testing-suite I have to do a bit more cleanup but it is a really solid baseline for this. – Rhiamon