What is the best/cleanest way to implement A-B testing in asp.net mvc?
Asked Answered
H

6

27

What is the best and cleanest way to implement A-B testing in asp.net mvc? That is, when we make new changes to an asp.net mvc web site, we want to test the new html/css/js with a certain subset of visitors (defined on cookie, login id, etc) and then analyze some metrics (page response time, number of pages visited, $$$ in sales, etc) afterwards to measure the level of success of the changes.

I am looking for a clean way to implement a way of choosing what view (html/css/js, etc...) to render using asp.net mvc.

Hagler answered 24/9, 2009 at 4:32 Comment(0)
V
14

Check out FairlyCertain (http://www.fairtutor.com/fairlycertain/) when you get a chance. It's a .NET A/B library that you can pretty much just drop into your project and start writing tests.

Unlike the Javascript libraries from Google and VisualWebsiteOptimizer, everything happens on the server so you don't suffer any performance, user experience or SEO issues. I've been using it in my stuff for a while now and it works quite well.

Verduzco answered 5/11, 2010 at 9:59 Comment(9)
Finally found where it stores the data (in a text file - took me a while...). Might be worth adding a disclaimer if it's your own product. ;-)Estrange
Seems like this is targeting web forms - is there an MVC/Razor implementation?Engeddi
@JasonKester What license is this distributed under?Thames
is it open source? is there a version that can be configured to store data on a database instead of text file?Titus
Does anyone know the answers to any of these questions?Dierolf
Hades, the linked site answers all 3 questions above. "Yes, it's just C# so it will work in any framework", "public domain", and "of course it's open source, change whatever you like" are the simplest versions of those answers.Verduzco
6 years later and from what I can find, this is still the best option out there (for free, that is).Anders
Unfortunately doesn't seem to manage version ratios unlike A/B Test Master witch is a paid solutionGunas
Sorry, finally it happens to be under MIT licenseGunas
F
5

There is an A/B testing framework specifically for ASP.NET MVC. This is an open source software I wrote myself when, just like you, didn't find a free tool which works nicely with ASP.NET MVC and doesn't require much setup.

Fleet answered 9/4, 2014 at 23:54 Comment(1)
The link for the framework doesn't work anymore. Can you provide a new link?Headley
T
3

Google Content Experiments? It's a Javascript-based solution that doesn't require anything from your backend.

  1. You include Google's Javascript on your page
  2. The script randomly substitutes elements on your page as defined by your A/B test
  3. Google's site shows you a nice breakdown of the results...
Terresaterrestrial answered 24/9, 2009 at 15:57 Comment(1)
Perhaps being biased here, but just about anything that gives Google more information for free [as they are a company, just like anyone else; not a charity], and, on the other hand, is a JavaScript test engine [which suffers from lack of SEO/client performance support], is not a good solutionAnders
P
2

If you are using the spark view engine, you could probably do it with a variation of the theme filter (http://sparkviewengine.com/documentation/viewlocations#Extendingfilepatternswithdescriptorfilters). For each new visitor to the site, determine if you want them to see the existing or new version of the site and set a cookie. Wire up a descriptor filter that looks for the presence of the cookie and modify the view location to look in the folder containing the modified views. If an alternative view exists, the Spark engine will automatically render it in place of the "normal" view, otherwise it will render the normal view.

If you are using the normal WFVE, then the simplest way to manage this would be to define a folder under Views where your view alternatives live. When you want to provide an alternative view, you place it in a location that matches its position within the normal Views folder but rooted at the alternatives folder e.g. to provide an alternative to Views/Users/login.aspx place your new view at Views/Alternative/Users/login.aspx.

With a convention in place for locating your alternative views, you can extend the WebFormViewEngine and overload CreatePartialView / CreateView to inspect some aspect of the ControllerContext to determine whether to render the default or overloaded view and alter the path as appropriate e.g. changing .../Views/Users/login.aspx to .../Views/Alternative/Users/login.aspx.

Parrot answered 8/10, 2009 at 21:22 Comment(0)
S
1

I suggest you use Display Modes to achieve A/B testing.

But Display Modes just support simple problems by default.

If you already implement Display Modes in some other scenario. You can consider DisplayModeMatrix (just google it). It helps you use Display Modes more efficiency.

https://www.nuget.org/packages/DisplayModeMatrix/

Wth Display Modes you can simply delete/rename views after A/B testing to clean up your project.

Smelt answered 19/12, 2016 at 8:26 Comment(0)
F
-1

I think there isn't a ready to use solution for this and you will have to improvise.

Try to override your current functionality in well defined points without breaking it. Explicitly draw a border where your regular code and A-B testing code lives.

Inversion of control principle might help a lot here too (i.e. - controller factory could provide derived controller instead of original one). For views&partialviews - you could change viewengine so it would try to look for 'MyPartialViewAB.ascx' instead of 'MyPartialView.ascx'.

And it might be a good idea to take a look what performance counters are (in case you haven't).

Faxun answered 24/9, 2009 at 7:20 Comment(0)

© 2022 - 2024 — McMap. All rights reserved.