Google Analytics Content Experiments A/B testing server-side code without page refresh

Question

Basically, we want to A/B test 2 different page layout headers. There are some structural differences (it's not just switching out the CSS). We also don't want to wait for Google to flip the coin to determine which variation the visitor should see; instead, we want to choose the variation server-side and avoid a page redirect.

The code below does what I hoped it would; the UTMX cookie it generates looks identical to the one that the Google-supplied javascript would generate if I didn't omit it from the head tag.

Server-side PHP code:

public function setUtmxCookie($cookieName, $experimentsString)
{
    $domainHash = $this->getDomainHash($_SERVER['SERVER_NAME']);
    $cookieVal = $domainHash . $experimentsString;
    $expire = time() + 60 * 60 * 24 * 30;
    $domain = '.' . $_SERVER['SERVER_NAME'];
    setrawcookie($cookieName, $cookieVal, $expire, '/', $domain);
}

private function getExperimentsFromUtmxCookie($cookieName)
{
    if (isset($_COOKIE[$cookieName])) {
        $cookieVal = $_COOKIE[$cookieName];
        $experimentsArray = array();
        $experimentMatches = preg_split('/\./', $cookieVal);
        $domainHash = array_shift($experimentMatches); //remove the first item.  All that will remain in $experimentMatches is an array of experimentIds with their combos.
        foreach ($experimentMatches as $m) {
            $segments = preg_split('/:/', $m);
            $experimentsArray[$segments[0]] = $segments[1];
        }
        return $experimentsArray;
    }
    return array();
}

private function getExperimentsString($cookieName, $experimentId, $variation)
{
    $experiments = $this->getExperimentsFromUtmxCookie($cookieName);
    $experiments[$experimentId] = $variation;
    $experimentsString = '';
    foreach ($experiments as $key => $val) {
        $experimentsString .= '.' . $key . ':' . $val;
    }
    return $experimentsString;
}

Why isn't my Google Analytics Content Experiments dashboard showing any visitors to my experiment, then? Did I set the utmx cookie imperfectly? Other than setting the UTMX cookie, is GACE looking for anything else?

Carabineer

We're now thinking about using a different approach: use Amazon load-balancers (AWS ELB) to deploy the "original" to some servers and the "variation" to one other server. We'll use a 3 to 1 ratio (75% of visitors will get the original) since we use 4 load-balanced servers. We've enabled "stickiness" to keep individuals' experiences consistent. The huge advantage to this approach is that there won't be conditional (if/else) code to remove after the experiment afterwards. Just a normal merge from a branch back into trunk. – Carabineer

Answer 1

We've been using a totally different approach for the past couple months: Amazon load-balancers (AWS ELB) plus Google Analytics (not Content Experiments). (See my comment above.) As we hoped, it has greatly improved our experience with merging back to trunk.

_gaq.push(['_setCustomVar', 2, varName, varValue, 2]);//https://developers.google.com/analytics/devguides/collection/gajs/gaTrackingCustomVariables
_gaq.push(['_trackPageview']);//This must come AFTER the setCustomVar
//varName should be whatever you want to call the experiment
//varValue should be something like "original" for the original and "variation" for the variation.  We just use "trunk" and [name of variation branch].

Obvious drawbacks are that Google doesn't do the math for us (telling us whether the variation has statistically significantly outperformed the original) and that we can't easily run multiple experiments at once. We also wouldn't be able to have many variations (we'd need to add more load-balanced instances than we'd want).

But for our purposes (e.g. given how important it is to us not to have a page refresh), it has worked better than other approaches.

Carabineer

Nice approach, I know this is old but I assume that the customVariable and the trackPageView are only to track how many visitors each variation has, but,Did you use some event tracking or something to measure the effectiveness of one variation over the other? – Injun

I can't remember the details of what I was doing in 2012, but yes, I must have somehow recorded events (the numerator of the fraction). – Carabineer

Answer 2

@danmaz74 took an interesting approach to this using only Google Analytics on the client side as well:

https://github.com/danmaz74/ABalytics

Flying

Client-side solutions aren't very useful for A-B testing layouts (as mentioned in the question). – Carabineer