Prusak.com

Serving up my thoughts since 2004

Author: ophir (page 3 of 8)

Google Analytics Referrer Override

A couple of weeks ago I posted about a known issue using A/B experiments in Google Website Optimizer.
http://www.prusak.com/google-website-optimizer-and-referer-data/

Basically, when conducting an A/B experiment in Website Optimizer, the original referrer data is lost because of the JavaScript redirect to the test versions. This means that the traffic source will be wrong (it will say direct instead of the true traffic source).

This issue isn’t just a problem with website optimizer. Any landing page that uses JavaScript to redirect will loose the original referrer data. For example if your site provides easy to remember URLs like www.mysite.com/freeshipping and then uses JavaScript to redirect users to a different URL.

I wrote in my previous blog post that I wished Google Analytics provided an easy way to over ride the referrer value (and not use the document.referrer value).

It seems I spoke to soon.

Google Analytics DOES have a way to overwrite the referrer data.

It just doesn’t seem to be documented anywhere 🙂

I planned on writing my own hack to override the referrer value in Google Analytics. I was doing some reverse engineering of the ga.js code and found this interesting tidbit:

a._setReferrerOverride=function(b){a.yb=b}

I’m still running some tests, but based on my initial results, the _setReferrerOverride function provides a simple way to manually set the referrer value.

So now, we can actually capture the original referrer even when the visitor is redirected. Here’s how:
1 – Store the original referrer data in a cookie named realreferrer right before we redirect
2 – On the test pages, look for the realreferrer cookie. If it exists use it to over ride the referrer value in Google Analytics and then delete the cookie (so it won’t mess up page views that weren’t a result of a redirect from the original version).

Here’s the code I’m currently using in my tests.
Add this on the original page before the redirect (the GWO control script):


On the test page, add this code before any GA or GWO calls:


Then change your GA code from this:


to this:


Please let me know if this works for you !

Update:
I just found another (and possibly better) solution. See:
http://www.prusak.com/google-ananlytics-initial-referrer-update/

Update #2 [Oct 2010]:
It's been over two years since my initial posting and this issue still exists. A few people have asked about an async version so here it is.

All of my custom JS stays the same.

Here is the change for the async code:

Before

_gaq.push(['_trackPageview']);

After

if (realreferrer.length > 0) {
    _gaq.push(['_setReferrerOverride',realreferrer]);
}
_gaq.push(['_trackPageview']);

Please note my blog messes up the single quotes for code snippets when copying / pasting.

- Ophir

Does Website Optimizer Append or Merge?

When running an A/B experiment Google Website Optimizer appends any parameters from the original page when redirecting to one of the test pages.

For example, the original page is index.php and the test page is index2.php.
If a user clicks on index.php?name=fred they will be redirected to index2.php?name=fred

So what happens if the test page URL already has a query parameter in it?

For example, the original page is index.php and the test page URL is index2.php?section=b
What will happen if the user clicks on index.php?section=d ?

About seven months ago I had a client with this exact scenario so I tested it.
The user was directed from index.php?section=d to index2.php?section=b&section=d
I found a work-around and didn’t think about it after that.

A few weeks ago I found out that Google was indeed aware of this “issue” and fixed the GWO code 🙂
Now GWO will merge the parameters from the two URLs, not just append them. The parameters defined during setup for the test page URL will get precedence over the parameters from the URL the user clicked on.

For example, the original page is index.php and the test page URL is index2.php?section=b
What will now happen if the user clicks on index.php?section=d&cat=22 ?

They’ll be redirected to index.php?section=b&cat=22

This doesn’t effect 99.99% of all A/B tests, but if it does effect you, Google’s fix is a life saver 🙂

Google Help Pages Feature Request

I’ve been doing a lot of reading on the Google Website Optimizer help pages:
http://www.google.com/support/websiteoptimizer/

For the most part, the help pages are kept up to date, but every now and then something falls between the cracks and I get the feeling the page does not reflect the latest updates.

I was just thinking how every Google help page should really have a time stamp, letting readers know when the page was first created and when it was last updated.

Something like:
This page was created on Sep 20th, 2007 and last updated Oct 24th, 2008.

Any thoughts?

Awesome New Features on Google Analytics

Google just announced some awesome new features on the official Google Analytics blog:

http://analytics.blogspot.com/2008/10/more-enterprise-class-features-added-to.html

Most of the features are “being added to all accounts in the coming weeks”.
This includes advanced segmentation, custom reports, motion charts, and a new Account Management Dashboard.

They’re also going to eventually be releasing a data API, but if you really wanted to, you can already grab the Google Analytics data manually.

I’m impressed.

Avinash also has a great posting on the advanced segmentation function.

Google Website Optimizer and Referer Data

Google Website Optimizer A/B experiments use JavaScript to redirect the visitor to a test pages.

When a visitor is redirected with JavaScript, you loose the original HTTP referer data (the page that did the redirecting now becomes the HTTP referer)

Here’s an example:
Lets say the original page is page.html and we have a test page page_1.html

Before the test, page.html got 1,000 visitors a day coming from Google searches (organic traffic).

So we have Google -> page.html

Now that the test is running you’ll have 500 visitors to page.html:
Google -> page.html

and 500 to page_1.html
Google -> page.html (JavaScript redirect) -> page_1.html

If the web analytics code is after the redirect on page.html (which is what Google tells you to do) then data based on the HTTP referer will be incorrect (traffic source, keywords, etc).

If you REALLY want to capture the HTTP referer related data you can put the analytics code before the redirect. You’ll now be able to capture the original HTTP referer data, but you’ve also just inflated the page views.

I’ve solved this issue for some in-house code that uses the referer data by simply storing the HTTP referer value in a cookie right before the redirect, and then using the cookie value on the test page.

Unfortunately Google Analytics doesn’t do this.

Here’s a feature request for the Google Analytics team:

Add a way pass in the HTTP refer data manually.

I’m sure Website Optimizer isn’t the only tool using JavaScript redirects from a landing page.

Back from the Dead

Hi All,

I just found out today that www.prusak.com has been down for about 48 hours 🙁

I’ve been with my hosting provider – www.DreamHost.com – since 1999 and they’ve been rock solid so I never had any reason to setup any site monitoring services.

Also, I use Google apps for prusak.com email so that was working fine.

After 6 hours I finally got a reply to my urgent help ticket (they don’t have a number you can call for phone support) stating that:

Your domain is not pointed to our nameservers. Due to that, we cannot
update your DNS information if/when we need to change IPs (such as in the
cases of failing over pf services, moving apache instances, or
circumventing DDOS attacks).

Due to the sheer number of sites we host (700,000+) its not particuarlly
plausable to email our customers these changes, as <1% of our customer
base actually manage their own IPs.

Give me a break!

1 – It’s trivial to check which customers manage their own IPs
2 – When you do need to change IPs (which really shouldn’t be that often) I’m guessing only a fraction of your 700,000 sites are affected.

On a side note:
– I changed my DNS to dreamhost (so this won’t happen again)
– I signed up for a few free web site monitoring services. The only one which kept on sending me email while the site was down is BasicState.com (I figure I own them a free link). I’m not crazy about the user interface, but it works.

– Ophir

Are You Optimizing The Wrong Pages?

What is the best page to start optimizing?
This question comes up quite often in conversion rate optimization.

It’s an excellent question that really deserves a detailed (and lengthy) answer, but today I want to tackle a specific misconception about choosing a page that I’ve seen many people make (and even some books).

The misconception is that “It’s always better to test pages that have more traffic“.

Here’s a simple example with some numbers to explain my point.

Lets say you have an ecommerce site with these stats:

  • 10,000 visitors a month (which land on the homepage)
  • 3,000 visitors a month who reach your product detail page (the one with the add to cart button)
  • 600 visitors a month buy (convert)

This means you have a 6% conversion rate for homepage visitors and a 20% conversion rate for visitors who reach your product detail page.

All things being equal, do you start testing on your homepage or the product page?
Lets run the numbers and see.

Assuming you run a simple test with just two variations (the control/original and a single test version) and that you’ll be able to get a 15% (relative) increase in conversions from your changes, here are the results:

The control group
5,000 Homepage visitors
1,500 product page visitors
300 conversions

Testing the Homepage
Your change to the Homepage causes 15% more visitors to reach the product page which leads to 15% more conversions:
5,000 Homepage visitors
1,500 * 1.15 = 1,725 product page visitors
300 * 1.15 = 345 conversions

Using my simple Conversion Confidence Calculator it’s showing a 93% confidence level (screen shot below).

Testing the product page
Your change to the product page causes 15% more visitors to “add to cart” which leads to 15% more conversions:
5,000 Homepage visitors
1,500 product page visitors
300 * 1.15 = 345 conversions

Using my simple Conversion Confidence Calculator it’s now showing a 95% confidence level (screen shot below).

In terms of statistical significance, I’ll actually get results a bit faster testing the product page which has far less traffic than the homepage.

So, am I getting something wrong or is this a general misconception?

– Ophir

Always Be Testing: The Complete Guide to Google Website Optimizer

There’s a new book in town.

Always Be Testing:
The Complete Guide to Google Website Optimizer

While the book was being written a few months ago, I had the opportunity to discus the technical side of using Google Website Optimizer with one of the authors, John Quarto-vonTivadar.

The book mentions me in a few places and lists www.prusak.com in the resources section 🙂

I’m currently working on something and I can’t go into details but I’ll have an update soon.

– Ophir

Google Website Optimizer Video Tutorial

Google Website Optimizer Tutorial
A few months ago a created a Google Website Optimizer Video Tutorial

I really didn’t do much with it, and since then they’ve slightly updated the interface.

The good thing is that it provides a step-by-step guide explaining exactly how to setup a test, as well as some interesting background information and common mistakes to avoid.

If you find it helpful, please let me know.

Check it out Here.

Ophir

Conversion Confidence Level Calculator

One of the most common questions when setting up an experiment in Google Website Optimizer is how long will the experiment take?

In other words, how long will it take to have “enough” data.

The official Google Website Optimizer docs say:
https://www.google.com/analytics/siteopt/siteopt/help/techoverview.html

If one combination emerges as a clear winner, in the future it will exceed the performance of any of the other combinations with a 95% confidence or better.

But what do you do if you want to estimate the experiment length in advance?

Just use my conversion confidence level calculator

Here is a simple example:

Your site gets 2,000 visitors a week.
If you’re doing a simple A/B split with just two versions (original and test A) then each one gets 1,000 visitors a week.

Your current conversion rate is 2% * 1,000 visitors = 20 conversions a week for the original version.
You are hoping to get a 50% improvement for a 3% conversion rate * 1,000 visitors = 30 conversions a week for the test version.

Here’s what it will look like. The fields in yellow are the fields you enter.

The confidence calculator shows an 84% confidence level.
Not bad, but your boss says she wants at least a 95% confidence before declaring a winner.

You’ll see the confidence level for twice the traffic (two weeks worth of data) is 95%.
This means that if you get the same conversion rates and double the traffic, you’ll have a 95% confidence level.

– Ophir

Older posts Newer posts

© 2024 Prusak.com

Theme by Anders NorenUp ↑