Prusak.com

I came across this question on the Google Website Optimizer group:

Hi, I run a website where I sell 7 different products each at a different price level.
I am wondering whether there’s a way so that I can track the changes I make in terms of their effect on my overall revenue?

From a marketing perspective, this raises an interesting question.
Will the test version with the best conversion rate always bring the most revenue?

If what you are testing has no relationship to what you are selling, then probably yes.

On the other hand, lets say you want to test what product you should be highlighting.
In such a case, measuring overall revenue makes sense.

So here is how I would do it…

Continue reading

One of the limitations of Google Website Optimizer is that you have no control over which combination will be displayed to a user.

Wouldn’t it be great to have total control over which variation is displayed in each section?

For example, lets say you want to test a new navigation scheme that required testing of the page header and footer, but they had to be in sync with each other. In other words, header #1 must be displayed with footer#1, header #2 must be displayed with footer#2 and so forth.

This “hack” gives you the ability to read and write which variation is displayed in each section.

Continue reading

I just received an email with this question:

I’m trying to set up an GWO A/B test, but I’m having a problem
completing the set-up of the test because the confirmation page
requires a login.  It’s not a public page because there’s sensitive
order info on there.   So we’ve got the CWO code posted on the
confirmation page but GWO can’t validate it because it’s not
a public page.   

However, when a real order comes through, the Javascript will
fire and it should work okay. Any idea how I can complete the set-up
of the test without going through successful validation?  Right now I’m
stuck because I can’t hit the continue button in GWO to get past the
“Validate pages” step.

Jonathan

So I checked and indeed Jonathan is right.
You can’t continue without a “live” conversion page that is publicly accessible.

The solution
Create a dummy conversion page that is publicly accessible. Enter the dummy page URL during the experiment setup and simply copy the conversion page javascript code to both the dummy page and the real page.

Once an experiment is live, Google Website Optimizer doesn’t care what the URL of the conversion page is. The conversion event will be triggered when the javascript code is executed, regardless of the page URL (if it’s not on the same domain – you need to add some code).

When creating a Multivariate experiment, this is a non-issue because you can upload the conversion page from your local file-system, bypassing the need for the file to be publicly accessible.

Hope that helps
– Ophir

I’ve written in past posts about how to get all of the great GA data for each of your combinations in GWO.

A couple of months ago I ran an experiment that showed me why doing so isn’t just a “nice thing to have”, but a necessity.

I was comparing two significantly different versions of a promotion page (whole page split). You can see them here:
Old (control) Version
New Version

After a couple of weeks of testing, GWO was reporting that the new version was performing 7.03% worse than the old version. I was a bit surprised because of the radical difference between the two pages.

At that point I started looking at numbers GA provided as well. At first I didn’t see anything out of the ordinary. Then I took a look at the page performance segmented by visitor type (New / Return visitors) and my eyes opened wide.

For new visitors, the new version of the page was performing much better (over 40% improved conversion rate) and for return visitors the old version of the page was performing much better (over 20% better conversion rate).

Going forward, we now show new users the new version of the page and return users the old version. I have a few theories on why we’re seeing this behavior, but that’s a bit beyond the scope of this post. I’d love to hear your ideas in the comments.

What this shows us is that while GWO is an excellent tool, it can be a bit blunt when used on it’s own. Very often the most valuable information is in the details.

– Ophir

UPDATE: (Sep 6, 2007) Google just redid the entire A/B split testing process. See:
http://adwords.google.com/support/bin/answer.py?answer=71362

While GWO’s was originally created to test specific elements within a page, sometimes you want to test totally different versions of a page.

Lets say you want to test three totally different versions of page1.html. You create three new pages: page1a.html, page1b.html and page1c.html. You and then add some custom code to page1.html which redirects visitors to one of the new pages.

GWO added this feature a while back. You can read the official page at Comparing entire pages (A/B testing)

Recently, one of the posts on the Official GWO Forum mentioned a shortcoming of Google’s solution – the fact that it doesn’t carry forward the query string.

The Problem:
Lets say the URL of your main test page is landing.html and one of your test pages is landing_1.html.

If a user is sent to landing.html?code=123 they will be forwarded to landing_1.html and not landing_1.html?code=123 which is what you probably wanted.

The Solution:
This is easier then you might think.
Look for this line in Google’s original code for your test page:

document.location = test_url;

and change it to:

document.location = test_url + self.location.search;

That’s it!

A few weeks ago Shawn Purtell from ROI Revolution wrote an excellent article on combining GWO with GA. If you haven’t read his article – please do so now. This posting describes a bug fix for his code, but you really should first read his article to fully understand how to use it.

I just found a bug in his code and am providing a fix for it. Under some circumstances, his code will provide the wrong combination number for the combination that is being displayed. Different combinations can be given the same combination number.

Continue reading

 Google just announced a new and improved version of their analytics software:

http://analytics.blogspot.com/2007/05/new-version-of-google-analytics.html

– Ophir

A few weeks ago I wrote in my Google Optimizer wish list the ability combine Google Analytics and Website Optimizer. I was actually in the process of creating a solution myself when it was brought to my attention that someone just created a solution:

Google Website Optimizer and Google Analytics: A Perfect Marriage

Kudos to Shawn Purtell from ROI Revolution for beating me to this 🙂

I cannot emphasize strongly enough how much this adds to GWO.
Almost every metric available in GA can now be broken down based on what combination the user saw. 

His solution uses the combination as a query parameter. I was originally thinking of using the user defined visitor segmentation for combinations, but after thinking about it, his solution seems better. The only situation I can think of where using user defined segmentation would be better is if you are doing advanced stuff like multi-page tests or full page testing (yes this is possible – more info to come soon).

One small note – when using the solution from ROI Revolution, be sure to call the new GA code after the GWO code. Otherwise you won’t be recording first time visitors to the page.

The official Google Website Optimizer group just changed status from “announcement only” to “general discussion”.

http://groups.google.com/group/websiteoptimizer

If you’re looking for a place to ask questions or share your experiences, it’s a great place to start.