PAIN RELIEF: Demystifying the Testing Process

Testing mystifies me.  I know I should be testing more, but not sure how to optimise that process for the maximum benefit. How can I improve my testing effectiveness?

Newman: In order to test effectively, you need to determine what it is you’re testing and why you’re testing it. Is it to drive conversion? Is it to reduce abandonment? Is it because you’ve got a high bounce rate or a high exit rate on a specific page?

You also need to have analytics set up so that you can measure the effect of what it is you end up testing.

There are essentially a number of ways you can test different elements of your website.

The three most common are as follows:

1. Setting up goals to measure: Using your analytics tool (Google Analytics, Webtrends, Omniture, Coremetrics etc) to measure performance. The most common and effective goal to measure is the conversion funnel where you can get a clear view of how many customers begin each stage of the checkout, how many drop out and how many convert and make a purchase. But there are many other goals you can measure including newsletter sign ups, add to bag, as well as measuring a host of other metrics including exit rates, bounce rates and so on. The testing would come in when you decide to change a page or some other specific part of the customer journey in order to drive an increase in conversion or to reduce an exit rate etc.

2. A/B split testing: You can conduct split testing using a number of different tools including Google’s free ‘web optimizer.’ This isolates a single page and gives you the opportunity to test two variants of a page, where every second user that lands on the page sees a different version or an element of a page that you’ve changed to determine which version converts best.

A great advantage of A/B testing is that it is relatively quick and easy to set up. For example, you can think of a new page headline over lunch, implement a test, and see the results within a few hours or days.

A/B testing also offers the key benefit of unambiguous results. If you’re testing a single page variable only with two possibilities, the results are typically available quickly and the best course of action is usually quite clear. This takes the subjectivity out of site design.

3. Multi variate testing (MVT): This is normally undertaken using paid for tools and applications where as the term MVT suggests, you can test multiple variants of a page or elements of a page. In some cases, retailers will test hundreds of different versions of a page and often changing many aspects of the page until deciding upon the most effective designs.

It takes some time to set up a multivariable test. You have to design and implement all of the individual page variations at once.  However, if you have a number of variations and combinations to test, multivariable testing can save you much time and aggravation in the long run.

In terms of what you could or should be testing either on an A/B split test or a MVT, it will include one or more of the following elements:

  • Key landing pages (Both SEO landing pages and key site pages such as category landing page, homepage, product list pages, product detail pages etc
  • The checkout/the conversion funnel
  • Headline and other key calls to action
  • Copy
  • Navigation
  • Promotional offers
  • Buttons
  • Colours
  • The position of different elements on a page including promotional and placeholders

In terms of frequency, you can never finish testing.  Test and optimise, test and optimise, test and optimise.  It should be a continual and every day task to drive an increase in the performance of your website.

One thought on “PAIN RELIEF: Demystifying the Testing Process

  1. Check out This website sends you a free weekly A/B split test of a real-life company on any of the tests Martin has supplied above and gets you to guess, and then see the results. There is usually a furious discussion whether the test is what people guessed!


Leave a Reply

Your email address will not be published. Required fields are marked *

PowerRetail Extra Enewsletter