Online Testing and Optimization Solutions: Quick guide to Adobe Test&Target

What really sets Adobe Test&Target (powered by Omniture) apart is both its reporting and support. While it can be expensive, what you get is a professional/polished, customizable tool that’s great for in-depth, multi conversion tests. And even better, you have an assigned support rep that can help you quickly overcome any stumbling blocks before or during a test.

To illustrate the level of reporting, with one of our Research Partners, we setup 35 success metrics for just one test. In the second in our blog series about testing solutions, here’s a quick guide to Test&Target…

Key benefits

* Transaction- and product-level reports
* Revenue tracking can be added to any campaign
* “Push winners” functionality – with a single click, pushes winning test scenarios to standard content for all audience segments
* Option for Omniture to host images, HTML or you can use your own server
* Tracking within Test&Target – ability to setup any metrics within tool; can change conversion metric during test (on the fly)
* Reports on a daily level
* Can track pretty much anything you want (links, buttons, etc)
* Behavior targeting – serve landing pages based on user defined segments (traffic sources, etc)
* Test&Target segmentation for reporting – can report on different campaign codes, offer IDs, etc
* Data privacy due to contractual relationship with a company
* Support – includes testing ideas, test implementation, report setup
* Ability to tack on other data-mining tools (i.e., Discover can track on the visitor level)
* Increased capabilities for segmentation and tagging of actions on a page – can get extremely granular in what you track, i.e. different content for different segments at different times
* If you don’t want to send all traffic to the page, you can set a tool to only receive a certain percent of traffic (would have to write custom code for Google Website Optimizer)

But keep in mind…

* Cost of tool and cost for additional support
* IT requirement for setup


adobe-omniture2-oWhat really sets Adobe Test&Target (powered by Omniture) apart is both its reporting and support. While it can be expensive, what you get is a professional/polished, customizable tool that’s great for in-depth, multi conversion tests. And even better, you have an assigned support rep that can help you quickly overcome any stumbling blocks before or during a test.

To illustrate the level of reporting, with one of our Research Partners, we setup 35 success metrics for just one test. In the second in our blog series about testing solutions, here’s a quick guide to Test&Target…

Key benefits

  • Transaction- and product-level reports
  • Revenue tracking can be added to any campaign
  • “Push winners” functionality – with a single click, pushes winning test scenarios to standard content for all audience segments
  • Option for Omniture to host images, HTML or you can use your own server
  • Tracking within Test&Target – ability to setup any metrics within tool; can change conversion metric during test (on the fly)
  • Reports on a daily level
  • Can track pretty much anything you want (links, buttons, etc)
  • Behavior targeting – serve landing pages based on user defined segments (traffic sources, etc)
  • Test&Target segmentation for reporting – can report on different campaign codes, offer IDs, etc
  • Data privacy due to contractual relationship with a company
  • Support – includes testing ideas, test implementation, report setup
  • Ability to tack on other data-mining tools  (i.e., Discover can track on the visitor level)
  • Increased capabilities for segmentation and tagging of actions on a page – can get extremely granular in what you track, i.e. different content for different segments at different times
  • If you don’t want to send all traffic to the page, you can set a tool to only receive a certain percent of traffic (would have to write custom code for Google Website Optimizer)

But keep in mind…

Internet Marketing: Landing page optimization for beginners

Dustin Eichholt |
http://www.marketingexperiments.com/blog/research-topics/internet-marketing-for-beginners.html

C = 4m + 3v + 2(i-f) – 2a ©
Wherein:
C = Probability of conversion
m = Motivation of user (when)
v = Clarity of the value proposition (why)
i = Incentive to take action
f = Friction elements of process
a = Anxiety about entering information

By dealing with the heuristic piece by piece, you’re able to get in the mindset of your visitor/ potential customer and deal with the page as they would.

So far, some work I’ve done with the CIA includes analyzing motivation of visitors – are they getting what they’re expecting? Does it match what they’re looking for at that point in time?

Clarity of the value proposition is a very important part of any landing page. It’s important to not only state it clearly but ask yourself if the other elements support that value proposition. The customer wants to know not only why they want a product, but why they should get it from you.

Friction can be pretty straightforward– are you asking too much of your visitor? Is it difficult to navigate or are you asking them to enter a lengthy signup process? In addition to length, you must also ask yourself if there is any field or informational request that causes concern or anxiety. Do you explain that the visitor’s email address will not be used deceitfully?

You can think of all the elements of the conversion heuristic working together as a fulcrum or scale. Positive elements, such as matching visitor motivation and communication of value, are on one side of the scale. Negative elements, such as friction and anxiety, are on the other.

On a particular landing page, does the balance of the scale tip too much towards the negative or positive? If it’s tipped towards the negative elements, what steps can you take to tip it the other way? Would an incentive such as a free gift or free shipping help offset some of the friction you can’t get rid of?

All of these questions can be daunting. Writing, designing, and building a website is hard, a successful one even harder. However, at MarketingExperiments, we look at these questions as opportunities for testing. The team I’ve been working with in the labs is very good at looking for opportunities, even on already successful sites.


Dustin Eichholt |
http://www.marketingexperiments.com/blog/research-topics/internet-marketing-for-beginners.html

C = 4m + 3v + 2(i-f) – 2a ©
Wherein:
C = Probability of conversion
m = Motivation of user (when)
v = Clarity of the value proposition (why)
i = Incentive to take action
f = Friction elements of process
a = Anxiety about entering information

By dealing with the heuristic piece by piece, you’re able to get in the mindset of your visitor/ potential customer and deal with the page as they would.

So far, some work I’ve done with the CIA includes analyzing motivation of visitors – are they getting what they’re expecting? Does it match what they’re looking for at that point in time?

Clarity of the value proposition is a very important part of any landing page. It’s important to not only state it clearly but ask yourself if the other elements support that value proposition. The customer wants to know not only why they want a product, but why they should get it from you.

Friction can be pretty straightforward– are you asking too much of your visitor? Is it difficult to navigate or are you asking them to enter a lengthy signup process? In addition to length, you must also ask yourself if there is any field or informational request that causes concern or anxiety. Do you explain that the visitor’s email address will not be used deceitfully?

You can think of all the elements of the conversion heuristic working together as a fulcrum or scale. Positive elements, such as matching visitor motivation and communication of value, are on one side of the scale. Negative elements, such as friction and anxiety, are on the other.

On a particular landing page, does the balance of the scale tip too much towards the negative or positive? If it’s tipped towards the negative elements, what steps can you take to tip it the other way? Would an incentive such as a free gift or free shipping help offset some of the friction you can’t get rid of?

All of these questions can be daunting. Writing, designing, and building a website is hard, a successful one even harder. However, at MarketingExperiments, we look at these questions as opportunities for testing. The team I’ve been working with in the labs is very good at looking for opportunities, even on already successful sites. Leer más “Internet Marketing: Landing page optimization for beginners”

Online Testing and Optimization Solutions: Quick guide to Google Website Optimizer

Key benefits

* Combine with Google Analytics for ecommerce product SKU tracking
* Robust tracking with Google Analytics and Adwords
* Testing a landing page setup is simple – just add a couple lines of JavaScript code and setup the test in Google Website Optimizer ’s interface (you will need the page URLs as well as the conversion page)
* Overall reliability of Google’s servers (we’ve never heard of an outage of Google Website Optimizer servers)
* Open application programming interface (API) – you can draw data from the tool into your own environment without having to log into Google Analytics or Google Website Optimizer
* Large base of marketers have had exposure to the tool and, at a minimum, have some sort of expertise on the tool
* Google name brings credibility – May help get buy-in from others that might be unfamiliar with the tool but recognize the Google name
* With multivariate testing, once the JavaScript switching code is in place, you can add and rotate in elements without IT needing to alter the code for each test
* Active product development (constantly rolling out updates and bug fixes) [Más…]

But keep in mind…

* Google does not deal directly with clients for support – while Google does not offer support, they do have an exclusive group of Certified Partners that are screened by Google and qualified to offer support with any Google Website Optimizer question or issue you may encounter
* Security of data is a concern – Google has access to all data
* Google Website Optimizer only provides aggregated data – you need to add Google Analytics to get reports on daily data
* If you only use Google Website Optimizer, tracking is limited – Google Analytics needs to be added to get tracking beyond visits and conversion metric
* For both Google Website Optimizer and Google Analytics, data is not real time – it can take a few hours (this can elongate quality assurance, or QA, time)
* Have to get creative when working with secondary conversions – i.e. use other tools like Google Analytics to measure secondary conversions/clicks like newsletter sign-up in a sales process
* Cannot add additional metrics collections points within the tool – for example, segmentation of conversion rates, elements that users interact with other than the conversion point (you will need to leverage Google Analytics for this additional tracking)

What types of elements can you test?

* A/B and multivariate testing

How does it validate?

* Reports – chance to beat original, chance to beat all
* Method – Google Website Optimizer uses statistical significance of the difference in sample means among the treatments based upon a Gaussian distribution presumption

Cost

* FREE

Support

* This is important enough to mention again – Google does not deal directly with clients for support, although there are Certified Partners that may be able to help you
* Step-by-step instructions and FAQ’s
* Google Website Optimizer blog
* No contact person (unless you have one through AdWords)
* Forums – lots of great questions answered
* Instructional info available on Google Website Optimizer site

Technology / Development

* For Google Website Optimizer implementation, will need to add JavaScript code – or have Apache Subversion (SVN) access to add JavaScript code
* Google Analytics /Google Website Optimizer moved to AJAX calls – speeds up loading to end user
* A/B testing – has to be hosted on your server (you only have to add hosted page URLs to tool)
* Multivariate testing – has to be hosted on Google’s server

Reporting

* Alone Google Website Optimizer only reports aggregate data
* With Google Analytics it can track daily

We’d love to hear some customer reviews as well. Use the comments to share your experience with Google Website Optimizer. And stay tuned to this blog as we provide quick guides for more online testing solutions to help you choose a platform that is best for your individual situation.


Gina Townsend

Gina Townsend
http://www.marketingexperiments.com/blog/

I just flew back from the Googleplex and boy is my brain tired. It’s exciting to see the growth of Google Website Optimizer and the online testing and optimization community in general.

Tasked with writing a blog post about my trip, at first, I just wanted to focus on the Googleplex itself (yes, it’s that cool). Then I thought, a fawning post about Google Website Optimizer might be apropos.

But given the number of questions we receive about online testing and optimization tools, I’m going to take a third, more aspirational path. Today I’ll give you some info about Google Website Optimizer, but this will just be the first in a series of posts aimed at helping you answer that “Which testing tool?” question we often receive.

Of course, I can’t answer that question for you per se (the real answer is, “It depends”), but hopefully I can provide enough info over this series of posts to help you make an informed decision.

And here’s my promise to the vendors out there, even if you didn’t invite me to your “-plex” and ply me with delicious locally grown organic food, I still may write about your tool (and if the FTC is reading, Google did, in fact, serve up quite a bounty).

Now all full disclosures aside, here’s a quick look at Google Website Optimizer…

The biggest benefits for Google Website Optimizer are the obvious, it’s free and it’s Google.

But beyond the cost, Google is committed to making quality products and is continuously working on enhancements. Keep in mind that Google doesn’t offer much support, if any, so you’ll need to put in some time researching if you come across any obstacles. Here’s a quick overview…

Key benefits

  • Combine with Google Analytics for ecommerce product SKU tracking
  • Robust tracking with Google Analytics and Adwords
  • Testing a landing page setup is simple – just add a couple lines of JavaScript code and setup the test in Google Website Optimizer ’s  interface (you will need the page URLs as well as the conversion page)
  • Overall reliability of Google’s servers (we’ve never heard of an outage of Google Website Optimizer servers)
  • Open application programming interface (API) – you can draw data from the tool into your own environment without having to log into Google Analytics or Google Website Optimizer
  • Large base of marketers have had exposure to the tool and, at a minimum, have some sort of expertise on the tool
  • Google name brings credibility – May help get buy-in from others that might be unfamiliar with the tool but recognize the Google name
  • With multivariate testing, once the JavaScript switching code is in place, you can add and rotate in elements without IT needing to alter the code for each test
  • Active product development (constantly rolling out updates and bug fixes) Leer más “Online Testing and Optimization Solutions: Quick guide to Google Website Optimizer”

An Introduction to Split Testing in WordPress

Kirk Kaiser

Split testing is a way to experiment with a live site and find which headlines and text are the most effective. Amazon uses split testing to determine which versions of their site convert customers better. We’ll use WordPress and Google’s Website Optimizer to test two different headlines, and find which works best at capturing customer emails. [Más…]

Step 1: Decide on Your Experiment Type

First, we’ll need to decide what sort of an experiment we’d like to run. In Google’s Website Optimizer, we have two choices: Multivariate Testing or A/B Testing.

Multivariate testing is a good way to test a lot of things at once. Big online stores use multivariate testing to figure out which layouts and ad copies work best. For the purpose of this tutorial, however, we’ll use A/B testing.

A/B Testing is a way to test two different types of copy, and see which works best. We start with two different variations of the same page, send some traffic to the pages, and see which converts best statistically. For the purposes of our split test, we’ll want to have at least 100 conversions before deciding which page is the statistically significant winner.


Split testing is a way to experiment with a live site and find which headlines and text are the most effective. Amazon uses split testing to determine which versions of their site convert customers better. We’ll use WordPress and Google’s Website Optimizer to test two different headlines, and find which works best at capturing customer emails. Leer más “An Introduction to Split Testing in WordPress”

Multivariate Testing: Can you radically improve marketing ROI by increasing variables you test?

In response, one emerging MVT service model offers getting to a “lift” faster by using adaptive elimination of likely underperformers, in exchange for the test results providing limited information beyond identifying the winner. Such test results are not as useful as their full-factorial brethren for designing subsequent tests because adaptive elimination of treatments makes it difficult to extrapolate the psychological factors and consumer preferences responsible for the test outcome. The immediate business benefits, however, are more immediate.


As I was reading a few LinkedIn discussions about multivariate testing (MVT), I began to wonder if 2010 was going to be the year of multivariate.

1,000,000 monkeys can’t be wrong

Multivariate Testing (MVT) is starting to earn a place in the pantheon of buzzwords like cloud computing, service-oriented architecture, and synergy. But is a test the same thing as an experiment? While I am not a statistician (nor did I stay at the Holiday Inn last night), working at MarketingExperiments with the analytical likes of Bob Kemper (MBA) and Arturo Silva Nava (MBA) has helped me understand the value of a disciplined approach to experimental design.

MonkeyWhat I see out there is that a little knowledge is indeed a dangerous thing. Good intentions behind powerful and relatively easy-to-use platforms like Omniture® Test&Target™ and Google® Website Optimizer™ have generated a misleading sense that as long as a multivariate test is large enough (several hundred or more combinations being tested), at least one of the combinations will outperform the control.

This notion has become the value proposition of a growing number of companies offering services around either the big-name or their own (simpler, and often therefore easier to set up) MVT tools. They are ostensibly betting on the technology, and not on a systematic approach to experimental design or any particular UI/UX (user interface/user experience) optimization theory.

Even though, as Bob has pointed out to me, it is reasonable that an MVT setup with a billion combinations may not yield a lift over the control, my contention is that the risk-weighted business cost of a dissatisfied customer is low. Therefore, little stops the burgeoning MVT shops from safely offering a “100% lift guarantee.” Just like the proverbial million monkeys with typewriters, somewhere among thousands of spray-and-pray treatments their MVT tests are expected to produce one that’s better than the rest.

1 monkey with a stick Leer más “Multivariate Testing: Can you radically improve marketing ROI by increasing variables you test?”