e-dreaming

Some call it binary testing, others call it A/B testing, but the principal is the same regardless of what you call it.  This structured process of constant improvement allows practitioners to methodically change the elements of ads, websites, etc., and pragmatically compare the results.

For example, you could run two ads, ads A & B, in a popular publication, simultaneously for one month, each identical except for the phone number and caption.  At the end of the month, compare the number of calls received on each number, and declare a winner (let's say that's ad B).  The next month, you run ad B again (exactly as it was but with a new phone number), and a new ad, ad C.  Ad C is exactly the same as ad B, but again with a new phone number, and a new caption.  At the end of the month, you compare the number of calls received from ad B and ad C, and declare the winner (let's say at B again).  Next month, do it again, with ad B and a new ad D.

This is an overly simple example, and you would likely not do something like this, because of the expense of running two print ads, the unpredicatable effects of ad location within the publication, and the cost of all of those phone lines.  But, from this example you can begin to see that by methodically changing a single element at a time (in this case the ad caption) and comparing results, your ad continues to improve over time, simply by tossing the lesser-performing ad, and keeping the best.

That's A/B testing.  While it sounds simple, it takes time, patience, persistence and a commitment to constant improvement.  In the above example, eventually, you are likely to discover that your ad caption is performing optimally (all new ad caption versions perform worse), and you can move on to other changes in the ad (the image, layout, etc.)  And ultimately, you'll have an optimized ad that brings you the business.

Where A/B testing is really most viable is in new media advertising, where you can make those changes in minutes, and at little expense.  For example, you could have two Google ads running at the same time.  At the end of a period of time or number of impressions or clicks, make the determination of which of the two ads is performing the best (the most number of clicks, the highest click-thru rate, or the highest number of conversion), kill the lesser performing ad, and create a new ad.

This simple process is proving to be a HUGE money-making innovation.  Another example of A/B testing is to test the effects of changes in label of a buy button on your website.  You might discover that changing the button text from CHECKOUT to GET FREE SHIPPING NOW results in a 10% increase in folks actually buying your product, even if you offer free shipping elsewhere in your shopping cart.  These small, incremental changes can make the difference between a website that makes money and one the costs.

The mantra for A/B testers is TEST TEST TEST.