Our Blog

What Is AB Testing and How Does It Work?

By Snap Agency March 27, 2016

What is AB Testing and how does it work?

Well, simply put, AB Testing involves serving up two different versions of a web page, or website, to different people in the same period of time and seeing which page gets more action out of users.

A simple example of an AB Test

If  you serve up a website 1,000 times with a headline that reads “Get the best above ground pool” with a big button that says “shop now,” and another one that says “Imagine yourself relaxing poolside after cooling off” with a big button that says “get an affordable pool now,” you can see which one of these could work best at getting people to shop. Testing these variations in language is simple AB Testing.

You can test two different billboard images on your website and see which one works better, or two different button colors, or even totally different color schemes.

What is AB Testing and How does it work?

After your AB Test runs for long enough, you can start to see a pattern emerge. One of the versions got the person to the next step 23% of the time while the other one only got them there 11% of the time. Clearly if the number of people qualifies the test as statistically significant, it’s time to make the more successful change permanent.

To get the most out of your AB Testing, every AB Test needs a clear hypothesis

The key with AB Testing is that you have to come up with a good hypothesis to test. You can’t play the role of sensitive designer type and try to test a drop shadow: You have to come up with a clear scenario like, “I think that when a person is halfway down a key service page, they might want to take the next step to contact us before reading the second half of the page.” In this case, you could try inserting a button that allowed a visitor to do that and see how it is affecting people getting to the contact page. Every test should have a hypothesis behind it.

How do I start AB Testing on my website?

There are some amazing services for AB Testing. If you have an existing site and want to try out making some changes, you could try:

  • Optimizely: This is my preferred tool as the user interface is amazing and I love the clear indication when a test reaches statistical significance
  • Visual Website Optimizer

Here is a great breakdown of Optimizely vs. Visual Website Optimizer for different aspects such as usability.


Visual Website Optimizer vs. Optimizely


How do I make landing pages separate from an existing website to A/B Test?

New tests are coming out every day and some of the coolest things in digital marketing right now are landing pages that you can roll out quickly and test against each other very easily. A/B Testing is now standard in so many of these landing page tools that I thought I’d profile a couple of them and help you get an idea of how you might use one, or more than one of them, in your marketing campaigns.

  • Unbounce: Incredibly easy to use and roll out new landing pages with existing templates
  • LeadPages: Lots of pre-built pages, and easy to get started
  • ClickFunnels: More customization, start from scratch, add affiliate marketing easily

With any of these tools you can test versions against each other, but don’t expect conclusive results unless you have quite a bit of traffic going to them. This goes for any A/B Test really. Here is a more in-depth audio breakdown of the different features between LeadPages and ClickFunnels:

Here are some wonderful posts that can kickstart your hypothesis creation process:

Or if you’d prefer, here are 10 things I might start with if I were you:

  1. Try adding different trust factors like badges or testimonials next to key places where people will take action
  2. Try adding still images instead of carousels, and think long and hard about the photography in the still image and how it tells your company’s story or demonstrates the key value proposition
  3. Try testing a highly emotional “What is the core benefit to me as a customer” headline vs. a very descriptive, clear and concise headline
  4. Try adding more/bigger/higher-contrast call-to-action buttons higher up on the page, with little copy underneath, that give one last little positive nudge to someone who’s about to take the next step
  5. Try removing form fields (make sure they’re not required to submit the form in the code), and see if people fill out the form with three fields instead of one with seven
  6. Try giving away a 15-day trial or 30-day trial, offering a money-back guarantee, or offering a free version with the opportunity to upgrade for more features, and see what people find more compelling
  7. Try making a version of the page that has answers to every key question a prospect would ask. Create one with two or three sentences giving a clear answer for each question, and make another version that only has one main section with key points bulleted
  8. Create different versions of your navigation; some with very few items, some with deeper options, and see which one gets people more involved
  9. Test social proof: Add more social media evidence to support your product’s viability and social sign-off if appropriate
  10. Add video or deeper visual explanations of points where you may want to illustrate the value of the product or service. Serve up a version with a short infographic or illustration and one without

How can I use either of these methods for marketing my products and services?

The possibilities are endless, but A/B Testing on an existing site just requires a bit of imagination and perhaps some good resources for types of things to test and knowing how to apply that creativity to your unique situation. The good news is that with tools like Optimizely, it’s pretty easy to get started. You will have to know some code or at least be able to create the alternate scenarios in certain situations, but there’s much you can do without touching code.

To me, A/B Testing is super fun and informative. And sometimes simple changes can swing your profitability by $5,000, which, to me, is one of the most fun aspects of all.