A/B testing — why even consider it? To put it simply: because we’re wrong – about 80 percent of the time, according to some estimates.
We all know the marketing manager who fancies herself a Web designer. Or the business unit leader who can’t make up his mind on which hero image he wants to use on a banner ad landing page. What about the IT team who designs applications and functionality completely without the consumer in mind? Or the Web designer who is emotionally tied to his or her design, even though sales have plummeted after its implementation? Maybe you’ve been guilty of this behavior a few times yourself? It’s easy to do. We see something we like, and become blind to anything else. In short, we think we know best. But we don’t.
This is precisely why you should employ A/B testing, as one tool in your marketing strategy. It won’t save you, as there are many tools needed to complete the job, but it can definitely help. Because online consumers have constantly evolving needs and expect relevant, even personalized content from the wide variety of sites and content sources they choose to visit, it’s critical that online marketers quickly identify which offers and content are relevant and compelling to their audiences.
What Is A/B Testing?
To some marketers, it’s a dirty word. To others, it’s taken on a god-like importance. And then there’s me, somewhere in the middle. Done right, A/B testing can make a huge impact on your bottom line in a short amount of time. In a traditional A/B test, you experiment with 2 or more versions of a design and see which performs and converts the best. Sometimes, A and B are directly competing designs served to two groups of equal number. Other times, A is the current design and serves as the control that most users see while B, which might be more experimental, is served only to a small percentage of users until it has proven itself worthy of full exposure.
What Isn’t It?
It isn’t the answer to everything, it won’t single-handedly turn your business around and it can be over-used and abused. It will provide you with what most likely works in said scenario, but it won’t tell you why. Unlike focus groups, surveys or other direct consumer feedback, A/B testing doesn’t provide a direct window into a consumer’s mind. There are no thought bubbles that pop up in your analysis tool that tell you what consumers were thinking when they clicked “Start for free.” So you’ll never know why the visitors who saw design B decided to take the desired action 85% of the time vs. those who saw design A and only took the desired action 15% of the time. You will simply know that in this particular test, design B converted better.
For deeper qualitative analysis, you must have a comprehensive marketing communications strategy in place that could include other tools, such as focus groups, online communities, surveys and usability studies that allow you to interact directly with your consumers.
Again – What are the benefits?
A/B testing has 3 benefits:
- It measures actual behavior of your consumers under real-world conditions. You can confidently conclude that if version B sells more than version A, then version B is the way to go.
- It eliminates internal conflict by encouraging factual, data-driven decisions rather than decisions based on gut feelings and opinions.
- You can develop an archive of learnings, which can be applied to other similar areas of your website. In other words, you don’t have to reinvent the wheel every time.