Do your leads respond better to a red button or a blue button? Do you see higher open rates on emails whose subject lines ask a question or those that make a statement? Are conversions higher when your form is on the left side or the right side of the page?
In the digital world, even the most minute details can make a big difference. And the responsibility of making decisions about those details typically falls to marketers.
But how do you answer questions like these? It’s actually not as hard as it seems, it’s simply a matter of conducting some straightforward A/B testing.
What & Why: Behind the Power of A/B Testing
Why settle for “good enough” when you can make informed decisions that will take your efforts to the next level? Success in marketing, or anything else for that matter, requires always looking beyond the status quo and asking what you can do better to increase results. That’s the whole idea behind A/B testing in marketing.
A proper A/B testing strategy can help you achieve key goals (e.g. increased email open rates, conversion rates, etc.) by using data to make more informed, purposeful decisions. It accomplishes this objective by allowing you to test one version of something (e.g. button color, subject line, form placement, etc.) with a slightly different version of the same thing. Your audience (website visitors, email recipients, whoever it may be) is split equally between each of these versions at random, allowing you to gather data about which one garners a better response.
A/B testing can be an extremely powerful tool to drive your marketing activities forward, but the results are only as good as your testing strategy. Unfortunately, many marketers fall short in this area.
How to Conduct a Proper A/B Test: Two Mistakes to Avoid
A/B testing doesn’t have to be hard, it’s just a matter of knowing what you’re doing and using the right tool to help conduct the test and monitor its results. That said, there are two common mistakes that typically plague A/B testing efforts:
1. Testing Too Many Variables At Once
The key to A/B testing is to keep it simple: Make one difference and measure the impact. If you change several things between the different versions, then how do you know which change makes the difference? For example, if you’re trying to increase conversions on a landing page but you change three things — form placement, form length and page copy — from one version of the page to the next, how do you know which of those changes made the winner come out on top? To answer that question, you’ll need to move beyond the A/B testing model and into the multivariate testing space.
The difference between A/B testing and multivariate testing is the number of test versions in play. A multivariate testing model changes several different variables to look at the interactions among them and their overall combined impact on the end objective (e.g. increasing conversions). To do so, it creates different versions with every possible combination of the different variables. There’s nothing wrong with multivariate testing, but it’s a much more complex concept and should really be reserved for the most advanced among us.
2. Making Changes Based On Statistically Insignificant Results
Most marketers aren’t too keen on math, but just bear with us for a moment here — we promise not to make it painful!
Let’s look at the common example of a coin flip: Every time you flip it, there’s a 50% chance it will land heads up. So we can then say that if we flip a coin 100 times, it should land heads up around 50 times. If we do two trials, one inside and one outside, and when we’re inside we get heads 48 times but when we’re outside we get heads 50 times, does that mean that coins are always more likely to land heads up more often when we’re outside? No, this difference was just random and there’s nothing to which we can attribute it. That means that these results are statistically insignificant.
In addition to differences due to chance (as in the coin example above), another factor that contributes to statistical significance is sample size. For example, if you’re testing form placement to boost conversions on your website, your results will be statistically insignificant if only four people participate in your test, since that is (hopefully!) not enough people to accurately represent your typical website traffic.
While these examples are dramatically simplified, the good news is there are plenty of tools out there that can help you confirm the statistical significance of your results. If the results of your A/B test do end up being statistically insignificant, then you would be remiss to make any changes based off of them.
One Simple Test Can Go A Long Way
Although A/B testing might sound difficult to master, when you break down the what, why and how, you’ll quickly see that it doesn’t have to be so hard. And once you get started, you’ll find that the results are well worth the while, as proper A/B testing can arm you with the data you need to make more effective decisions.