Marketing conversion optimization expert Brooks Bell demystified A/B split testing for marketers last week and shared why split testing is valuable. The president of conversion optimization agency Brooks Bell Interactive spoke to TIMA members and guests at Marbles Kids Museum in downtown Raleigh, NC. Thanks to Arik Abel for the photos.
The Five T’s of Testing
The focus of Brooks’ presentation was her Five T’s of Testing framework (right):
- Traffic (you need enough “success” actions to have statistical significance)
- Technology (your tests must be random and concurrent)
- Time (it takes longer than you expect, and it’s a continuous process)
- Trust (you have to trust that the data is accurate)
- Team (you need a multi-disciplinary team of marketers, designers, and technologists)
Why Is Split Testing Valuable for Marketers?
Brooks shared how she optimized the Washington Post‘s website signup process, which I recognized as a WashingtonPost.com member. It was fascinating to see the work that went on behind the scenes to deliver the effective conversion pages.
The general idea is that if you’re always testing against a control, your conversion rate will always be improving. That makes sense. But it’s clearly a complicated process.
Technology and team challenges aside, the big challenge for smaller companies (that is, non-Fortune 500 firms) is having enough meaningful website traffic to achieve statistical significance. And for testing, you have to count “successes” — like email signups or info requests — instead of raw traffic.
For instance, Coalmarch is helping one of our clients get more info requests via their contact form. Overall website traffic aside, the small business is getting just three requests a week. We’re going to help them increase that, but even when that quadruples, it’d take two years to get the 200-300 actions (100-150 “successes” per cell, for a single A/B test) that Brooks mentions you need for statistical significance.
Ultimately, we’ll help the client get results, but we can’t attribute the improvement to a specific “recipe” of changes.
Easy Ways to Get Started
If you’re ready to add testing to your marketing department, Brooks mentioned several easy areas to get started:
- Email subject lines
- Offer emails
- PPC landing pages
- Lead form
- House ads
With a medium-sized email list or even a fairly small PPC campaign, it’s not too hard to get the hundreds of clickthrough “successes” to get meaningful results. I’ve used a rudimentary version of A/B split testing to test Facebook ads and the results are often dramatic.
If you want to learn more about using testing in your marketing campaigns, watch Brooks’ “Testing, 1, 2, 3” webinar from June 2010 (it’s a 40-minute recording, well worth the time).
Photo credits: Arik Abel