A beginner’s guide to A/B testing
Raise more money and build support for your mission by running testing experiments. Test your ideas and measure the results to find out what really works when it comes to engaging with your supporters and building support for your mission.
In December 2007, Barack Obama was running for the Democratic nomination for President of the United States. He was behind in the polls, so his team knew they needed to build more support.
What they did next raised an extra $60 million in donations, gathered 2,880,000 more supporter email addresses and converted 288,000 more volunteers.*
And they achieved all this by testing just two elements of their campaign website: their call-to-action button and image panel.
Finding the messages that work for your audience is key to fundraising success. And the only way to be sure of what works is by trying different ideas.
Don’t be afraid to fail
When it comes to testing, some of the things you try aren’t going to work. The most important rule in testing is to fail fast and learn quickly.
You want to try new ideas, new combinations of material and new ways of talking to people. And you want to learn which of these new ideas to keep, and which you can rule out.
You want to find out what effect these different variations have, so you can move towards the combination that works best.
What do you want to learn?
So, how do you go about finding out what works in your campaigns and what doesn’t?
First of all, you need to have some theories to test. For example, you might think that a gentle ‘learn more’ button will work better than a direct ‘sign-up now’ button on your website form.
Or you might want to know whether your emails work better when you use your own name, or your organisation’s name as the sender.
Keep things simple
At this point, it’s time to take a step back. It’s great that you’ve got loads of ideas, but make sure you don’t test everything at once.
You might get some brilliant new results, but how do you know which change has the biggest impact? Make sure you test your variables one by one, and measure the results of each change.
Over time, you can start combining the best of each variable for maximum impact. So, in our example, taking your best sender name and combining it with the ask that’s most successful.
Measuring your success
Of course, measuring what changes have the biggest impact on results means carefully tracking your actions. You need to know who receives what message, and what actions they take as a result.
You’ll want to try your variations on equal numbers of people, and make sure you try things out on enough people for your results to be reliable.
It’s no good testing your changes on a few dozen people, then expecting the result to be the same when you try your changes out on hundreds of people.
In the short term, this means being brave. Yes, you might not get good results from all your tests. But you’ll do much better in the future by using what you learn.
Keeping track of results
Having all of your supporter information in one place is a great start when it comes to measuring the success of your testing.
Having a single database helps you identify your sample groups (who receives what), and then track the outcome of those tests.
It can even help you start thinking about clever ways of segmenting your supporters, to see the effects tests have on different groups.
That’s where a fundraising solution like eTapestry comes in. It brings together your supporter information with email tools, website forms, and dashboards that help you track your success.
Find out how eTapestry can help you make changes that increase your fundraising and build support for your mission.
* And if you want to know more about the Obama team’s work in 2007, check out this blog here.
More about eTapestry
Find out how eTapestry, the fundraising solution designed for growing charities, can help you build stronger supporter relationships and support for your mission with eTapestry.