We talk to Kevin Conroy, Chief Product Officer of Global Giving about A/B testing with Optimizely and discuss how to think about testing your site. Then discuss how the Obama digital team used A/B testing to drive an extra $60 Million in donations.
More about the Behavior Model from Dr. BJ Fogg
Show notes
- Story of Obama A/B testing
- Optimizely.org – non-profit discount for A/B t
- Unbounce.com – A/B testing
- http://www.behaviormodel.org/ – Dr. BJ Fogg
- Globalgiving.org – great crowd funding platform
Transcription
Episode 4
This is using the whole whale, a podcast that brings you stories about data and technology in the nonprofit world. My name is George Weiner, your host, and the chief whaler of wholewhale.com, thanks for joining us.
So I learned a ton at NTEN, it isn’t always from the sessions that I attend but the people that I speak with. In this particular case I got a chance to speak to Kevin Conroy the chief product officer of Global Giving. And in this conversation it was interesting because his example of rolling out Optimizely and ab testing software for websites, perfectly demonstrated the idea that in the relationship of people process and product, just having the right product in place sometimes is just not enough to institute the type of behavioral change in an organization that you’re trying to affect. So let’s hear a bit more from kevin in this quick interview.
George: How’s it going? This is George Weiner at NTEN the 2014 conference, and I am here with Kevin from Global Giving. Kevin, can you tell us what you do at Global Giving?
Kevin: I’m the chief product officer at Global Giving, so I’m in charge of all the technology and marketing and Global Giving is an non profit that helps other nonprofits crowd fund in 140 countries around the world.
George: So these guys are the go to if you’re trying to do the online crowd giving. They’ve been in the game for quite sometime. but what really caught my attention today was how, Kevin you’re tackling AB testing, kevin why don’t you tell us about how you’re rolling out AB testing in your organization…
Kevin: So we had a lot of feedback from staff, that they wanted to do a lot more testing and iteration on our website, and they wanted to do it much more quickly and frequently, and they felt that they were hindered by the bandwidth or lack of bandwidth of our technical developers to be able to do all the testing that they wanted. So at the great advice of George, we got, a couple months ago, Optimizely installed. Optimizely is a great service that lets you very quickly and visually set up AB testing on your website, and then launch them. Optimizely handles all the heavy lifting of actually serving up the different pages, and most importantly doing the analysis. so it will tell you when you have hit statistical significance. Which is one of the biggest problems of AB testing, what has won and when can you determine that. Because it takes a fair amount of math to do that. which can be kind of daunting for most folks. So, Optimizely just wraps that up into a beautiful package, that’s very simple to use. So we rolled that out to our organization our staff of thirty, and said “hey you can actually go and set up an ab test.” in optimizely we have rolls set up, so that everyone on staff can set up an ab test, and then somebody on our development team, programming team, can actually approve it, watch it live. We thought this was going to solve all of our problems around testing, and what we found was that it wasn’t an access problem for technology, but that people were so afraid to make changes to the website. and so we need to do an education campaign, in terms of “ no. It’s ok, you’re not going to break everything we want you to make changes, you can actually do this.” so if you’re struggling with ab testing yourself, it may not be a technology problem, it could be a human problem as well, understanding of technology and how to improve your website.
George: Totally, so what would be an example of something like “ I can’t believe we’re not ab testing on this site, don’t you understand how many blabidy blahs come through the thingidy thing!?”
Kevin: right exactly were we encourage nonprofits to sign up and join Global Giving, and so we’ve been told “i don’t love the two sentences you have on the headline” thats a great thing to ab test. so wording actually you’d be suprised, has a huge impact on conversion rates, just saying the right words can convince users to do something, so we thought optimizely and ab testing could be the way to do this. And people are still afraid, it can be a real challenge to say “i don’t like what words are there, but i don’t have a better alternative to suggest,” so i think that one of the challenges is making it easier, and making people feel they have permission to contribute to all of this., because i feel as non profits we need to be really agile, and everybody needs to wear a lot of different hats,so it can be challenging and i’m realizing as the chief product officer, i need to be fostering that sense of community and empowerment within the organization in order to for us to hit high levels of scale.
George: thank you so much for your time, and insight into rolling out optimizely and ab testing. how do people find you on the interwebs?
Kevin: you can find us at Globalgiving.org, and globalgiving on facebook, and @globalgiving on twitter..
George: brilliant! thanks again!
kevin: thank you
George: I really do like hearing these kinds of stories about non for profits integrating new technology. In our data culture model where its people, process and product that all make up a successful implementation of technology and behavior in an organization. in this case we hear Kevin talking about the difficulty of , “i thought if i just put the right product in we would be able to suddenly have a culture where we would be testing things.” optimizely is certainly the right product, optimizely lets you test two versions of a landing page, in different variations within that, in a simple way that only shows it to ten percent of your audience. There are other tools out there such as unbounce, and also google experiments that will let you do the same thing. so you see the product is not necessarily the blocker, it is very frequently the training and motivation of our people. But let’s say you do have optimizely rolled out for testing, by the way optimizely.org does I think have non profit pricing, you have this in place, well, what are we testing? how do we begin think about which pictures we move and which places and how to deal with motivation? A framework that is a very excellent one to go by created by Dr. BJ Fogg, he is the founder of persuasive technology lab at stanford university, he breaks it down to three simple variables: ability, motivation and triggers. And here’s a quick clip from one of his youtube series…
Dr. BJ Fogg: There are three things that have to happen in order for a behavior to occur. People have to be sufficiently motivated, they have to have the ability, and they have to be triggered to do the behavior. So logging into facebook, and posting content, motivation ability trigger, they have to have all those things. so if I were facebook incorporated and I were trying to get those once a monthers to do it more often, I would look at those three factors and see which ones were missing for those people who aren’t logging in more often. Motivation, ability or trigger, and from a practical standpoint I learned you start at the trigger first. you say “are they being sufficiently triggered?”
George: I really love BJ Fogg, I love this model, but sometimes it can get a little heady, it can get a little out there. So what do we actually mean when we look at a webpage and when we say “ motivation, ability, trigger?” well, motivation would be what is that we spend most of our time building our content around videos and text. Say were trying to raise money for a homeless shelter, and were talking about all the incredible things that a single dollar will help the work that we do. The ability or the perceived ability is what do you want me to do and how hard is it going to be…do you want me to volunteer the rest of my days, do you want me to donate a couple dollars, do you want me to give an email, so we have this in the back of the mind, what is my ability to do the thing you want me to do. And then finally the trigger, what is the button, the activation point that i have to click and do, could be even an email a text message that came to me, that triggered this action that you want me to do in the next step.
Let’s make it practical by looking at a classic example by Optimizely, actually one of their founders was part of the Obama campaign that was responsible for incredible amounts of testing. During Obama’s 2007 campaign he had a landing page on Obama.com, the sole purpose of which was to get your freaking email address, that was it. And they tested a whole bunch of things on this layout, the motivation at the top where they had an image, they had a video of Obama, he was a fantastic orator so they figured sure why wouldn’t this video do much better in convincing people that they should give their email to the organization. What they tested was the difference between these two options and it broke down to 11 and 8 percent conversion rates. And it wasn’t actually the video it was a black and white picture of him with his family that got the 11.6% conversion rate meaning people who gave their emails on the spot, versus the video which had 8.2% of next best option. and that’s an improvement of 40 percent which is really significant, when you’re talking about a site that gets 10 million people going to it. That breaks down to a difference of about 2.8 million email addresses either way. And so when we realize that we set up these main landing pages, pages that many people are going to, and we don’t test them, we can be off by factors of significant amounts. That can translate to, what they said, those email addresses amounted to 60 million dollars. Now I don’t know about you but I can definitely see 60 million dollars in donations being a significant change for any campaign.
So those are the practical elements that I have for today around testing and implementing data culture to an organization. You can find out more in our show notes @wholewhale.com/podcasts, and we will have links to all of these things including BJ Fog and Optimizely, thanks so much for joining us. Take care.
This has been using the whole whale, the podcast for more information on topics covered in todays show please check out wholewhale.com and consider following us on twitter @wholewhale. Thanks.
esting
