Hey innovation! Have you met rapid experimentation?


The online dating website OKCupid recently posted the results of some real life experiments that they had performed on their user community. The experiments included three tests to evaluate the influence of certain user profile changes in compatibility matching (manipulating compatibility scores, suppressing user profile photos and the impact of rating scales and photos – the cool versus pretty test). All websites perform experiments, but at what risk?

OKCupid CEO Christian Rudder posted on the company’s blog: “If you use the Internet, you’re the subject of hundreds of experiments at any given time, on every site. That’s how websites work.” He goes on to say “OKCupid doesn’t really know what it’s doing. Neither does any other website…Most ideas are bad. Even good ideas could be better. Experiments are how you sort all this out.”

Like OKCupid, more and more data-centric companies are using rapid experimentation approaches (also called test-and-learn) to better understand how consumers react to their products and services.
This not only influences product development cycles, but drives more effective marketing and contact strategies.

But we already test-and-learn!
– the marketers shout. Not nearly enough, if at all!

A recent survey by WhichTestWon.com highlighted that more than 25% of online direct-to-consumer marketers aren’t doing any testing of their site or campaigns. In B2B organizations, more than 50% of marketers are not testing. With web and mobile becoming an important and integrated component of many organizations’ channel strategies, you can’t afford to not continually (and quickly) evaluate those strategies. To give a sense of scale, web giants (Amazon, eBay, Google, etc.) may run hundreds of tests per day. While that likely doesn’t make sense for your organization, there’s a happy medium between not enough and too much.

You have the data you need to target - use it!

You have the data you need to target - use it!

As consumers become more aware, knowledgeable and self-empowered, it becomes critical for the business to engage with the consumer from their point of view:

What do they want or need?
What’s the most effective way to engage with them?
How do I entice them into my store or website?
How do I get them to come back?

There’s no magic bullet or secret sauce. But with all of the data available to marketers today, you no longer have to stab in the dark to find out what your customers will respond to. Not taking advantage of insight around what customers respond to (or don’t!) seems almost criminal.

For one organization, their marketing group uses an iterative test-and-learn process in their marketing campaigns. As the campaigns are designed and executed, results are used to continually adjust the campaigns going forward. Marketing automation tools allow them to create multiple variants of a campaign, and data analysis drives a closed-loop measurement process. Creativity still remains an essential component of marketing, but data provides the connection between creativity and meaning. So while you might not be able to “Amazonify” your test-and-learn capability tomorrow, it’s never too late to get started.

Want a "real world" example of how Marketing Automation can make a difference in these scenarios? Check out this story about Staples - the office superstore giant that also operates the third largest e-commerce site in the world. Let us know what you think.



About Author

Rachel Alt-Simmons

Business Transformation Lead - Customer Intelligence Practice

Rachel Alt-Simmons is a business transformation practitioner whose expertise extends to operationalizing analytic capabilities vertically and horizontally through organizations. As the Business Transformation Lead for customer analytics at SAS Institute, she is responsible for redesign and optimization of operational analytic workflow, business process redesign, training/knowledge transfer, and change management strategies for customers. Prior to SAS, Rachel served as Assistant Vice President, Center of Excellence, Enterprise Business Intelligence & Analytics at Travelers, and as Director, BI & Analytics, Global Wealth Management at The Hartford. Rachel Alt-Simmons is a certified Project Management Professional, certified Agile Practitioner, Six Sigma Black Belt, certified Lean Master, and holds a post as adjunct professor of computer science at Boston University’s Metropolitan College. She received her master’s degree in Computer Information Systems from Boston University.


  1. Jim Hiepler-Hartwig on

    Hi Rachel,

    As Jake Sorofman, a consultant from Gartner, once told me and this holds true to your topic- Test and Fail, Learn and Scale.

    I enjoyed your blog!


Leave A Reply

Back to Top