This guest post was written by Andy Pulkstenis, Director of Advanced Analytics for State Farm Insurance. He leads a team of advanced analytics professionals providing statistical analysis and predictive modeling support for the enterprise across a variety of business units. His background includes more than a decade of experience improving business strategies with designed multivariate experiments. Pulkstenis will be a presenter at the Analytics 2015 conference in Las Vegas, Oct. 26-27. We hope to see you there.
In my 20-year applied analytics career, I’ve been fortunate to witness to the evolving landscape of business analytics. One notable shift was when companies finally discovered the power of predictive modeling. Initially a tough sell in a world then-dominated by tradition, experience, & classic MBA methodology, it’s now difficult to imagine any company a serious contender if they don’t include predictive modeling in their analytic arsenal. Today when you examine most market leaders, statistical modeling is as firmly entrenched in the corporate culture as Microsoft Windows, SAS, khakis, and snarky Dilbert cartoons. Predictive analytics finally made it, but its cousin experimental design (i.e. statistical testing, or MVT, or DOE, or A/B testing, or test-and-learn, etc.) remains largely on the outside looking in.
Despite the potential to radically transform currently-held anecdotal beliefs about a business, unlock new or deeper insights into drivers of customer behavior, and truly optimize strategy delivery, the applied analytics community has been very slow to embrace statistical testing in the business world, even in the midst of a growing number of success stories. I can say with confidence that my conference presentations on business experimentation are consistently the best talks on testing at a given event – unfortunately because I’m typically the only speaker there talking about the topic!
I suspect this slow adoption rate is due to being blind to the power of testing, misplaced fears around complexity of implementation (in reality the degree of difficulty is on par with building and implementing predictive models), and a scarcity of skilled corporate practitioners (outside of manufacturing, agriculture, and biostatistics, that is). We ignore this valuable tool at our own peril.
Many of us face highly competitive business environments due to regulatory limitations, practical or economic constraints, or unique consumer dynamics. Until now advanced analytics has provided a bit of a competitive differentiator, reshuffling the deck and resorting the corporate winners and losers, but what happens when eventually nearly everyone in a market is using models and data science on a daily basis? Where can we go for that additional analytic competitive advantage? One answer may be statistical testing.
Even in a highly analytic culture that has embraced advanced analytics and modeling, business experimentation has significant value-add:
- Testing can be used to further optimize strategy assignment, improving customer value as learned insights enable you to truly offer the customers what they want or need as individuals at a given point in the customer lifecycle.
- It can lower operational costs by discovering efficiencies (and identifying inefficiencies) in your processes or operations center activities.
- Rigorous experimentation can inform product or strategy development, whether it’s a new credit card configuration, marketing message, internet offering, customer retention effort, or something else entirely.
To learn more about the power of business testing, how we are building a culture of testing and experimentation at State Farm, and how to start or improve testing at your company, drop by my breakout session “Do You Know or Do You Think You Know? Building a Test-and-Learn Culture at State Farm” at Analytics 2015 in Las Vegas on Monday, October 26th, 2015.