When I was young, the use of analytics wasn’t widespread – even in very large companies. Organizations relied on their leaders’ experience built on years in the industry. The more experience and knowledge a leader had, the better the decisions they made and the more successful the business was. The introduction of business intelligence and predictive analytics technologies has triggered a shift toward data-driven decision making. That’s a good thing and a bad thing.

Often, basing your decisions on what the data says can be the safest route. But, Jay Liebowitz says, we still need to include our intuition as part of the decision-making process.

Liebowitz is the Orkand Endowed Chair in Management and Technology in the Graduate School at the University of Maryland University College (UMUC). He’s written several books about big data, analytics and decision making. Most recently, Liebowitz published “Bursting the Big Data Bubble: The Case for Intuition-Based Decision Making.”

Liebowitz recommends a balance of trusting your gut reactions without being overconfident. Use analytical techniques to validate or disprove your gut reaction, he says, and then learn from the exercise. He spoke Monday at the Analytics 2014 conference sponsored by SAS.

Trust your gut?

In  the journal article, “When Should I Trust My Gut?” Erik Dane and his associates found that intuition is often as good as analytics if you are very experienced in the domain where you are making the decisions. Liebowitz agrees and warns that the current trend to constrain employee hiring costs by cross-training employees can mean that key employees don’t develop the expertise they’ll need to make sound judgments.

Liebowitz went on to quote an MIT Sloan Management review article describing the value of intuition over statistical analysis. “For many complex decisions, all the data in the world can’t trump the lifetime’s worth of expertise that informs one’s gut feeling, instinct, or intuition.” Leibowitz says gut instinct can be taught, but that it requires time. An example he uses to illustrate the value of intuition in decision making is the career of Wayne Gretsky. Gretsky has been called the smartest hockey player ever. He defined the game for generations to come because of his uncanny sense of where the puck would be and where his team mates were on the rink.

In his autobiography he writes, “Some say I have a 'sixth sense' . . . Baloney. I've just learned to guess what's going to happen next. It's anticipation. It's not God-given, it's Wally-given. He used to stand on the blue line and say to me, 'Watch, this is how everybody else does it.' Then he'd shoot a puck along the boards and into the corner and then go chasing after it. Then he'd come back and say, 'Now, this is how the smart player does it.' He'd shoot it into the corner again, only this time he cut across to the other side and picked it up over there. Who says anticipation can't be taught?”

Of course, Liebowitz doesn’t discount the value of analytics. To the contrary – he believes decision makers should rely on their expertise, but then prove or disprove it based on the data.

Downside to gut reaction?

Another article on cfo.com about big data says, “We generally have good intuition about things that are similar to what we encounter every day … but we have poor intuition about things that are outside of the everyday.” That makes sense – but what about the times you have to make a decision quickly and you don’t have the benefit of analytics? Think about all of the possibilities. For example: Gary Klein uses a ‘pre-mortem’ technique where he critically evaluates the worst possible outcomes of his decision based on all of the information available at the time.

What can you do to improve your decision making from a business intuition perspective?

• Respect your intuition without rejecting it outright or following it blindly.
• Ask yourself what prompted your gut reaction.
• Review the evidence.
• Elicit good feedback from other experts.
• Prove or disprove your hunch (this is a good place for analytics).

For more about Liebowitz’ theories about intuition versus analytics, read, “Educating informed 'intuitants.'” In this SAS Insights article, Liebowitz discusses the new UMUC online M.S. in Data Analytics degree where up-and-coming leaders are taught “basic and advanced skills to support strategic and tactical decision making in the new big data world.”

Hear more from Liebowitz on this topics in this Inside Analytics video: