Retail is one of the most time-sensitive industries. Find out how mega-chain Family Dollar Stores Inc., which operates more than 7,400 stores in 45 states, relies on high-performance analytics to shrink data processing windows from days to less than an hour and speed decision making. Scott Zucker, Vice President of  Business Services at Family Dollar, shares his views on big data, high-performance analytics technology, and how coupling both helps his company stay competitive.

Alison Bolen: Can you describe what big data means in the retail industry?

Scott Zucker: Big data is a paradigm shift for retailers and opens up a new world of possibilities for those retailers who can manage it. Big data is data that we could never have processed and managed just a couple years ago. Of course, it’s relative to your market. For instance, a large bank might not consider our data “big.”

Big data allows us to look at product, time and location – our critical analytical levers – at a much lower level than we ever did before. We might have looked at class or subclass, at total company, and then at month and sometimes at week. Now we’re looking at SKU, store and day. As we start going down to that level, the amount of information that we need to manage and analyze goes up exponentially. In the past five years, the amount of data that we manage has increased by 10 times, and most of that is structured data. Right now, we’re not integrating unstructured data into our data model– that’s the next frontier. You can just imagine how combining structured and unstructured data at that same rate of growth will change the dynamics of data management. If you don’t have the tools to deal with big data, you’ll be at a competitive disadvantage.

Bolen: When you have that much flying at you, you really have to prioritize and pick what you want answered, right?

Zucker: You have to be very disciplined. In this era of big data, we really have to move to solutions that work in memory or in database computing. If you don’t have that capability, there’s no question you will be left behind. Small data is gone. Data is just going to get bigger and bigger and bigger, and people just have to think differently about how they manage it.

Bolen: What opportunities does prioritizing data at the transaction level create for retailers?

Zucker: Profit is made – in other words, you win or lose – at the store/SKU level. For instance, we used to plan pricing at the store and SKU level for three to six month seasons and hope that the financials worked as anticipated. Now we can crunch through and analyze huge levels of data on a daily basis and make changes in a much shorter window. Working in collaboration with SAS on a big data issue we were facing, we recently dropped a process that took 36 hours down to less than 45 minutes.

That enables me to implement a promotion and within one day I could probably get a read on it.. It changes my speed to market dramatically so I can make changes midweek on that sort of stuff versus monthly. You can move a lot faster.

The difference between exceeding Wall Street expectations and meeting Wall Street expectations is being able to, obviously, see those trends in advance, analyze that data, and react quickly.

Bolen: How has high-performance analytics helped you become more agile?

Zucker: High-performance analytics lets you bring to market ideas, services, products and marketing plans much faster than you would ever think possible. No one ever does just one iteration of an analysis, right? There’s always the first iteration that goes to management, and then they want to look at it another way. We go back and forth for multiple iterations. Before high-performance analytics, that could take weeks or even a month. Now you can get data back in front of management the next day.

Not having to spend time managing the analyses or that process opens up time for you to do other things, such as operationalize analysis. There are certainly efficiencies to doing that. For instance, by not having to duplicate your data across multiple data marts, you’re able to reduce your costs across a myriad of categories such as storage, maintenance, labor, etc. Any time you can transfer support costs into innovation costs, that’s a plus. For every dollar you spend on support, you get zero dollars of value. So if you can apply that incremental effort towards better analyses, reporting, decision making and forecasting, that’s real value.

Bolen: What other benefits are there in shrinking analytical times that used to take days down to less than an hour?

Zucker: It’s the time savings. All analytical exercises are iterative and the more complex problems could take six, eight or 10 iterations. When you reduce to almost on-the-fly processing, it really makes a significant impact to your ability to move fast and shorten that time.

In retail, time is your enemy, meaning you always want to be closer to the season when making decisions. Unfortunately, for many retailers the long lead times for imported goods forces you to make preliminary Spring 2013 decisions while still in early 2012. If you can make those decisions in July or August when the season’s done and you’ve sold through most of your markdowns, then you’re going to make a much better decision than if you have to give sales estimates of product a year in advance.

Bolen: Thinking more broadly, even outside of your industry, how could high-performance analytics change the world?

Zucker: I listened to a podcast recently featuring [management expert]Gary Hamel where he talks about the end of management. He was making a case  that because of the dramatic rise in processing power coupled with collaboration tools,  front line team members are going to be as equipped as their managers are today. In other words, these front-line folks are going to have accurate, timely and actionable information at their fingertips – information that was usually only available to their managers.  Pushing information and decision-making down in the organization tends to  flatten a lot of things, including organizational hierarchies. I don’t mean to sound like author Thomas Friedman, but high-performance analytics will enable companies to empower their people, which in turn flattens existing business models. That type of change will alter the competitive landscape in most industries including retail.

High-performance analytics will afford us the ability to do things that we probably, today, rely on companies to do for us. People will be empowered in ways that, frankly, we haven’t even thought of yet.

You'll find more answers to your tough "big data" questions in this special 32-page report on high-performance analytics.