How can big data coupled with high-performance analytics (HPA) help retail companies tailor their offerings in a way that is beneficial to both the company’s bottom line – and the consumer? Eric Williams, recently retired Chief Information Officer at Catalina Marketing, explains in this interview.

Alison Bolen: How do you define big data? And what value can it bring to an organization?

 Eric Williams: The term “big data” is relative to the individual company. Some companies will call big data something that is sizeable in nature, when to a larger company, that same amount of information might be easy to handle. The reason companies get into managing big data is that they have a multitude of databases that are disparate and unconnected; therefore, they are unable to answer many of the growing challenges that companies are facing today, such as what products are selling, what’s the association of one product to another, what do my consumers look like, what is the marketplace doing? Applying analytics quickly can help companies answer those questions that have been plaguing them for years.

 Bolen: Can you describe one of those opportunities in more detail – and explain how big data would benefit the organization?

 Williams: For example, companies want to be able to understand as quickly as possible if the launch of a new product is working. Given that there are more than 20,000 new products introduced in the US marketplace annually, products like new flavors of gelatin or yogurt, this isn’t easy. It used to take months to understand how well a new product was doing by looking at sales data, and adding surveys and exit interview data. By pulling in detailed sales data and using analytics, companies can figure out in a matter of days whether a new product is a success, and quickly decide to expand the launch or discontinue it. Now these companies can literally have access to that data in a matter of days after the launch of a product.

 Bolen: Specifically, where does high-performance analytics come into play?

Williams: With large data volumes it helps if you do not to have to move the data, because that can take as long as a week. This is where a key feature of high-performance analytics, in-database scoring, really helps. Catalina, for example, would frequently have databases with 1.2 trillion rows of information. By bringing the analytics into the database rather than bringing the data to the analytics, it allowed us to speed up functionality hundreds of fold. So the same number of people, who might have been able to build 40 to 50 models a year, can now build 600 a year.

Bolen: What can that mean for a company?

Williams: It helps increase revenues. Revenues are based on the number of clients we can help. If you can’t do it quickly, then you need more staff. At Catalina, we were actually able to quantify speed increases as direct revenue to the corporation. Our finance team was able to truly understand if you sped up a process X number of minutes, what that meant in potential revenue to the company.

Bolen: How do you think big data and high-performance analytics might influence the retail industry going forward?

Williams: All industries will be affected by this. High-performance analytics encourages all the business units to work together. You’ve got people looking at data from a financial view, from a marketing view, a buying group that needs to forecast, and an operations and sales team that wants to know what’s selling and why. You can begin to coordinate across business indices.

This allows businesses to look beyond sales and volume, and focus on per-customer profitability. I was just talking to a large specialty retailer the other day that has this new item, which is selling well. The first thing I asked was, “What’s the number-one item that’s always associated with that top-selling item in the basket?” They went back and ran the numbers – and it took awhile, which is where HPA would help – but they discovered that the most profitable item was purchased with a loss leader. So now they’ve re-merchandised this one section of the store to emphasize the hot, new profitable product and the loss leader. There has been a marked improvement in sales for that store.

Now, imagine if you could re-run that same model every single day or separately for every store to figure out how to get the most profitable item in your store into the consumer’s shopping cart. With high-performance analytics you can do that. This wasn’t even thinkable five years ago.

Bolen: We’ve talked about how analytics benefits companies, how does it benefit the consumer?

Williams: One hundred years ago, the merchant knew you, right? But now, it’s very challenging for companies to make recommendations for additional products or services at the individual level, based only on historic information. They can do it for segments of customers but not on a one-on-one basis. Cheap data storage and high-performance analytics are going to change that.  Because now we can begin to have sales associates who understand what’s in stock or what they have coming in a few weeks that you might like to purchase based on your past purchase history.  The information will be made available on mobile devices so every sales associate can access it quickly and provide a personal shopping experience to every customer.

This is day 19 of our "HPA once a day" blog post series. To read more, see all of the high-performance analytics posts on this blog or follow the high-performance analytics rss feed. 

You can also find more answers to your tough "big data" questions in this special 32-page report on high-performance analytics.