Since the advent of Amazon, in particular, retailers have been trying to find new ways to attract customers and meet their ever-increasing demands. Stores have moved online, adopted "bricks and clicks" models, expanded and contracted, and generally changed. As Toni Calvo recently pointed out, there is no one-size-fits-all for retail.

Two recent innovations are particularly important. First is the rise of hyperpersonalisation. Retailers have started to approach customers as segments of one, tailoring offers very precisely to individuals in real time to maximize engagement. This is both online and in store, where smart shelves are able to provide offers as customers pass by.

The second innovation or trend is what underpins this personalization: the use of analytics. Increasingly retailers are seeing that they can use analytics to improve operations across the board. Analytics is useful for looking at customer behaviour and improving customer experience. But retailers can also use it to make better stock and inventory decisions. Ensuring that stock matches demand reduces both waste and lost sales, improving profitability.

But if innovation is eminently possible in retail, why have we seen so many august retail names going to the wall in recent years? Some, like Woolworths in the UK, failed to innovate. However, we have also seen a number of innovative organizations fail, including J.C. Penney, the US department store that deliberately brought in a new CEO to drive innovation.

A question of innovation

There are a number of reasons why innovations may fail, and they are not always obvious. In J.C. Penney’s case, the reason for failure was that its innovations were not right for the company’s core customers. The new CEO was from Apple, and brought in all the innovations that had worked so well in Apple stores: wide aisles, bright lights, juice bars and so on.

The company’s core customers, however, weren’t really interested in the new innovations, and when the company stopped its regular sales and voucher systems, they deserted in droves. The innovations were simply not necessary for the existing customers. Unfortunately for the store, the changes did not attract younger customers either.

Innovation is a question of data. That is no secret. But how does a company transform its data into business models to drive change? That's what we asked 100 business leaders, and we got astonishing insights: Innovation - From data to the business.

Context is all-important in retail, as in so many other areas. Innovating for the sake of it, or just to be "cool," really does not work. Innovations generally have to be needs-based: They must solve a problem for customers. What’s more, customers have to recognize that they have this problem, or they will not buy into the solution. J.C. Penney forgot that – and it also forgot that it is far cheaper to retain an existing customer than attract a new one.

When innovation is a step too far

Participants in the SAS Innovation at Scale study support both getting it right for customers and needs-based innovation. For example, organizations that use analytics are experimenting with machine learning models. But there was doubt about whether these models were actually superior.

Sometimes, the traditional method of doing something – even an innovation itself, like statistical modelling – can actually be better than a new way. And organizations should not adopt new methods just for the sake of change. Interestingly, many suggested that the problems with new machine learning models may not be in the results, which are usually very good. Instead, the problem was that AI-based models are harder to explain and understand. Like J.C. Penney’s voucher system, existing models sometimes don’t need to be improved.

Other participants talked about innovations failing when they were not linked to a business problem or challenge, or when they were too far-fetched or futuristic. Major innovations were good, provided they headed in a known direction, and were consistent with the organization’s culture and strategy.

Retail-specific issues?

These issues are not specific to retail. The idea that innovations must be needs-based and solve problems, rather than simply be "cool," is not in any way exclusive to the retail sector. Neither is the idea that context matters.

These issues are perhaps, however, more obvious in retail than in many other sectors. After all, customers have a lot of choice about where they go in retail and aren’t at all bothered about shopping around to find what they want. It could be that other sectors can, and should, learn from the experience in retail to prevent themselves from going the way of J.C. Penney.

We also conducted a SASChat - Innovation@scale on Nov 14. See what participants were talking about. We asked five questions. Find an excerpt of that amazing discussion on twitter: 

How is analytics changing the scope of #innovation?
(1) As #innovation is all about changing the way of doing things, Analytics can bring a new dimension of being able to predict what results your innovation efforts will bring (Andreas Kitsios).
(2) I think #analytics is building the scope of #innovation more than changing it.(GorkemSevik)

What is the role of scalable analytics capability in driving #innovation at scale?
(1) #Analytics at scale can support decision makers in finding hidden patterns in tons of data. It wouldn't be possible in any other way (Federica Ballerini).
(2) To enable your business to have exponential growth ability  (Igor Dsiaduki).

How has analytics helped athletes collaborate more effectively with their support teams?
(1) Support teams want to know how an athlete's body and mind are progressing, responding to training and competition and recovering. Analytics may not provide the answer but it allows support staff to start the conversation needed for collaboration and build trust (Reece Clifford).
(2) Absolutely. Things like sleep can be monitored if all parties happy, so that what's happening elsewhere (away from training venue) can help create a clearer picture of the athlete's state of mind and body. After all, the 2 are interlinked! (David Smith).