Not so Secret Santa: Using customer data to make it a Merry Christmas for all

small, wrapped gift on door stepI love Christmas, but there is one thing I never seem to get right: the office Secret Santa. Every year I draw someone I’ve barely met and fruitlessly dig around for clues – only to find myself hastily wrapping a scented candle/novelty mug at the eleventh hour. Merry Christmas Sandra in accounts!

Now that’s okay for Secret Santa, but it’s not an option for retailers, who need to know exactly what consumers want. Not only that, but they also need to know which channel they want to buy it on and at what price – so they can tailor promotions and get stock to where it needs to be in time for Christmas.

Consumers expectations for online shopping

As a recent SAS/Conlumino report shows, the rise of the omni-channel shopper makes this more challenging than ever before. More than half (57 percent) of consumers plan to shop around online for the lowest price once they’ve picked their gifts, with 55 percent planning to use click & collect for some of their Christmas shopping. And 79 percent want the option of returning online purchases to a physical store. So how can retailers get a sneak preview of consumers’ shopping lists to make sure they can grant everyone’s Christmas wishes?

Read More »

Post a Comment

Predictive modeling competitions: the competitive dimension of predictive analytics

After sporting events or major elections like the recent U.S. mid-term Senate elections, I tend to look back at how various predictions performed prior to these events, to find out who got it right. My interest in this was spawned after reading Nate Silver’s book The Signal and the Noise, and starting to follow his blog FiveThirtyEight. In Europe there is not as much of an established industry around organizing predictions for various topics, either using judgmental or predictive modeling approaches, as there is in the US. See for example, the election projection website.


In fact, one of the aspects of predictive modeling that has fascinated me throughout my professional career is that it’s somewhat easy to make fair comparisons between various alternative modeling approaches - at least when compared to other data mining techniques, where the quality of a given solution will also depend on soft aspects, such as interpretability (e.g. for clustering results) or “interestingness” (e.g., for association rules).

Comparing the results of election or sports predictions is done mostly in fun, and for news value. But today, some organizations are relying on our competitive natures to solve interesting and worthwhile problems through analytics competition.

Admittedly, you first have to agree on a set of proper statistical accuracy criteria to measure the predictive performance and a proper holdout sample to apply those criteria, and I’m not saying that this is an easy decision given the fact that analytics competitions are still a subject of ongoing academic research. Yet, once you’ve settled these two aspects, you’re ready to compare various algorithms in a contest situation and pick the winner or produce an ensemble model (where you combine different model’s predicted outcomes using a weighted or unweighted average).

Read More »

Post a Comment

Big data without analytics is just data

Data has value IF you can analyze it, said participants at a big data analytics roundtable at the Premier Business Leadership Series in Las Vegas. In attendance were executives from some of the largest Communications companies in the world including from the US, Canada, Turkey, Japan, Australia and the Philippines as well as executives from leading edge companies like Accenture, Microsoft and Google.


SAS participants included big data heavy hitters Jill Dyché, VP Best Practices; Paul Kent, VP Big Data; and Scott Chastain, Senior Manager Big Data. Each provided thought leadership on big data strategy, big data challenges, and specific use cases. This made for a lively discussion with some interesting themes emerging.

Theme 1: All participants had a big data strategy that includes Hadoop. However, what differed was whether IT was driving this strategy or the business units were wrestling with the big data. For this group, it was pretty much an even split. And, there were definite challenges on the misalignment between IT and the business unit with no clear owners of the data. Plus, acquisitions over time make the data less trustworthy, thus creating a need for data management and data governance. Read More »

Post a Comment

What a sentiment word cloud reveals about Apple Pay

I’m a huge Apple fan. I woke up at 3 AM EST a few weeks ago to order my new iPhone 6. Ever since this new device was announced, I’ve been intrigued with Apple Pay as well. The new iPhone 6 uses NFC (Near Field Communication) to enable contact-less payments. You just hold your iPhone 6 near the payment reader with your finger on TouchID, and a little vibration and beep let you know that your payment was successful. It sounds amazing, but that’s all I really knew about it. How could I quickly find out more about who was using it, where they were using it, and what people thought about it? I turned to Twitter.

Rather than analyzing tweets one by one, I decided to import a few thousand tweets into SAS Visual Analytics. I knew that I could build a word cloud, but I wanted to go one step further and get a feel for what people actually thought about Apple Pay and where it’s being used. SAS Visual Analytics 7.1 provides sentiment (an attitude that is expressed about an item) by document (a text based data item) and topic (a grouping of important terms in a document collection). It conveniently color codes these and allows you to see the results in drop-downs and at the top right of the word cloud, as shown below:

Apple Pay word cloud

A sentiment word cloud of terms associated with mentions of Apple Pay on Twitter. Click to enlarge.

Read More »

Post a Comment

Words of wisdom from big data early adopters


With big data, data governance challenges escalate in many ways:

  • The diversity of data sources means that there are minimal standards for data structure, definition, semantics and content.
  • The lack of control over data production means that you can’t enforce data quality at the source as you can do with your internal operational applications.
  • Because the problem is not about accessing and storing the data, the issue moves to the question of relevance and meaningfulness.
  • Privacy and regulations is also an important challenge for data governance bodies to address, by setting up retention policies to comply with privacy regulations.
  • Finally, there is a real risk of “fast trash” raised with the promises of real time analytics, meaning you get results fast but they might not be good results.

From the experience of big data early adopters, we can already draw some lessons learned and form some recommendations for those who are joining the party.

Read More »

Post a Comment

Statistics on big data: Take it easy, but do take it


When legendary travelling folk singer-songwriter Woody Guthrie summarized his approach to organizing workers, he said, “Take it easy, but take it.” Wise words to ponder in any case, but certainly whenever we put big data on the back burner to talk statistics instead. In the context of Big Data, I would say, take it easy by not using a lot of heavy statistical jargon when proposing your big data solutions - but do take statistics, do vigilantly educate others in the value they bring, and do continue to bring sound statistical perspective to the whole big data hype.

This is no small task. Big data is trending, now and most likely for the foreseeable future. It is the it thing right now! Evidence can be found in coverage of the IT industry:

It's important to realize that IT is challenging and requires patience, humility, but also determination. And, big data requires statistics.

Read More »

Post a Comment

Data privacy doesn't have to be scary

Marketers are walking a tightrope today with data privacy issues: Data can simultaneously bring customers and brands together and further drive them apart.

Recent data breaches, potential changes in data-privacy legislation and regulations loom large as customer expectations concerning marketing data continue to rise. As a result, today’s complex data issues are becoming a more like a horror flick. The outcome of these discussions, customer expectations, data-security lapses and rules-making efforts could change everything.

Amidst this backdrop, SAS recently conducted a global study on how consumers balance their need for privacy and personalization: Finding the Right Balance Between Personalization and Privacy.

Some of the findings are appropriately depicted in this chilling infographic. Read More »

Post a Comment

How you can use Hadoop to be as agile and innovative as a start-up!

Weighed down by what has gone before, and what is needed to keep the lights on, the CIOs at many organizations I have worked with have turned to Hadoop with the hope of utilizing it as a major component of an IT infrastructure and as part of their modernization and migration program for analytics and BI.

In my previous posts, I explored the world of traditional IT as it relates to Hadoop for those CIOs. We have looked at how Hadoop can be deployed without throwing away your warehouse and put forward some approaches people are taking around the data lake concept. All of these are generally focused on finding economically more viable approaches to what we expect to come in the future. If you like, these organizations are focused on improving what they do today while driving down costs.

In parallel to this, two questions have come up time and time again as I have worked with established organizations, of all sizes, over the past 6-12 months. Those questions are:

  • How can we, with all their legacy technology constraints, hard to change processes and need to focus on cost control, possibly enable all our business units to compete with nimble competitors that are starting to cast a shadow over many parts of our business?”
  • “How can we challenge the age old perceptions and approaches of IT, in order to support the business in getting answers to their questions?

Read More »

Post a Comment

How to improve sales and operations planning with analytics

business worker on laptop in warehouseHave you ever thought about how to improve your Sales and Operations Planning (S&OP) process beyond where you might be today? There's certainly no lack of advice on the topic of S&OP on the Internet. Some articles focus on the overall process while others focus on S&OP software and related support tools. In my experience, effective sales and operations planning is a combination of both process and technology – with a concentration on collaboration, stakeholder buy-in, executive  support and the adoption of analytics.

I like the Wikipedia definition of Sales and Operations Planning:

S&OP as an integrated business management process through which the executive/leadership team continually achieves focus, alignment and synchronization among all functions of the organization.

Read More »

Post a Comment

Big data in an age of uncertainty

42-53683135In recent times, Britain has increasingly developed an "inquiry culture." Whenever there’s malpractice or a scandal – be it the Leveson inquiry or the recent investigation into the Mid Staffordshire NHS Foundation Trust – an inquiry serves to give citizens, public leaders and governing bodies an explanation of how and why things went wrong.

As a result, inquiries often uncomfortably uncover the truth, with no passing the buck or hiding behind ifs and buts. But what if you had evidence to back up your decision for enacting a particular policy in the first place? In today’s uncertain times, no one knows when the next bump in the road is going to occur. But one thing is certain – evidence is necessary. Doctors use it to make diagnoses based on patient symptoms. Police officers need it to obtain search warrants. So when it comes to government leaders enacting policies, wouldn’t evidence be just as useful?

When it comes to policy making, the government has much to gain from adopting an approach based on empirical evidence. We recently conducted some research in conjunction with Dods which revealed that, while evidence-based decision making has improved under the coalition government, further progress is needed to realise the full potential of its big data. Evidence isn’t hard to come by – the very nature of the public sector means that it is the largest source of big data in existence. Yet, without the right training and solutions, civil servants are unable to use data to its best effect.

Read More »

Post a Comment