Words of wisdom from big data early adopters


With big data, data governance challenges escalate in many ways:

  • The diversity of data sources means that there are minimal standards for data structure, definition, semantics and content.
  • The lack of control over data production means that you can’t enforce data quality at the source as you can do with your internal operational applications.
  • Because the problem is not about accessing and storing the data, the issue moves to the question of relevance and meaningfulness.
  • Privacy and regulations is also an important challenge for data governance bodies to address, by setting up retention policies to comply with privacy regulations.
  • Finally, there is a real risk of “fast trash” raised with the promises of real time analytics, meaning you get results fast but they might not be good results.

From the experience of big data early adopters, we can already draw some lessons learned and form some recommendations for those who are joining the party.

Read More »

Post a Comment

Statistics on big data: Take it easy, but do take it


When legendary travelling folk singer-songwriter Woody Guthrie summarized his approach to organizing workers, he said, “Take it easy, but take it.” Wise words to ponder in any case, but certainly whenever we put big data on the back burner to talk statistics instead. In the context of Big Data, I would say, take it easy by not using a lot of heavy statistical jargon when proposing your big data solutions - but do take statistics, do vigilantly educate others in the value they bring, and do continue to bring sound statistical perspective to the whole big data hype.

This is no small task. Big data is trending, now and most likely for the foreseeable future. It is the it thing right now! Evidence can be found in coverage of the IT industry:

It's important to realize that IT is challenging and requires patience, humility, but also determination. And, big data requires statistics.

Read More »

Post a Comment

Data privacy doesn't have to be scary

Marketers are walking a tightrope today with data privacy issues: Data can simultaneously bring customers and brands together and further drive them apart.

Recent data breaches, potential changes in data-privacy legislation and regulations loom large as customer expectations concerning marketing data continue to rise. As a result, today’s complex data issues are becoming a more like a horror flick. The outcome of these discussions, customer expectations, data-security lapses and rules-making efforts could change everything.

Amidst this backdrop, SAS recently conducted a global study on how consumers balance their need for privacy and personalization: Finding the Right Balance Between Personalization and Privacy.

Some of the findings are appropriately depicted in this chilling infographic. Read More »

Post a Comment

How you can use Hadoop to be as agile and innovative as a start-up!

Weighed down by what has gone before, and what is needed to keep the lights on, the CIOs at many organizations I have worked with have turned to Hadoop with the hope of utilizing it as a major component of an IT infrastructure and as part of their modernization and migration program for analytics and BI.

In my previous posts, I explored the world of traditional IT as it relates to Hadoop for those CIOs. We have looked at how Hadoop can be deployed without throwing away your warehouse and put forward some approaches people are taking around the data lake concept. All of these are generally focused on finding economically more viable approaches to what we expect to come in the future. If you like, these organizations are focused on improving what they do today while driving down costs.

In parallel to this, two questions have come up time and time again as I have worked with established organizations, of all sizes, over the past 6-12 months. Those questions are:

  • How can we, with all their legacy technology constraints, hard to change processes and need to focus on cost control, possibly enable all our business units to compete with nimble competitors that are starting to cast a shadow over many parts of our business?”
  • “How can we challenge the age old perceptions and approaches of IT, in order to support the business in getting answers to their questions?

Read More »

Post a Comment

How to improve sales and operations planning with analytics

business worker on laptop in warehouseHave you ever thought about how to improve your Sales and Operations Planning (S&OP) process beyond where you might be today? There's certainly no lack of advice on the topic of S&OP on the Internet. Some articles focus on the overall process while others focus on S&OP software and related support tools. In my experience, effective sales and operations planning is a combination of both process and technology – with a concentration on collaboration, stakeholder buy-in, executive  support and the adoption of analytics.

I like the Wikipedia definition of Sales and Operations Planning:

S&OP as an integrated business management process through which the executive/leadership team continually achieves focus, alignment and synchronization among all functions of the organization.

Read More »

Post a Comment

Big data in an age of uncertainty

42-53683135In recent times, Britain has increasingly developed an "inquiry culture." Whenever there’s malpractice or a scandal – be it the Leveson inquiry or the recent investigation into the Mid Staffordshire NHS Foundation Trust – an inquiry serves to give citizens, public leaders and governing bodies an explanation of how and why things went wrong.

As a result, inquiries often uncomfortably uncover the truth, with no passing the buck or hiding behind ifs and buts. But what if you had evidence to back up your decision for enacting a particular policy in the first place? In today’s uncertain times, no one knows when the next bump in the road is going to occur. But one thing is certain – evidence is necessary. Doctors use it to make diagnoses based on patient symptoms. Police officers need it to obtain search warrants. So when it comes to government leaders enacting policies, wouldn’t evidence be just as useful?

When it comes to policy making, the government has much to gain from adopting an approach based on empirical evidence. We recently conducted some research in conjunction with Dods which revealed that, while evidence-based decision making has improved under the coalition government, further progress is needed to realise the full potential of its big data. Evidence isn’t hard to come by – the very nature of the public sector means that it is the largest source of big data in existence. Yet, without the right training and solutions, civil servants are unable to use data to its best effect.

Read More »

Post a Comment

True digital marketing intelligence is all in the data!

At a recent MESA - Hollywood IT Society Digital Marketing conference, Claudio Ludovisi, SVP, Operational Strategy NBCU Entertainment for Marketing & Digital took a live poll of about 200 or so media marketing executives in the audience and asked them to “Think about what it takes to achieve true multiplatform marketing ... what ONE word comes to mind?”

Within seScreen Shot 2014-10-07 at 2.18.22 PMconds this word cloud began to appear and it was apparent that the winner was data! Besides marketing synergy, audience engagement, and strategy, it is all about the data – specifically big data that comes from:  offline first-party CRM and transactional data; online digital analytics data (structured and unstructured, known and anonymous) from web analytics and advertising offerings; and third-party data from data management platforms.  All of these are equally important and are a summation of how today’s ever connected consumers are interacting with brands and content.

Read More »

Post a Comment

Can data governance stop data lakes from becoming data swamps?

At a time when most of the buzz about big data is focused on the technology and the exciting opportunities that it creates, I find that little attention is being given to the way that big data is impacting the culture of organizations. In particular, I'd like to see more discussions on how organizations should adapt to prevent their data lakes from becoming data swamps!

The adoption of big data technologies has the potential to radically change the way organizations make decisions and how the business and IT collaborate in managing data assets to create useful insights.

New data sources and new business scenarios are calling for an even stronger data governance framework.

Read More »

Post a Comment

Behold, pathetic mortals, a vampire's data visualization dashboard

Valuable information for vampiresGreetings, insignificant souls.

You may call me … Gregor Castamere. It is not my true name. I stalked this strip of land when it was called New Amsterdam. My family’s name is well known. I have lived for more than 600 years, concealing my terrible secret. I am a vampire.

Why, after six centuries, would I risk my anonymity? I am in possession of such useful data visualization technology that I am compelled to share it with the world. The people of SAS were likewise … compelled, to provide this forum. They will be rewarded with their lives.

Today, I share with you my SAS dashboard. It contains information a vampire needs, at a glance, to hunt, feed and survive.

Read More »

Post a Comment

Four tips for putting business users in touch with Hadoop

The Global Hadoop market was valued at $1.5 billion in 2012 and is expected to grow at a compound annual growth rate of 58.2 percent, to reach $50.2 billion by 2020, according to a Hadoop Market Analysis report prepared by Allied Market Research

There is no doubt that IT teams are often taking the lead in driving the adoption of Hadoop, as they look to optimize spend and prepare for a future featuring a lot more data, but they are not alone. Increasingly, analysts and business users are starting to significantly influence that growth as the potential of Hadoop becomes clear to them.

Based on my interactions with analysts and business leaders, there are a four main drivers I often hear users talking about when it comes to why they want to see the adoption of Hadoop:

  • Access to more data: Analysts and business users are looking to have quick access to more data, oftentimes not pre-aggregated, in order improve the accuracy of reports and analytical models. Access to more historical and granular data can help analysts to further tailor messages to a specific customer or market segment for example.
  • Access to new data sources: Quick access to new data sources such as Social Media, Open Data, Dar Data has fast became the need of the day in order to better understand consumer behavior and market shifts. These new data sources require quicker on-boarding of the data to be useful for decision making purposes and in many cases business users and analysts are frustrated by their lack of access to these emerging data sources.
  • Existing data warehouse projects are slower in on-boarding new data sources: Big data sources such as Web logs, sensor and machine data, social data, etc., are blending up with enterprise data for better & accurate business insights. However, many on-going DWH initiatives were or are focused on dealing with structured data. According to a recent Deloitte article in the Wall Street Journal, “90 percent of the data warehouses process just 20 percent of an enterprise’s data. Consequently, many enterprises have only been able to use their data warehouses for historical analysis and past performance reporting.” Fulfilling requirements of new big data sources for business insights are seeing longer turnaround cycles and this is frustrating business users and analysts.
  • Self-service data discovery & analysis: Business users are looking to explore new data sources themselves for business insights so that they can quickly operationalize business decisions either for competitive advantage or in order to avoid monetary risks. Read More »
Post a Comment