Earlier this week I managed to catch up briefly with Christoph Sporleder, Vice President Centers of Excellence for EMEA & Asia Pacific, to talk Hadoop, big data and get some of his views on where we might be headed with big data.

Mark Torr: Is big data just a buzz word?

Christoph Sporleder: Actually, if you had asked me three to four years ago, I would have said yes (and I actually did). Of course, it is a label built by analysts, promoted by marketers, which makes it look suspicious at first glance. Now seeing what is happening at companies and how big the effect is I will never again say it’s just a buzz word.

Torr: Can you share a little of your view on why companies need to pay attention to Hadoop and big data?

Sporleder: To start with we should separate the two things. Hadoop and big data are not the same. If you use Hadoop, it does not mean you have defined a big data strategy or even that you have big data. It is important to state that Hadoop can benefit organizations with “small” data as well as being vital for those who want to deal with Big Data. In my view, companies need to pay attention and build a strategy for big data, as this is probably one of the biggest drivers of change that organizations have experienced over the past couple of years due to the impact on core business models that big data causes. At the same time, companies also need to pay attention to Hadoop as this is one of the enabling technologies you need to master big data at any sort of sensible economic price point.

Torr: Do you think Hadoop is ready for primetime?

Sporleder: I think we need to be more precise here… core Hadoop yes, it is. But there are many projects being built around Hadoop in the community, and from a corporate perspective, each one of them needs to be looked at carefully.

Torr: What do you think the major challenges are around big data / Hadoop that are impeding adoption?

Sporleder: Skills, skills and skills again. Hadoop skills are actually relatively rare, companies will need to be creative in acquiring, building and maintaining skilled contributors. This ranges from the
technical Hadoop Expert through to the Data Scientist as well as the need to establish the role
of a Chief Data Officer.

Torr: Which industries do you think are going to be most disrupted over the next 12-18 months?

Sporleder: Obviously the disruption is already well under way in the Telecommunications industry. Telecommunications company margins decline significantly when they are “only“ an infrastructure provider so they are busy trying to work out what value added services they can offer, and a lot of that is empowered through the use of big data. But, if we also look into other industries, we see plenty of disruption ahead. Car makers will look more like services companies, insurance organizations will leverage telematic data for dynamic tariffs and the list just goes on like this. I do believe that literally all industries are going to be impacted and it is now just a matter of how fast the individual companies or organizations will be able to adopt and embrace the change.

Torr: Can you share any examples of how companies are already benefiting from big data or Hadoop based on your work?

Sporleder: My favourite is a Telco in Scandinavia who has started their big data journey in the form of a ”big data laboratory” where they are empowering experimentation around various use cases based on big data … to either prove them or to fail fast. It is definitely the approach that we see being discussed and actually put into practice, more and more. I think your previous blog post covers that topic pretty well. Beyond that we are working with multiple Telcommunication, Financial Services and Retail organizations as they embark on their big data journey. Public sector is also starting to do some pretty interesting things, such as combining social media data in the public domain to help them look for indicators of possible fraud.

Torr: What do you think the next big development will be in the Hadoop / big data space?

Sporleder: After the organization and building of analytical models based on big data has been mastered, I think the deployment of these models into event streams at the edges of corporate ecosystems will be the next big thing. If you like, that is where we will see the convergence of big data, analytics and the Internet of Things.

Torr: Thanks for your time today Christoph.

If this post was interesting be sure to check out the Eight Considerations for Utilizing Big Data Analytics with Hadoop TDWI On-Demand Webinar.