With the number of streaming data sources growing by 20 percent annually, I’m not the first person to notice this trend. As executives in every industry start to see the potential in these data streams, they’re all asking the same types of questions.
Recently, I’ve talked to banking CEOs, telecom CIOs, retail CMOs and heads of government agencies, and the conversations are very similar. Primarily, they’re asking, "How can I use all this data to discover new opportunities?" More specifically, the questions – and basic answers – sound like this:
Q. With all the data out there, how can I store it efficiently?
Q. What if I need the data right away? How can I get it quicker?
A. Streaming data.
Q. Now that I have access to all this data, where do I start?
A. Data visualization.
Q. How can I use this data to discover new possibilities?
A. Advanced analytics.
Q. How can I get my analysis done quicker to get a jump on the competition?
A. In-memory, distributed processing.
Let’s look at each of those answers a bit more closely:
Hadoop. I cannot over emphasize the importance of understanding what you’re capable of doing with Hadoop. The best way to work with Hadoop is to create an analytical platform, so you can do more than just store your data there. You need to be able to access data in Hadoop, run analytics inside the Hadoop environment, inside the cluster, and even run in-memory calculations inside the Hadoop cluster.
Data management. Data management for analytics is not the same thing as data management for an enterprise data warehouse. Analytical data management adds value along the way by completing summarizations and adding metadata to variables before putting them into memory.
Visualization. Visual analytics provides capabilities beyond general business reporting, by giving you a way to explore and understand all your data. Visual statistics takes it even further by making it easy to explore, discover and predict by implementing statistical algorithms without having to write any code.
Advanced analytics. This one is a no brainer. To see real value in your data, you need to move beyond basic analytics to optimization, forecasting, text analytics, event stream processing and more. With a wide range of advanced analytics at your disposal, you’ll reveal optimal and lucrative opportunities, expose risks, deepen customer understanding, and deliver predictive insights.
In-memory, distributed processing. Now that Hadoop is easily available, storing your data is no longer an issue. The real issue is whether you can process it quickly enough. In-memory processing can help you keep up with increasing data demands. You can not only do things quicker but you can do new things -- and change the way you’ve always done things in the past, so true innovation can happen.
Your competitors might be looking at one or two of these areas, but if you can develop a strategy that excels in all five, you’ll operate more efficiently, make faster, smarter decisions, and get big value out of data from the Internet of Things.