Is this big data? Or is it complex data?


Yes, we have all heard about the 3 V's of big data by now. Or the 4 V's or possibly 5 V's, depending on who you ask.   However, I would argue the value of data isn't necessarily tied to the "volume" part of big data (unless of course you are a storage vendor, and then the value is all about volume and storage).

Everyone else should be focused on how best to monetize complex data.  Obviously, there can be a relationship between complex data and big data but let's define complex data so we can talk about solving business problems. Complex data is found in all industries, but for this example I am going to use the data associated with drilling a well (land based or off-shore).

Complex data streams from an oil rig
Complex data streams from an oil rig (click to enlarge)

In this case, the data coming off one drilling rig may not meet the "volume" requirements of big data. However, it does meet the criteria of variety: multivariant, multivariate, multidimensional, and stochastic (MMMS) - it really doesn't get more complex than this type of data. It also meets the velocity criteria since it needs to be analyzed in near real-time in order to provide the drilling engineer with feedback to help guide drilling an optimal well.  (NOTE: An optimal well is not the same as drilling a well as fast as possible.)

The data, and the use of the data, is complex because you need to continuously provide input into a variety of different predictive models (all of which interact or impact each other) in order to provide the best possible well to be drilled from both a quality and speed perspective.

This type of analysis was really not possible until recently, when new technologies in both hardware, software, and the ability to add more sensors on all parts of a drilling rig and to collect this data efficiently.  We now have the capability of building the necessary system to allow for near real-time feedback to drilling operators.

As a matter of fact, a group of us at SAS have a patent in process detailing how this system can be built by combining event stream processing and a variety of advanced analytic capabilities (all available within the SAS industry analytic framework) to do just what I described.

You can read more about this and other complex data topics in the papers our team has authored for the Society of Petroleum Engineers.


About Author

David Pope

Technical Leader, Senior Manager US Energy

David leads the pre-sales technical team for SAS US Energy which solves business problems in the Oil & Gas and Utilities industries using advanced analytics. He is a lifetime learner who enjoys sharing information and helping others to grow their careers. He earned a BS in Industry Engineering and a Computer Programming Certificate from North Carolina State University. Furthermore, he has over 29 years of business experience working with SAS across R&D, IT, Sales and Marketing in the Americas and Europe. He is an expert in working with data and producing insights through the use of analytics. David has presented at SAS Global Forum, the 2012 SAS Government Leadership Summit, IBM’s Information on Demand(IOD), EMC World, CTO Summit Conferences, is the author of the book: "Big Data Analytics with SAS", and he currently holds 14 patents for SAS in several countries: US, CA, Norway, UK, China, Mexico, and Hong Kong.

Comments are closed.

Back to Top