Yes, we have all heard about the 3 V's of big data by now. Or the 4 V's or possibly 5 V's, depending on who you ask. However, I would argue the value of data isn't necessarily tied to the "volume" part of big data (unless of course you are a storage vendor, and then the value is all about volume and storage).
Everyone else should be focused on how best to monetize complex data. Obviously, there can be a relationship between complex data and big data but let's define complex data so we can talk about solving business problems. Complex data is found in all industries, but for this example I am going to use the data associated with drilling a well (land based or off-shore).
In this case, the data coming off one drilling rig may not meet the "volume" requirements of big data. However, it does meet the criteria of variety: multivariant, multivariate, multidimensional, and stochastic (MMMS) - it really doesn't get more complex than this type of data. It also meets the velocity criteria since it needs to be analyzed in near real-time in order to provide the drilling engineer with feedback to help guide drilling an optimal well. (NOTE: An optimal well is not the same as drilling a well as fast as possible.)
The data, and the use of the data, is complex because you need to continuously provide input into a variety of different predictive models (all of which interact or impact each other) in order to provide the best possible well to be drilled from both a quality and speed perspective.
This type of analysis was really not possible until recently, when new technologies in both hardware, software, and the ability to add more sensors on all parts of a drilling rig and to collect this data efficiently. We now have the capability of building the necessary system to allow for near real-time feedback to drilling operators.
As a matter of fact, a group of us at SAS have a patent in process detailing how this system can be built by combining event stream processing and a variety of advanced analytic capabilities (all available within the SAS industry analytic framework) to do just what I described.
You can read more about this and other complex data topics in the papers our team has authored for the Society of Petroleum Engineers.
