In the oil and gas industry, analytics are used to improve both upstream and downstream operations, from optimizing exploration and forecasting production to reducing commodity trading risk and understanding customer's energy needs.

If you plan to derive value from the digital oil field, big data, and analytics, one of the first things you'll need is a proper data management strategy.

Your oil and gas data management strategy should consider data from existing systems as well as new sources, including all the sensors being added to equipment used in the upstream as well as the downstream areas of the oil and gas industry.

With data flowing in from exploration, drilling, production and usage meters, the rapid growth of data sources presents a data management challenge for the industry.

Consider this excerpt from Chapter 2 of Keith Holdaway's book, Harness Oil and Gas Big Data with Analytics:

Oil and gas operators are faced with a daunting challenge as they strive to
collate the raw data that serve as the very foundation of their business success, transforming that raw data into actionable knowledge. However, with the exponential growth in data volumes and the breadth of siloed, disparate data sources increasing at ever-faster rates, the industry is realizing that data management is fundamental to their success.

Even as data volumes grow, it's important to remember that data management isn't just about storing larger and larger amounts of data but figuring out what data is most relevant for the problems being explored, and then processing this data in a timely manner. Depending on your analytics needs, that processing might involve simple queries or more intensive resources like analytic workloads with more iterations to produce more informative results.

Data management involves handling streaming data, near real-time data, data stored on disk, and data stored in archive systems and being able to integrate all or some of these together in the best formats for analytics and reporting.

It's important to incorporate predictive analytics in some or all of the data gathering or data reporting points.

For example, event stream processing is part of most modern data management strategies because it  allows you to apply if-then type rules to data in stream. It also applies analytics to your sensor data in stream, and take actions with that data as a result of the activity in the data.

Data quality capabilities should also be applied to clean up messy or missing data so decision makers receive quality information that helps them make better decisions in running your organization.

For more details on these topics, please read our whitepaper, Analytic Innovations Address New Challenges in the Oil and Gas Industry.