Oil and gas data management overview


Black background with shiny squaresIn the oil and gas industry, analytics are used to improve both upstream and downstream operations, from optimizing exploration and forecasting production to reducing commodity trading risk and understanding customer's energy needs.

If you plan to derive value from the digital oil field, big data, and analytics, one of the first things you'll need is a proper data management strategy.

Your oil and gas data management strategy should consider data from existing systems as well as new sources, including all the sensors being added to equipment used in the upstream as well as the downstream areas of the oil and gas industry.

With data flowing in from exploration, drilling, production and usage meters, the rapid growth of data sources presents a data management challenge for the industry.

Consider this excerpt from Chapter 2 of Keith Holdaway's book, Harness Oil and Gas Big Data with Analytics:

Oil and gas operators are faced with a daunting challenge as they strive to
collate the raw data that serve as the very foundation of their business success, transforming that raw data into actionable knowledge. However, with the exponential growth in data volumes and the breadth of siloed, disparate data sources increasing at ever-faster rates, the industry is realizing that data management is fundamental to their success.

Even as data volumes grow, it's important to remember that data management isn't just about storing larger and larger amounts of data but figuring out what data is most relevant for the problems being explored, and then processing this data in a timely manner. Depending on your analytics needs, that processing might involve simple queries or more intensive resources like analytic workloads with more iterations to produce more informative results.

Data management involves handling streaming data, near real-time data, data stored on disk, and data stored in archive systems and being able to integrate all or some of these together in the best formats for analytics and reporting.

It's important to incorporate predictive analytics in some or all of the data gathering or data reporting points.

For example, event stream processing is part of most modern data management strategies because it  allows you to apply if-then type rules to data in stream. It also applies analytics to your sensor data in stream, and take actions with that data as a result of the activity in the data.

Data quality capabilities should also be applied to clean up messy or missing data so decision makers receive quality information that helps them make better decisions in running your organization.

For more details on these topics, please read our whitepaper, Analytic Innovations Address New Challenges in the Oil and Gas Industry.


About Author

David Pope

Technical Leader, Senior Manager US Energy

David leads the pre-sales technical team for SAS US Energy which solves business problems in the Oil & Gas and Utilities industries using advanced analytics. He is a lifetime learner who enjoys sharing information and helping others to grow their careers. He earned a BS in Industry Engineering and a Computer Programming Certificate from North Carolina State University. Furthermore, he has over 29 years of business experience working with SAS across R&D, IT, Sales and Marketing in the Americas and Europe. He is an expert in working with data and producing insights through the use of analytics. David has presented at SAS Global Forum, the 2012 SAS Government Leadership Summit, IBM’s Information on Demand(IOD), EMC World, CTO Summit Conferences, is the author of the book: "Big Data Analytics with SAS", and he currently holds 14 patents for SAS in several countries: US, CA, Norway, UK, China, Mexico, and Hong Kong.

1 Comment

Leave A Reply

Back to Top