Digital Oilfield: ten years of progress and pain

We have been building data management software for oil and gas for over 25 years. In that time, we have watched a lot of well-funded digital initiatives succeed brilliantly in a pilot and quietly disappear afterwards. This paper asks why. 

The conclusions shaped how we built EnergySys. They still do.

Executive summary

The Digital Oilfield initiative promised better automation, integration, and efficiency across oil and gas. After more than a decade of investment, the results are mixed. Some companies have achieved real gains. Most have not moved beyond a pilot or two.

This paper reviews the state of digital change in oil and gas. It draws on research and case studies to look at what worked, what did not, and what the evidence tells us about where value actually comes from.

Introduction

A lesson we keep having to learn…

In the early 2000s, we were working with a large energy services company. They wanted to reduce the friction between teams. Facilities engineers were working from a defined data set provided by reservoir engineers, but it kept leaving out things that later turned out to be vital. The assumption was that better tools would fix it.

We spent a good while looking at ways to link data together. An information bus, shared stores, and discipline-specific tools. The design was sound. The intent was right.

The change that actually made a difference was a room. A physical meeting space where people from different teams could sit down and talk. Not a platform. A room.

It sounds almost too simple. But the research confirms it. Team change, good working practices, and coaching matter at least as much as any technology spend. People achieve great things with basic tools when the environment and motivation are right. The reverse is also true: no amount of technology compensates for teams that are not set up to use it well.

This lesson has been learned and re-learned many times over. The frustrating part is how rarely it shapes what companies choose to buy.

Despite the real potential of digital tools, uptake has been slower than the scale of investment would suggest. McKinsey research from 2024 finds that 70 per cent of oil and gas companies have not moved beyond the pilot phase. Average digital maturity across the industry stands at 2.3 out of 5.0, with the best-performing companies reaching above 4.0.

A KPMG survey of energy sector leaders found that a lack of skills is seen as the most likely barrier to progress. The same research noted that energy companies are more likely than any other sector to say that a lack of AI expertise is holding back their ability to innovate. A 2024 KPMG Energy CEO Outlook found that 52 per cent of energy companies planned to invest more in their people, up seven percentage points from 2022.

The scale of spend has not translated into wide results. Much of the focus has been on technical capability rather than clear business goals. Where progress has been made, it has often stayed inside a single company rather than being shared more broadly.

What is a Digital Oilfield?

The term was never well defined. That was part of the problem.

Depending on who was talking, Digital Oilfield referred to sensor networks, data links, workflow tools, predictive analytics, remote working, or some mix of all of them. A large share of the published work was vendor-generated, which meant definitions tended to reflect whatever the author was selling.

Across more neutral sources, six themes came up again and again:

These are not odd goals. Most data-heavy, regulated operations would recognise all six. The problem was that Digital Oilfield became an umbrella term so broad that it stopped meaning anything in practice. Without a shared definition, it was hard to set clear goals, measure results, or share what worked.

The drivers behind the Digital Oilfield ambition were real.

Operations were getting more complex and more costly. Experienced staff were leaving faster than they could be replaced. The people with the deepest knowledge of assets and processes were heading for the door, and years of hard-won experience were going with them.

Digital tools were supposed to help. Automate the routine tasks. Capture what experts know. Make it easier for less experienced staff to work well. That ambition remains valid today.

The challenge was that the business case was often hard to pin down. Return on investment was uncertain. Benefits played out over years rather than quarters. Pilot projects worked well in controlled settings but struggled when rolled out across a portfolio. The companies best placed to invest and report on progress were the largest ones, so the published evidence skewed heavily toward major operators. Smaller companies, which arguably had most to gain from shared tools and cloud delivery, were largely absent from the conversation.

Culture and resistance to change were the most consistently cited barriers across the research. Not technical limits. Not cost. Culture.

Change management was seen as important, but investment in it was thin. Dropping new tools into a team that has not been prepared for the change rarely goes well. That is as true today as it was ten years ago.

A few other patterns came up across the evidence.

Pilot success did not predict wider success. Individual projects worked well. Rolling them out across a multi-asset company exposed gaps in governance, data quality, and alignment that the pilot had never addressed. McKinsey documented cases where companies ran dozens or even hundreds of digital projects without the ability to track their impact or decide which ones to scale.

Projects moved at the wrong speed. Either too slow for organisational patience, or too fast for genuine change to take root. The mismatch was rarely flagged at the start.

Proprietary approaches created silos. Large operators built their own solutions. These could not easily be shared or adapted by others. Progress that could have been industry-wide stayed locked inside individual companies.

Without agreed benchmarks, results were hard to verify. Gains were reported, but linking them to digital projects, rather than other changes happening at the same time, was genuinely difficult.

Despite the mixed picture, some areas showed real results.

Analytics and decision support

The clearest example of analytics delivering value in this period came from Devon Energy, a mid-size US operator. They built a dedicated analytics team, worked with external partners, and were clear about scope. Their own definition of analytics was deliberately plain: the discovery and communication of meaningful patterns in data.

What made it work was the discipline around problem framing. Before any analysis, the team identified the exact question they were trying to answer. Data quality issues were treated as a real problem to fix, not a background constraint. Subject matter experts were central to the work throughout.

That approach holds up today. Analytics tools have become far more powerful, but the limiting factor is still rarely the technology. McKinsey research shows that upstream companies using advanced analytics have captured more than five dollars of added value per barrel of oil equivalent. But realising that value depends on asking the right questions and having domain experts involved in reading the answers.

Asset monitoring

Remote monitoring of equipment was the most clear-cut area of progress, largely because the return on investment was easy to measure and quick to demonstrate. Knowing when a piece of kit is likely to fail, before it fails, has an obvious value. This part of the Digital Oilfield agenda became standard practice faster than most.

Cloud computing

Cloud was identified early as the right direction for infrastructure. The case against it, mainly security and data volume concerns, was always more cautious than the evidence required. Both were manageable.

The more honest problem was that many companies moved to cloud in name only. Existing systems were shifted onto hosted servers without meaningful change to how they were built or used. The result was the same old approach as before, now accessed through a remote screen. Most of the real benefits of cloud delivery were left on the table.

The companies that made cloud work were those that rethought their approach to software at the same time as changing their infrastructure.

Integrated asset modelling

This area generated a lot of debate about methods and rather less progress on results. The core tension was between model accuracy and speed. More detailed models took longer to run. Simpler models introduced uncertainty. There is no single right answer. The best approach depends on the asset, the question, and the level of risk in the decision.

That is a fair conclusion. It is also worth being honest that it reflects the limits of what standard tools could offer.

Two areas attracted a lot of attention and produced much less than promised.

Big data became a technology in search of a use case. The demands on infrastructure were heavy. The definitions were slippery. The pattern from the most grounded work was clear: tools should answer a meaningful question. Without a clear question, large volumes of data produce large volumes of noise.

The Internet of Things has since found its place in many operations. Low-cost sensors, connected infrastructure, and remote monitoring are now common. But the early hype was well ahead of practical use, and the value has come from specific, well-scoped applications rather than sweeping change.

Innovation in complex industries

Digital Oilfield was never a single initiative with a clear owner. It was a loose collection of technologies and ambitions, pursued by hundreds of companies at once, with no shared design, no agreed measures, and no central direction.

That structure makes innovation genuinely hard. Research on innovation networks places the cross-industry Digital Oilfield effort in the most complex category: a wide range of participants with different and sometimes conflicting goals, highly distinct knowledge bases, and no single body in charge of where things go. The most consistent success came from single companies with clear leadership and the power to drive change across teams.

A broader pattern runs through the evidence. Oil and gas have historically been cautious about adopting new things, which is often right for operations where failure carries serious consequences. But caution becomes a problem when it stops companies from learning from each other. Lessons learned in one company get relearned from scratch in the next, at a cost.

A KPMG survey of energy sector leaders found that senior executives named organisational inertia as the main barrier to digital change, ahead of both budget and technical limits. Field engineers and operational teams often distrust algorithm-based outputs, preferring familiar ways of working. Both responses are understandable. Neither is inevitable.

Where things stand now

Digital Oilfield, as a distinct concept, has largely faded. What replaced it was a quieter recognition that digital tools work best when built around the people who run operations, not delivered to them as a finished system.

The tools have changed a great deal. Cloud platforms have matured. AI and machine learning have moved from trials to day-to-day use in many settings. Sensors and monitoring kits are cheaper and more reliable. Data that was once hard to collect is now often hard to manage.

But the underlying challenges have not shifted as much as the technology has.

Culture still matters more than capability. Companies that buy tools without investing in the people and processes around them still see limited returns. The skills gap that drove much of the original Digital Oilfield ambition has not been resolved. KPMG’s 2023 global tech report found that energy companies are more likely than any other sector to say that AI skills gaps are holding back their ability to compete. If anything, the gap between what experienced experts know and what systems can reliably do has grown as the technology has become more complex.

The most important thing the Digital Oilfield era showed us was not which technologies worked. It was that the value of any tool depends almost entirely on how it is deployed, by whom, and with what knowledge of the underlying operation.

That is not a reason to be cautious about new tools. It is a reason to be clear about what problem you are actually trying to solve before you commit to solving it.

Key takeaways

Tools follow people, not the other way around. The greatest gains came from better collaboration and clearer problem framing, not more sophisticated software.

Pilots are not programmes. Success in a controlled setting does not predict success at scale. Scaling needs investment in people, process, and governance alongside the technology.

Clarity of purpose drives return on investment. Companies that defined clear goals before deploying tools consistently did better than those that deployed tools and hoped results would follow.

Cloud delivers most value when you rethink, not just rehome. Moving systems to the cloud without changing how they are built and used captures very little of the available benefit.

Domain expertise is not optional. Analytics, AI, and automation all depend on subject matter experts to frame the right questions, check outputs, and take ownership of decisions. Tools that reduce the need for expertise tend to reduce the quality of results.

Sharing matters. The industry’s habit of keeping progress proprietary has slowed adoption for everyone. The companies that will benefit most from the next wave of digital tools are those willing to learn from outside their own walls.

These conclusions are not comfortable reading for a software company. But they are why EnergySys is built the way it is. We did not set out to replace what your experts know. We set out to give them a platform they can actually control: one that works with how they already think, scales when they need it to, and does not require a specialist team to run. 

The research points to a gap between domain expertise and the tools built to support it. That gap is what we are here to close.