SHOULD YOU DERIVE ? OR LET THE DATA DRIVE ?TOWARDS A FIRST-PRINCIPLES DATA-DRIVEN SYMBIOSIS

Lior Horesh - IBM TJ Watson Research Center

Joint work with Ted van Kessel, Andrew Conn, Matthias Kormaksson, Omolade Saliu

Optimization and Uncertainty Quantification in Energy and Industrial Applications - IMA, MN, Feb 2016

Injection of steam in a process called Steam Assisted Gravity Drainage (SAGD) is required to lean oil viscosityand allow its extraction

Oil prices rapidly decline worldwide operations optimization is critical for survival of non-conventionalnatural resources companies

Determination of optimal controls requires comprehension of the underlying state of the system as well as realization of its response to the controls

Modeling the causal response of the reservoir to various controls is a complex multi-scale, multi-physics problem

BACKGROUND

PROBLEM DEFINITION

Control Interface Set Control Variables• Steam Injection • Gas Pressure • Emulsion Pressure• Etc.

SAGD on-site equipment injects steam under set control configuration. Production continues until another action is taken

Sensor data captured and stored in historical database in real time:

• Control Variables: Steam Injection, Gas Pressure, Emulsion Pressure, etc.

• Observable Variables: Temperatures along Producer and Injector wells, Steam Injection Surface Pressure, Inj. Blanket Gas Pressure, etc.

• Output Variables: Emulsion, Oil Production, etc.

Predictive Modeling:(using historical data for training)

Input: Control variables (Steam)

Output: Predicted Oil, Emulsion, etc.

Observables (from sensors):Predicted Temperatures, etc.

Optimization:(using predictive model)

DecreaseSteam flow

IncreaseGas Pressure

Maintain sameEmulsion Pressure

Verify constraints on observables:

Find optimal configuration that maximizes oil, emulsion, etc.

Optimum solutionhonors constraints

1. Visualize model & consider recommended optimal configuration2. Evaluate model performance

Analytical Well Model

Physical models are the state-of-the-art

Geology - each well pair is of specific characteristics, immersed in different geology and bitumen environment

Requires prescription of model parameterseverywhere at all times (great uncertainty)

Ad-hoc, nuisance values are being prescribed leading to biased predictions

Misspecification - the underlying multi-physics phenomena is not fully realized

PRIOR ART

Data driven approaches so far offered poor results

Require notable ramp-up time to accumulate sufficient amount of data to train upon

Overly specific for a given well instance, may notgeneralize to other wells

Non-linearity - system response is non-linear

Causality - system reacts differently to the sameinput at different time

BROADER PERSPECTIVE

Thinking outside the Box…

Frequently, simulation models are mis-specified

Fidelity of the simulation process is critical in attainment of meaningful capabilities

Prominent modeling errors creep into the simulation output and dependencies

Inaccurate state descriptions

Unstable model inferences

Erroneous control output, designs or decisions

A MODEL IS A MODEL…

“All Models are Wrong… But Some are Useful” [George E. P. Box 1987]But Some Can be Made Less Wrong / More Useful…”“All Models are Wrong…

FIRST PRINCIPLES VS. DATA DRIVEN MODELING

First Principles Data Driven

Data reliance Small Data Big Data to train upon

Domain reliance

High reliance upon deep domain expertise

Can provide useful results with littledomain knowledge

Fidelity &robustness

Universal links can handle highly non-linear and complex relations

Limited to the range of values spanned by the training set and model complexity

Adaptability &Deployability

Requires complex and time consuming derivation to account for new relations

Rapidly adapt to the circumstances of a specific problem instance

Interpretability Consistent, physically meaningful link between parameters

Physics agnostics surrogate, limited by the rigidity of the model functional form

SHOULD YOU DERIVE,

OR LET THE DATA DRIVE ?

At some level they are very similar, simply a different functional form is used for modeling

Great synergy in combining the strengths of both

SHOULD YOU DERIVE, OR LET THE DATA DRIVE ?

AA

AA

Proxy - one used as a proxy / surrogate for the other

Complement - the solution can be a combination of the two

Supplement - one can provide correction for another

Embedment - one model can be embedded in another

Integrate - output of one can serve as input of another

Inspiration - structure of one model can be drawn from another

HOW PHYSICAL AND DATA-DRIVEN MODELS CAN COOPERATE ?

B

B

AB

A

A

BB

BB

A

THE PARTICULAR APPROACH DEPENDS UPON THE EXISTING STATE OF SIMULATION AND/OR PROXY MODEL IMPLEMENTATION

Physical/Knowledge Base

Vo

lum

e o

f R

elev

ant

His

tori

cal

Dat

a

Simulation/Proxy Model

(Physical model or model of physical

model)

Knowledge-basedapproach + possible

early Simulation

Machine Learning

(Data-driven model)

Hybrid ModelSimulation/proxy + Machine Learning

(Physical model augmented by Data-driven model)

• If substantial amount of “training” data exist, can quickly use ML techniques to improve start-up performance

• If little data, can start with knowledge / simulationapproaches, and blend ML in as data generated, to increase slope of improvement curve

• “Living Model Management” system can monitordeviations to continually improve models and enable continuous performance improvement

BACK TO SAGD …Cap Rock (shale & glacial t ill) 250m thickSteam

Chambers

UnrecoveredHeavy Oil

6mo6mo

2yr2yr5yr5yr

8yr8yr

10yr10yr

~ 1 kilometer~ 200m

40m

Cap Rock (shale & glacial t ill) 250m thickSteamChambers

UnrecoveredHeavy Oil

6mo6mo

2yr2yr5yr5yr

8yr8yr

10yr10yr

~ 1 kilometer~ 200m

40m

~160 m

• The state of the system evolves over time, and depends upon the cumulative energyand mass inserted and exerted to the system

• In addition to raw controls and observables, the following conservative physicalentitles were approximated and fed a machine learning predictive model:

• Integrated total mass inserted - integrated steam inserted to the system

• Integrated total energy inserted - product of the inserted steam multiplied by computed inwards enthalpy

• Mass differential - emulsion flow subtracted from the total injected steam

• Energy differential - integrated approximated energy differential over life of i.e. the product of the outwards enthalpy by the extracted emulsion subtractedfrom a product of the inserted mass by the inward enthalpy

total inserted energy

total inserted mass

daily inserted energy

daily inserted mass

total steam

daily steam

HYBRID PHYSICS-INSPIRED DATA DRIVEN MODELING

• To accommodate the causal nature of the underlying physical system, the model observables outputare fed back to the model as input in an auto-regressive fashion

• Input entities (exogenous controls, observables feedback, other parameters of relevance) are introduced to the model with a set of delays

• Delays specify how far back previous values are of relevance to the system

• Due to heterogeneous sources of information and requirement for heterogeneous set of outputparameters multiple outputs and inputs

CAUSAL PYSICS-INSPIRED NON-LINEAR AUTO-REGRESSIVE MODEL

EmulsionBlanket Gas PressureTemp Zone1 VariationTemp Zone2 VariationTemp Zone1 MinTemp Zone2 MinSub Cool Heel Steam PressureToe Steam Pressure

total inserted energy

total inserted mass

daily inserted energy

daily inserted mass

total steam

daily steam

heel steam ratetoe steam rategas casing

pressureemulsion pressure

Illustration of Predictive Horizon for Emulsion Two Different Wells

predictions

PHYSICS-INSPIRED DATA-DRIVEN MODELS -VALIDATION AND FUTURE PREDICTIONS

Emulsion Flow Well 1 Emulsion Flow Well 2

PHYSICS-INSPIRED DATA-DRIVEN MODELS -CONSTRAINTS OUTPUT

Average BHP

Toe Surface Pressure

Average SubCool

Heel Surface Pressure

PHYSICS-INSPIRED DATA-DRIVEN MODELS -CONSTRAINTS OUTPUT

Heel Min Temp

Toe Min Temp

Heel Temp Range

Toe Temp Range

[m3 ]

SIMULATION EMBEDDED OPTIMIZATION OBJECTIVE

SIMULATION EMBEDDED OPTIMIZATION CONTROLS

[m3 ]

[m3 ]

SIMULATION EMBEDDED OPTIMIZATION EFFICENCY RESULTS

Physics-inspired machine-learning modeling offers unique advantages for real-life problems

The symbiosis of the two approaches yield predictions that

Adapt to the specific circumstances of each instance (well pair geology and state)

Generalizes well while accounting for the causal nature of the system

Hybrid model was made differentiable for consequent symbolic differentiation

Model was embedded within a non-linear optimization framework with non-linear constraints successfully projected improvement of ~10% recovery

Business significance: 1% improvement per year materializes to order of $M field scale savings, across lifecycle may be reflected in the order of $B

SUMMARY

REFERENCES

R. Lam, L. Horesh, H. Avron, K. Willcox, Should you Derive, or let the data Drive ? Stochastic optimization framework for design of hybrid first-principles phenomenological data-driven models, Cooper Mountain Conference on Iterative Methods, 2016

N. Hao, L. Horesh, M. Kilmer, Nuclear norm optimization and its application to observation model recovery, Compressed Sensing and Sparse Filtering, Springer, 2014

N. Hao, L. Horesh, M.E. Kilmer, Model Correction using a Nuclear Norm Constraint, The Householder Symposium XIX on Numerical Linear Algebra, Domain Sol Cress, Spa, Belgium, 2014

Disclosure YOR8 2015 2102 - Cross well allocation optimization in Steam Assisted Gravity Drainage (SAGD) wells

Disclosure YOR8 2015 2142 - Two-Level Modeling of Steam Assisted Gravity Drainage (SAGD) Wells

Disclosure YOR8 2015 2151 - Method of SAGD Well Modeling Using Physical Model Blending

Disclosure YOR8 2015 2155 - Method for Extended Horizon SAGD Controls Optimization

Disclosure YOR8 2015 2156 - System and Method for Causal Modeling of SAGD Based on Auto Regressive Non-Linear Neural Network

Disclosure YOR8 2015 2157 - Method for Optimal SAGD Operation Control and Sensitivity Analysis by Symbolic Differentiation of Implicit Predictive Models

Disclosure YOR8 2015 2158 - Method for Incorporation of integer variables in derivative free optimization of SAGD controls

Disclosure YOR8 2015 2159 - System and Method for Adaptive Hierarchical Meta-Level Modelling of SAGD Processes

Disclosure YOR8 2015 2160 - System and Method for Balanced Exploration and Exploitation in SAGD Controls Optimization

THANKS