Upload
nguyencong
View
228
Download
3
Embed Size (px)
Citation preview
Effective Solvers for Reservoir Simulation
Xiaozhe Hu
The Pennsylvania State University
Numerical Analysis of Multiscale Problems & Stochastic Modelling,
RICAM, Linz, Dec. 12-16, 2011
X. Hu (Penn State) Effective Solvers for Reservoir Simulation Linz,Dec.14, 2011 1 / 47
Collaboration with:I ExxonMobil Upstream Research CompanyI China National Offshore Oil CooperationI Monix Energy Solutions
Main Collaborators:I James Brannick (PSU), Yao Chen (PSU), Chunsheng Feng (XTU),
Panayot Vassilevski (LLNL), Jinchao Xu (PSU),Chensong Zhang (CAS), Shiquan Zhang (Fraunhofer ITWM),and Ludmil Zikatanov (PSU)
X. Hu (Penn State) Effective Solvers for Reservoir Simulation Linz,Dec.14, 2011 2 / 47
Outline
Outline
1 Reservoir Simulation
2 Solvers for Reservoir Simulation
3 Applications & Numerical Tests
4 Ongoing Work and Conclusions
X. Hu (Penn State) Effective Solvers for Reservoir Simulation Linz,Dec.14, 2011 3 / 47
Reservoir Simulation
1 Reservoir Simulation
2 Solvers for Reservoir Simulation
3 Applications & Numerical Tests
4 Ongoing Work and Conclusions
X. Hu (Penn State) Effective Solvers for Reservoir Simulation Linz,Dec.14, 2011 4 / 47
Reservoir Simulation
World without Oil?
List of Oil Products:Antiseptics, Aspirin, AutoParts, Ballpoint pens,Candles, Cosmetics, Crayons,Eye Glasses, Fishing Line,Food Packaging, Glue, HandLotion, Insect Repellant,Insecticides, Lip Stick,Perfume, Shampoo, ShavingCream, Shoes, Toothpaste,Trash Bags, VitaminCapsules, Water Pipes, · · · .
Global oil discovery peaked in the late
1960s. Now we consume much more
than the amount we discover and this
gap is still growing!
(http://www.aspo-ireland.org/)
X. Hu (Penn State) Effective Solvers for Reservoir Simulation Linz,Dec.14, 2011 5 / 47
Reservoir Simulation
Oil Recovery
CSEF 2011 Kunming, China C.-S. Zhang
!"#$%&!'()"*(&+'$,
Antiseptics, Aspirin, Auto Parts, Ballpoint pens, Candles, Cosmetics, Crayons, Eye Glasses, Fishing Line, Food Packaging, Glue, Hand Lotion, Insect Repellant, Insecticides, Lip Stick, Perfume, Shampoo, Shaving Cream, Shoes, Toothpaste, Trash Bags, Vitamin Capsules, Water Pipes, !
List of Oil Products
CSEF 2011 Kunming, China C.-S. Zhang
-.)/.01%&+'$&210"31#4
! Primary Recovery ! Secondary Recovery ! Enhanced Oil Recovery
Primary Recovery
Secondary Recovery
Enhanced Oil Recovery
X. Hu (Penn State) Effective Solvers for Reservoir Simulation Linz,Dec.14, 2011 6 / 47
Reservoir Simulation
Enhanced Oil Recovery
Enhanced Oil Recovery (EOR) isa generic term for techniques forincreasing the oil recovery ratefrom an oil field:
Thermal Method
Microbial Injection
Gas Injection
Chemical FloodingI AlkalineI PolymerI SurfactantI GelI · · ·· · ·
Recovery Rate:
Primary and Secondaryrecovery: 20-40%
Enhanced oil recovery:30-60%
X. Hu (Penn State) Effective Solvers for Reservoir Simulation Linz,Dec.14, 2011 7 / 47
Reservoir Simulation
Black Oil Model
Mass Conservation:
∂
∂t
[φ
(So
Bo+
Rv Sg
Bg
)]= −∇ ·
(1
Bouo +
Rv
Bgug
)+ qO
∂
∂t
[φ
(Sg
Bg+
RsSo
Bo
)]= −∇ ·
(1
Bgug +
Rs
Bouo
)+ qG
∂
∂t
(φ
Sw
Bw
)= −∇ ·
(1
Bwuw
)+ qW
Darcy’s Law:
uα = −kkrα
µα(∇Pα − ραg∇z), α = o, g ,w
Constitutive Laws:
So + Sg + Sw = 1, Pcow = Po − Pw , Pcog = Pg − Po ,
X. Hu (Penn State) Effective Solvers for Reservoir Simulation Linz,Dec.14, 2011 8 / 47
Reservoir Simulation
Modified Black Oil Model
Add polymer and sodium chloride:
∂
∂t
(φS∗
w CP
Br Bw
)+
∂
∂t(ρr (1− φ)Ca) = −∇
(CP
BwuP
)+ qW CP
∂
∂t
(φSw CN
Br Bw
)= −∇
(CN
BwuN
)+ qW CN
Effects on Viscosity:
uw = − kkrw
µw ,ef f Rk(∇Pw − ρwg∇z)
Add more components (brine, gel, surfactant, · · · )
X. Hu (Penn State) Effective Solvers for Reservoir Simulation Linz,Dec.14, 2011 9 / 47
Reservoir Simulation
Numerical Method
Discretization in time:I Fully Implicit Method (FIM)
(Douglas, Peaceman & Rachford 1959)I Implicit Pressure Explicit Saturation (IMPES)
(Sheldon, Zondek & Cardwell 1959; Stone & Garder 1961)I Sequential Solution Method (SSM)
(MacDonald & Coats 1970)I Adaptive Implicit Method (AIM)
(Thomas & Thurnau 1983)
Discretization in space: finite difference, finite volume, finiteelement, etc.
Linearization: Newton-Raphson Method.
Grid: Structured grids are still used by the main-stream; butunstructured grids are catching up.
X. Hu (Penn State) Effective Solvers for Reservoir Simulation Linz,Dec.14, 2011 10 / 47
Reservoir Simulation
Fully Implicit Method
1
∆t
[φ
(Sg
Bg+
RsSo
Bo
)]n+1
−[φ
(Sg
Bg+
RsSo
Bo
)]n
=
∇ · (T n+1g ∇Φn+1
α + Rn+1s T n+1
o ∇Φn+1o )
1
∆t
[(φSw
Bw
)n+1
−(φSw
Bw
)n]
= ∇ · (T n+1w ∇Φn+1
w )
1
∆t
[(φSo
Bo
)n+1
−(φSo
Bo
)n]
= ∇ · (T n+1o ∇Φn+1
o )
Potential:Φα := Pα − ραgz , α = w , o, g ,
Transmissibility:
Tα :=krαk
µαBα, α = w , o, g .
X. Hu (Penn State) Effective Solvers for Reservoir Simulation Linz,Dec.14, 2011 11 / 47
Reservoir Simulation
Newton-Raphson Method
In each Newton Iteration, we need to solve the following Jacobian system:
JgP JgSw JgSo
JwP JwSw JwSo
JoP JoSw JoSo
δPδSw
δSo
=
Rg
Rw
Ro
Different blocks have different properties, for example
JgP =1
∆tcgP −∇ · DgP∇− CgP · ∇ −RgP ,
usually the diffusion term in JgP is dominating, which makes JgP like an ellipticproblem. However,
JoSo =1
∆tcoSo − CoSo · ∇ −RoSo ,
here convection term is dominating, which makes JoSo like a transport problem.
X. Hu (Penn State) Effective Solvers for Reservoir Simulation Linz,Dec.14, 2011 12 / 47
Reservoir Simulation
Including Wells
Peaceman Model
One-dimentional radial flow
Single-phase flow in thewellbore
qi,α = −NperfX
j
2πhperf Kkrα
ln( rerw +s
)µα(Pj−Pbhp−Hwj)
Jacobian system with wells:
[JRR JRW
JWR JWW
] [δRδW
]=
[RR
RW
]
CSEF 2011 Kunming, China C.-S. Zhang
!"#$%&#'()*+,'-$)'!"#&)
∂
∂t
φS∗
wCP
Bw
+
∂
∂t(ρr(1− φ)Ca) = −∇
CP
BwuP
+ qW CP
uw = − kkrw
µw,effRk(∇P − ρwg∇z)
Add polymer(s) to the drive
Effects on viscosity
Add more components (polymer, brine, gel, surfactant, etc)
CSEF 2011 Kunming, China C.-S. Zhang
.&))'!"#&)/
• One-dimensional radial flow into the wellbore; • Single-phase flow; • Pseudo steady-state pressure behavior.
Peaceman Model
o Multi-segment o Multi-lateral o Complex wells o Couple wells with reservoir
X. Hu (Penn State) Effective Solvers for Reservoir Simulation Linz,Dec.14, 2011 13 / 47
Reservoir Simulation
Numerical ChallengesStrongly coupled and complicated PDE system
High nonlinearity
Complicated geometry: irregular domain with faults, pinch-out and erosion
Heterogeneous porous media; large permeability jumps
Complex wells
Unstructured grids: corner point grid, anisotropic mesh
Highly non-symmetric and indefinite large-scale Jacobian system
Different properties of the physical quantities (the equation that describesthe pressure is mainly elliptic, the equation that describes saturation ismainly hyperbolic)
Commercial Simulators set a high bar: ECL2009 (Schlumberger), STARS(CMG), VIP (Halliburton), · · ·
Goal: develop fast solvers, deliver a solver package
X. Hu (Penn State) Effective Solvers for Reservoir Simulation Linz,Dec.14, 2011 14 / 47
Solvers for Reservoir Simulation
1 Reservoir Simulation
2 Solvers for Reservoir Simulation
3 Applications & Numerical Tests
4 Ongoing Work and Conclusions
X. Hu (Penn State) Effective Solvers for Reservoir Simulation Linz,Dec.14, 2011 15 / 47
Solvers for Reservoir Simulation
Solvers
“For a reservoir simulation with a number of gridblocks of order 100,000,about 80%− 90% of the total simulation time is spent on the solution oflinear systems with the Jacobian.”—- Chen, Huan, & Ma, Computational Methods for Multiphase Flows in Porous Media, 2005
Direct solvers: (Price & Coats 1974)
Incomplete factorization preconditioner: (Watts 1981, Behie & Vinsome 1982; Meyerink 1983;
Appleyard & Cheshire 1983)
Constrained Pressure Residual (CPR) preconditioner: (Wallis 1983; Wallis,Kendall, Little &
Nolen 1985; Aksoylu & Klie 2009)
Algebraic Multigrid (AMG) Method :
I Used in CPR preconditioner (pressure equation): (Klie 1997; Lacroix, Vassilevski &
Wheeler 2001, 2003; Scheichl, Masson & Wendebourg 2003; Cao, Tchelepi, Wallis & Yardumian 2005;
Hammersley & Ponting 2008; Dubois, Mishev & Zikatanov 2009; Al-Shaalan, Klie, Dogru & Wheeler 2009;
Jiang & Tchelepi 2009)
I For whole Jacobian system: (Papadopoulos & Tchelepi 2003; Stuben 2004; Clees & Ganzer
2007)
X. Hu (Penn State) Effective Solvers for Reservoir Simulation Linz,Dec.14, 2011 16 / 47
Solvers for Reservoir Simulation
Solvers
“For a reservoir simulation with a number of gridblocks of order 100,000,about 80%− 90% of the total simulation time is spent on the solution oflinear systems with the Jacobian.”—- Chen, Huan, & Ma, Computational Methods for Multiphase Flows in Porous Media, 2005
Direct solvers: (Price & Coats 1974)
Incomplete factorization preconditioner: (Watts 1981, Behie & Vinsome 1982; Meyerink 1983;
Appleyard & Cheshire 1983)
Constrained Pressure Residual (CPR) preconditioner: (Wallis 1983; Wallis,Kendall, Little &
Nolen 1985; Aksoylu & Klie 2009)
Algebraic Multigrid (AMG) Method :
I Used in CPR preconditioner (pressure equation): (Klie 1997; Lacroix, Vassilevski &
Wheeler 2001, 2003; Scheichl, Masson & Wendebourg 2003; Cao, Tchelepi, Wallis & Yardumian 2005;
Hammersley & Ponting 2008; Dubois, Mishev & Zikatanov 2009; Al-Shaalan, Klie, Dogru & Wheeler 2009;
Jiang & Tchelepi 2009)
I For whole Jacobian system: (Papadopoulos & Tchelepi 2003; Stuben 2004; Clees & Ganzer
2007)
X. Hu (Penn State) Effective Solvers for Reservoir Simulation Linz,Dec.14, 2011 16 / 47
Solvers for Reservoir Simulation
Auxiliary Space Preconditioning
1 construct preconditioner via auxiliary (simpler) problems
2 solve the auxiliary problems by efficient solvers (such as multigrid)
3 apply preconditioned Krylov subspace methods
Examples:
Fictitious Domain Methods (Nepomnyaschikh 1992)
Method of Subspace Correction (Xu 1992)
Auxiliary Space Method (Xu 1996)
Nodal Auxiliary Space Preconditioning in H(curl) and H(div) spaces(Hiptmair & Xu 2007)
· · · · · ·
X. Hu (Penn State) Effective Solvers for Reservoir Simulation Linz,Dec.14, 2011 17 / 47
Solvers for Reservoir Simulation
Auxiliary Space Method (Xu 1996)
Abstract problem: A : V → V is SPD
Au = f
Auxiliary spaces:V = V ×W1 × · · · ×WJ
Additive preconditioner:
B = S +J∑
j=1
Πj A−1j Π∗
j
With Πj : Wj 7→ V (transfer) and S : V 7→ V (smoother). It can beshown that under some assumptions
κ(BA) ≤ C
X. Hu (Penn State) Effective Solvers for Reservoir Simulation Linz,Dec.14, 2011 18 / 47
Solvers for Reservoir Simulation
Auxiliary Space Method for Reservoir SimulationFast Auxiliary Space Preconditioning
Auxiliary Spaces:
V = V ×WP ×WS ×Wwell
FASP Algorithm: Given u0, Bu0 := u4 where
u1 = u0 + ΠwellA−1wellΠ
∗well(f − Au0)
u2 = u1 + ΠSA−1S Π∗
S(f − Au1)
u3 = u2 + ΠPA−1P Π∗
P(f − Au2)
u4 = u3 + S(f − Au3)
Auxiliary problems Awell , AS and AP are solved approximately.
X. Hu (Penn State) Effective Solvers for Reservoir Simulation Linz,Dec.14, 2011 19 / 47
Applications & Numerical Tests
1 Reservoir Simulation
2 Solvers for Reservoir Simulation
3 Applications & Numerical Tests
4 Ongoing Work and Conclusions
X. Hu (Penn State) Effective Solvers for Reservoir Simulation Linz,Dec.14, 2011 20 / 47
Applications & Numerical Tests Single Phase Flow
Single Phase Flow (pressure equation)Intergrated Reservoir Performance PredictionExxonMobil Upstream Research Company
Possion-like Model Problem (After linearization):
−∇ · (a∇p) + cp = f , in Ω
p = gD , on ∂ΩD ,
(a∇p) · nnn = gN on ∂ΩN ,
Discretization: Hybrid-Mixed FEM (Kuznetsov & Repin 2003)
Linear system: Symmetric and Positive definite
X. Hu (Penn State) Effective Solvers for Reservoir Simulation Linz,Dec.14, 2011 21 / 47
Applications & Numerical Tests Single Phase Flow
Difficulties for the solver
Models are from real problems,the geometry is complicated
Mesh (hexahedral grid): Highlyunstructured; Degenerated,nonmatching, and distortedgridblock.
Permeability: Large jumps: varyform 10−4 to 104; Anisotropic.
Only two types of pinchout hexahedrons satisfy these assumptions. Dueto the terminology in [6], we call them the “Element H1” (the vertical edgeconnecting node 0 with node 4 vanishes) and “horizontal triangular prism”(two neighboring vertical edges (0, 4) and (1, 5) disappear), see Figure 4.
5
6
70, 4
1
23
6
70, 4
1, 5
3 2
Figure 4: Element H1 (left) and horizontal triangular prism (right)
The element H1 has seven vertexes, two triangular and four quadrilateralfaces, and 11 edges. The horizontal prism has six vertexes, two triangularand three quadrilateral faces.
13
Also, due to the presence of the so called fault surfaces, nonmatchingmeshes are possible. For instance, consider a case when we have a domaindivided into geological layers which allows us to construct a hexahedral meshfitting to the interfaces between layers. Suddenly, something happens, forinstance, an earthquake or any other natural disaster. In this case, certainhexahedral cells may change their positions, and some of the hexahedral cellsmay have two or more neighbors from the side of the fault. Therefore, weobtain a nonmatching mesh. Several examples of nonmatching hexahedralmeshes were considered in the previous report [2]. In this report, the side of ahexahedral cell which is adjacent to a fault interface was divided into severalparts. The number of this parts is equal to the number of the neighboring cellsfrom the other side of the fault interface. For instance, the green hexahedralcell shown in Figure 3 has three neighbors adjacent to its right side.
Figure 3: An example of a distorted nonmatching hexahedral mesh from theprevious project
11
X. Hu (Penn State) Effective Solvers for Reservoir Simulation Linz,Dec.14, 2011 22 / 47
Applications & Numerical Tests Single Phase Flow
AMG with ILU smoother?
Preconditioner #iter Setup time Solve timeILU(0) 3458 0.16 96.19AMG 362 0.85 40.32
ILU(0)/1+AMG 2255 1.00 305.17ILU(0)/2+AMG - 1.18 -
ILU or AMG alone does not work well
Using ILU as smoother in AMG may not work
Why AMG with ILU smoother does not work in this case?
Consider (B denotes ILU and S denotes AMG)
u ← u + B(f − Au), u ← u + S(f − Au), u ← u + B(f − Au).
This gives:I − BA = (I − BA)(I − SA)(I − BA).
B might not be positive definite, and cannot be applied to PCG
References about ILU smoother: Wittum 1989; Stevenson 1994
X. Hu (Penn State) Effective Solvers for Reservoir Simulation Linz,Dec.14, 2011 23 / 47
Applications & Numerical Tests Single Phase Flow
AMG with ILU smoother?
Preconditioner #iter Setup time Solve timeILU(0) 3458 0.16 96.19AMG 362 0.85 40.32
ILU(0)/1+AMG 2255 1.00 305.17ILU(0)/2+AMG - 1.18 -
ILU or AMG alone does not work well
Using ILU as smoother in AMG may not work
Why AMG with ILU smoother does not work in this case?
Consider (B denotes ILU and S denotes AMG)
u ← u + B(f − Au), u ← u + S(f − Au), u ← u + B(f − Au).
This gives:I − BA = (I − BA)(I − SA)(I − BA).
B might not be positive definite, and cannot be applied to PCG
References about ILU smoother: Wittum 1989; Stevenson 1994
X. Hu (Penn State) Effective Solvers for Reservoir Simulation Linz,Dec.14, 2011 23 / 47
Applications & Numerical Tests Single Phase Flow
AMG with ILU smoother?
Preconditioner #iter Setup time Solve timeILU(0) 3458 0.16 96.19AMG 362 0.85 40.32
ILU(0)/1+AMG 2255 1.00 305.17ILU(0)/2+AMG - 1.18 -
ILU or AMG alone does not work well
Using ILU as smoother in AMG may not work
Why AMG with ILU smoother does not work in this case?
Consider (B denotes ILU and S denotes AMG)
u ← u + B(f − Au), u ← u + S(f − Au), u ← u + B(f − Au).
This gives:I − BA = (I − BA)(I − SA)(I − BA).
B might not be positive definite, and cannot be applied to PCG
References about ILU smoother: Wittum 1989; Stevenson 1994
X. Hu (Penn State) Effective Solvers for Reservoir Simulation Linz,Dec.14, 2011 23 / 47
Applications & Numerical Tests Single Phase Flow
AMG + ILU!Consider
u ← u + S(f − Au), u ← u + B(f − Au), u ← u + S(f − Au).
This gives:I − BA = (I − SA)(I − BA)(I − SA).
Theorem (Falgout, H., Wu, Xu, Zhang, Zhang & Zikatanov 2011)
Assume that S : V → V satisfies ‖(I − SA)x‖A ≤ ‖x‖A, ∀x ∈ V andoperator B : V → V is SPD. Then the operator B is SPD.
Theorem (Falgout, H., Wu, Xu, Zhang, Zhang & Zikatanov 2011)
Assume that ((I − SA)v , v)A ≤ ρ(v , v)A, ρ ∈ [0, 1), B is SPD and thecondition number of BA is κ(BA) = m1/m0. If m1 > 1 ≥ m0 > 0. thenκ(BA) ≤ κ(BA), Furthermore, if ρ ≥ 1− m0
m1−1 , then κ(BA) ≤ κ(SA).
X. Hu (Penn State) Effective Solvers for Reservoir Simulation Linz,Dec.14, 2011 24 / 47
Applications & Numerical Tests Single Phase Flow
AMG + ILU!Consider
u ← u + S(f − Au), u ← u + B(f − Au), u ← u + S(f − Au).
This gives:I − BA = (I − SA)(I − BA)(I − SA).
Theorem (Falgout, H., Wu, Xu, Zhang, Zhang & Zikatanov 2011)
Assume that S : V → V satisfies ‖(I − SA)x‖A ≤ ‖x‖A, ∀x ∈ V andoperator B : V → V is SPD. Then the operator B is SPD.
Theorem (Falgout, H., Wu, Xu, Zhang, Zhang & Zikatanov 2011)
Assume that ((I − SA)v , v)A ≤ ρ(v , v)A, ρ ∈ [0, 1), B is SPD and thecondition number of BA is κ(BA) = m1/m0. If m1 > 1 ≥ m0 > 0. thenκ(BA) ≤ κ(BA), Furthermore, if ρ ≥ 1− m0
m1−1 , then κ(BA) ≤ κ(SA).
X. Hu (Penn State) Effective Solvers for Reservoir Simulation Linz,Dec.14, 2011 24 / 47
Applications & Numerical Tests Single Phase Flow
ExxonMobil Test Problems
Preconditioner #iter Setup time Solve timeILU(0) 3458 0.16 96.19AMG 362 0.85 40.32
ILU(0)/1+AMG 2255 1.00 305.17ILU(0)/2+AMG - 1.18 -
FASP (AMG+ILU) 41 0.99 5.83
Model 1 Model 2 Model 3 Model 4size 388,675 156,036 287,553 312,851nnz 4,312,175 1,620,356 3,055,173 3,461,087
#iter Setup time (s) Solve time (s) Total time (s)FASP(AMG+ILU) + CG
Model 1 16 2.12 4.63 6.75Model 2 41 0.99 5.83 6.82Model 3 47 1.97 12.81 14.78Model 4 11 1.75 2.68 4.43
X. Hu (Penn State) Effective Solvers for Reservoir Simulation Linz,Dec.14, 2011 25 / 47
Applications & Numerical Tests Single Phase Flow
ExxonMobil Test Problems
X. Hu (Penn State) Effective Solvers for Reservoir Simulation Linz,Dec.14, 2011 26 / 47
Applications & Numerical Tests Single Phase Flow
ExxonMobil Test Problems
X. Hu (Penn State) Effective Solvers for Reservoir Simulation Linz,Dec.14, 2011 27 / 47
Applications & Numerical Tests Single Phase Flow
ExxonMobil Large Test Problems
X. Hu (Penn State) Effective Solvers for Reservoir Simulation Linz,Dec.14, 2011 28 / 47
Applications & Numerical Tests Single Phase Flow
ExxonMobil Large Test Problems
X. Hu (Penn State) Effective Solvers for Reservoir Simulation Linz,Dec.14, 2011 29 / 47
Applications & Numerical Tests Enhanced Oil Recovery
(Modified) Black Oil ModelCNOOC and Monix: SOCF Simulator
Decoupling: Alternative Block Factorization (Bank, Chan, Coughran &
Smith 1989)
Auxiliary problems:I WP : pressure (+ bottom hole pressure)I WS : saturation (+ concentrations)I Wwell : wells (+ perforations)
Solvers for each auxiliary problems:I AP : AMG (+ILU)I AS : Block Gauss-Seidel with downwind ordering and crosswind blocks
(Bey & Wittum 1996; Hackbush & Probst 1997; Wang & Xu 1999; Kwok 2007)
I Awell : ILU/Direct solver
Smoother S : Block Gauss-Seidel with downwind ordering andcrosswind blocks.
Krylov iterative method: GMRes.
Ref: H., Liu, Qin, Xu, Yan and Zhang 2011, SPE 148388
X. Hu (Penn State) Effective Solvers for Reservoir Simulation Linz,Dec.14, 2011 30 / 47
Applications & Numerical Tests Enhanced Oil Recovery
Optimality
X. Hu (Penn State) Effective Solvers for Reservoir Simulation Linz,Dec.14, 2011 31 / 47
Applications & Numerical Tests Enhanced Oil Recovery
East Beverly Oil Field Test
Black Oil Model
Unstructured grid
83,592 grid blocks
169 wells
Faults: 994 non-localconnections
40 years of water flooding
X. Hu (Penn State) Effective Solvers for Reservoir Simulation Linz,Dec.14, 2011 32 / 47
Applications & Numerical Tests Enhanced Oil Recovery
East Beverly Oil Field Test
We use sequential version of FASP in this table. OpenMP version of FASPcosts 46 min (8-core, 2.8GHz).
X. Hu (Penn State) Effective Solvers for Reservoir Simulation Linz,Dec.14, 2011 33 / 47
Applications & Numerical Tests Enhanced Oil Recovery
JZ9-3 Oil Field Test
Polymer flooding(5-compoment)
Number of grids: 157x53x57
Lots of inactive grids
Several faults
37 wells
10 years simulation in total
3 years of polymer flooding
Simulator # Newton Iter Total CPU Time CPU Time/NewtonECL2009 1562 81.0 (min) 3.11 (sec)
SOCF(FASP) 1653 40.0 (min) 1.45 (sec)
X. Hu (Penn State) Effective Solvers for Reservoir Simulation Linz,Dec.14, 2011 34 / 47
Applications & Numerical Tests Enhanced Oil Recovery
JZ9-3 Results Comparison: Oil Production Rate
CSEF 2011 Kunming, China C.-S. Zhang
!"#$%&'()*+,)&-./0123).45&678'
!
CSEF 2011 Kunming, China C.-S. Zhang
!"#$%&'()*+,)&-./0123).45&698'
!
X. Hu (Penn State) Effective Solvers for Reservoir Simulation Linz,Dec.14, 2011 35 / 47
Applications & Numerical Tests Enhanced Oil Recovery
JZ9-3 Results Comparison: Gas Production Rate
CSEF 2011 Kunming, China C.-S. Zhang
!"#$%&'()*+,)&-./0123).45&678'
!
CSEF 2011 Kunming, China C.-S. Zhang
!"#$%&'()*+,)&-./0123).45&698'
!
X. Hu (Penn State) Effective Solvers for Reservoir Simulation Linz,Dec.14, 2011 36 / 47
Applications & Numerical Tests Enhanced Oil Recovery
JZ9-3 Results Comparison: Water Cut
CSEF 2011 Kunming, China C.-S. Zhang
!"#$%&'()*+,)&-./0123).45&67-8
!
CSEF 2011 Kunming, China C.-S. Zhang
9:;<$;&=$-./0.4(4,&&8(),
! !SOCF CMG
Field Oil Saturation
X. Hu (Penn State) Effective Solvers for Reservoir Simulation Linz,Dec.14, 2011 37 / 47
Applications & Numerical Tests Enhanced Oil Recovery
ld101-1 7-component Test
Polymer-Gelflooding
5 wells: 1 injectwell, 4 productwells
32 yearssimulation intotal
No wellILU FASP
Size # Iter Time (s) # Iter Time (s)40x40x4 37 2.20 12 0.47
40x40x20 153 19.59 15 2.8640x40x40 296 56.05 13 4.11
5 wellsILU FASP
# Iter Time (s) # Iter Time (s)40x40x4 55 2.70 13 0.50
40x40x20 236 28.21 15 2.9040x40x40 319 65.04 14 5.78
X. Hu (Penn State) Effective Solvers for Reservoir Simulation Linz,Dec.14, 2011 38 / 47
Applications & Numerical Tests Enhanced Oil Recovery
ld101-1 7-component Test
3/20/2011
75
4
149Isoquant contours of water saturation
SOCF CMG
4LD10-1
Polymer flooding Gel profile-control flooding
Isoquant contours of oil saturation150
3/20/2011
75
4
149Isoquant contours of water saturation
SOCF CMG
4LD10-1
Polymer flooding Gel profile-control flooding
Isoquant contours of oil saturation150
3/20/2011
76
4
151
Poly
mer
floo
ding
Gel
pro
file-
cont
rol
flood
ing
Oil profile
LD10-1
4
LD10-1
0 1000 2000 3000 4000 5000 6000 70000
50
100
150
200
250
300
350
400
Fiel
d O
il P
rodu
ctio
n R
ates
(SM
3)
Time(Day)
A=0 A=-3 A=-4
0 1000 2000 3000 4000 5000 6000 70000.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
Time(Day)
Wat
er C
ut
A=0 A=-3 A=-4
Field Oil Production Rate (SM3) Water Cut152
0 2 31 2 3 4 1[1 ( ) exp( )]i
i i i i
spp w i p i p i p sep iA c A c A c c A t
3/20/2011
76
4
151
Poly
mer
floo
ding
Gel
pro
file-
cont
rol
flood
ing
Oil profile
LD10-1
4
LD10-1
0 1000 2000 3000 4000 5000 6000 70000
50
100
150
200
250
300
350
400
Fiel
d O
il P
rodu
ctio
n R
ates
(SM
3)
Time(Day)
A=0 A=-3 A=-4
0 1000 2000 3000 4000 5000 6000 70000.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
Time(Day)
Wat
er C
ut
A=0 A=-3 A=-4
Field Oil Production Rate (SM3) Water Cut152
0 2 31 2 3 4 1[1 ( ) exp( )]i
i i i i
spp w i p i p i p sep iA c A c A c c A t
Oil Saturation (Left: Polymer; Right: Polymer/Gel)
X. Hu (Penn State) Effective Solvers for Reservoir Simulation Linz,Dec.14, 2011 39 / 47
Ongoing Work and Conclusions
1 Reservoir Simulation
2 Solvers for Reservoir Simulation
3 Applications & Numerical Tests
4 Ongoing Work and Conclusions
X. Hu (Penn State) Effective Solvers for Reservoir Simulation Linz,Dec.14, 2011 40 / 47
Ongoing Work and Conclusions Ongoing Work
Move to GPUNVIDIA
Parallel AMG algorithm suits GPU:
Regular sparsity pattern
Low complexity
Low setup cost
Fine-grain parallelism
Take advantage of hierarchical memory structure
Our choice: Unsmoothed Aggregation AMG (UA-AMG)
Controllable sparsity pattern
Low complexity (less fill-in)
Low setup cost (no triple matrices multiplication)
Large potential of parallelism
X. Hu (Penn State) Effective Solvers for Reservoir Simulation Linz,Dec.14, 2011 41 / 47
Ongoing Work and Conclusions Ongoing Work
UA-AMG with Nonlinear AMLI-cycleProblem: UA-AMG with V-cycle usually dose not converge uniformly!
Solution: UA-AMG with Nonlinear AMLI-cycle (Variable AMLI-cycle / K-cycle).
Nonlinear AMLI-cycle: Bk [·] : Vk → Vk .
Assume B1[f ] = A−11 f , and Bk−1[·] has been defined, then for f ∈ Vk
Pre-smoothing u1 = Rk f ;
Coarse grid correction u2 = u1 + Bk−1[Qk−1(f − Aku1)]
Post-smoothing Bk [f ] := u2 + Rtk(f − Aku2).
Nonlinear coarse grid correction: Bk−1[·] : Vk−1 → Vk−1
Defined by n iterations of a Krylov subspace iterative methods usingBk−1[·] as the preconditioner.
Ref: Axelsson & Vassilevski 1994; Kraus 2002; Kraus & Margenov 2005; Notay & Vassilevski
2008.
X. Hu (Penn State) Effective Solvers for Reservoir Simulation Linz,Dec.14, 2011 42 / 47
Ongoing Work and Conclusions Ongoing Work
UA-AMG with Nonlinear AMLI-cycleProblem: UA-AMG with V-cycle usually dose not converge uniformly!Solution: UA-AMG with Nonlinear AMLI-cycle (Variable AMLI-cycle / K-cycle).
Nonlinear AMLI-cycle: Bk [·] : Vk → Vk .
Assume B1[f ] = A−11 f , and Bk−1[·] has been defined, then for f ∈ Vk
Pre-smoothing u1 = Rk f ;
Coarse grid correction u2 = u1 + Bk−1[Qk−1(f − Aku1)]
Post-smoothing Bk [f ] := u2 + Rtk(f − Aku2).
Nonlinear coarse grid correction: Bk−1[·] : Vk−1 → Vk−1
Defined by n iterations of a Krylov subspace iterative methods usingBk−1[·] as the preconditioner.
Ref: Axelsson & Vassilevski 1994; Kraus 2002; Kraus & Margenov 2005; Notay & Vassilevski
2008.
X. Hu (Penn State) Effective Solvers for Reservoir Simulation Linz,Dec.14, 2011 42 / 47
Ongoing Work and Conclusions Ongoing Work
Convergence Analysis of Nonlinear AMLI-cycle (SPD)
Theorem (H., Vassilevski & Xu 2011)
Assume the convergence factor of V-cycle MG with fixed level differencek0 is bounded by δk0 ∈ [0, 1), if n is chosen such that (1− δn)δk0 + δn ≤ δhas a solution δ ∈ [0, 1), which is independent of k, then we have
‖v − Bk [Akv ]‖2Ak≤ δ‖v‖2
Akand ‖v − Bk [Akv ]‖2
Ak≤ δn‖v‖2
Ak,
A sufficient condition of n: n > 11−δk0
> 1.
This is a generalization of the result in Notay & Vassilevski 2008.
X. Hu (Penn State) Effective Solvers for Reservoir Simulation Linz,Dec.14, 2011 43 / 47
Ongoing Work and Conclusions Ongoing Work
Primary Results
1024×1024 2048×2048
Setup Solve Total Setup Solve Total
CPU 0.80 7/0.69 1.49 3.28 7/2.75 6.03
GPU (CUSP) 0.63 36/0.35 0.98 2.38 41/1.60 3.98
GPU (FASP) 0.13 19/0.47 0.60 0.62 19/2.01 2.63
Processor:
I CPU: Intel i7-2600 3.4 GHzI GPU: Tesla C2070
Algorithm:
I CPU: Classical AMG + V-cycle;I GPU(CUSP): Smoothed Aggregation AMG + V-cycle;I GPU(FASP): UA-AMG + Nonlinear AMLI-cycle
Ref: Brannick, Chen, H., Xu, Zikatanov, 2011
X. Hu (Penn State) Effective Solvers for Reservoir Simulation Linz,Dec.14, 2011 44 / 47
Ongoing Work and Conclusions Ongoing Work
Other ongoing work
Compositional Model (Equation of State)
Complex wells
Improve robustness of FASP solvers
Improve robustness of nonlinear iteration
Heterogeneous parallel computing
X. Hu (Penn State) Effective Solvers for Reservoir Simulation Linz,Dec.14, 2011 45 / 47
Ongoing Work and Conclusions Conclusions
Conclusions
We developed effective solvers for reservoir simulation, which workswell for reservoirs from real world.
We developed a solver package (FASP). Part of the solver packagehas be used for the chemical flooding simulator (SOCF) by MonixEnergy Solutions and CNOOC.
We are working on parallelization of FASP package, includingOpenMP, MPI and CUDA.
We are working on the robustness of our package.
X. Hu (Penn State) Effective Solvers for Reservoir Simulation Linz,Dec.14, 2011 46 / 47
Ongoing Work and Conclusions Conclusions
Thank you!
X. Hu (Penn State) Effective Solvers for Reservoir Simulation Linz,Dec.14, 2011 47 / 47