System identification of the partial differential equations ......System identification of the...

System identification of the partial differential equationsgoverning pattern-forming in biophysics

Zhenlin Wang Xun Huan Krishna GarikipatiDepartments of Mechanical Engineering and Mathematics, University of Michigan, Ann Arbor, MI, USA. http://www.umich.edu/ compphys/

IntroductionPattern formation is central to the developmental biological processes of anymulticellular organism. Identification of the PDEs governing patternformation delivers insights to the biophysics of developmental dynamics.Approaches to data-driven discovery of PDEsI Inference of the parameters in a known governing equationI Learning the parameters that define an approximate modelI Learning the governing partial differential equations by identifying their

operatorsI Symbolic regression by genetic algorithms M. Schmidt, H. Lipson, Science, 3, 81-85, 2009I ”Physics-informed” deep neural networks

(incorporating the strong form in the loss function to learn parameters of knownoperators) M. Raissi, et al., J. Comput. Phys., 378, 686-707, 2019

I Sparse regression S. L. Brunton, et al. Proc. Natl. Acad. Sci., 113, 3932-3937, 2016; S. H. Rudy, et al., Sci. Adv., 3, e1602614, 2017

Parabolic PDEs that model pattern formation in biophysicsI Diffusion-reaction equations following Schnakenberg kinetics

∂C1∂t

= D1∇2C1 + R10 + R11C1 + R13C21C2∂C2∂t

= D2∇2C2 + R20 + R21C21C2

I Cahn-Hilliard equation for segregation of cell types in tissues

∂C1∂t

= ∇ ·(

M1∇(∂g∂C1

− k1∇2C1))

∂C2∂t

= ∇ ·(

M2∇(∂g∂C2

− k2∇2C2))

I Allen-Cahn equation for tissue patterning by nucleation and growth

∂C1∂t

= ∇ ·(

M1∇(∂f∂C1

− k1∇2C1))

∂C2∂t

= −M2

(∂f∂C2

−∇ · k2∇C2)

Identification of governing parabolic PDEs inweak form

I General strong form for first-order dynamics:∂C∂t

− χ ·ω = 0 χ = [1, C, C2, ...,∇2C, ...]

I The Galerkin weak form (using NURBS basis functions) leads to theresidual form Ri = 0, i = 1, . . . , N:∫

Ω

w(∂C∂t

− χ ·ω)

dv = 0⇒ Ri = Fi

(∂Ch

∂t, Ch,∇Ch, ..., N,∇N...

)Aim to findω for many operators in weak form!

I Examples of basis in the weak form

ΞĊi |n =

∫Ω

Ninb∑

a=1

can − can−1∆t

Nadv time derivatives

Ξ∇2C

i |n = −

∫Ω

∇Ni ·nb∑

a=1

can∇Nadv Laplace operator

Ξ∇2CBC

i |n =

∫Γ

Ninb∑

a=1

1ds Neumann boundary: constant flux

...

Matrix-vector form of residual equations

I Target vector: Matrix containing operators in weak form:

y =

...

ΞĊ|n−1

ΞĊ|n

ΞĊ|n+1...

Ξ =

... ... ... ... ... ... ... ...Ξcons|n−1 Ξ

C|n ΞC2|n−1 ... Ξ∇

2C|n−1 Ξ∇4C|n−1 Ξ

∇2CBC|n−1 ...Ξcons|n Ξ

C|n ΞC2|n ... Ξ∇

2C|n Ξ∇4C|n Ξ

∇2CBC|n ...Ξcons|n+1 Ξ

C|n+1 ΞC2|n+1 ... Ξ∇

2C|n+1 Ξ∇4C|n+1 Ξ

∇2CBC|n+1 ...... ... ... ... ... ... ...

I Residual equation:

R = y − Ξω

Regression problemI Findω to minimize the loss function, l = ||R||2:

ω = arg minω

l(ω)

Standard regression will result in a non-parsimonious solution forωwithnonzero contributions in each component of this vector

I Penalization to drive pre-factors toward zero

ω =arg minω

l(ω) + λ2 ∑j

ω2j

Ridge Regressionω =arg min

ω

l(ω) + λ1 ∑j

|ωj|

LASSO-sparsity inducingSelecting few relevant terms from large candidate set is challenging

Identification of operators via stepwiseregression

Stop

1 2 3 4 5iterations

0

200

400

loss

func

tion

Low fidelity and noisy dataI Data generated by simulation on high fidelity mesh, but may be

subsampled by collection over a subset of nodes (lower fidelity mesh)

High delity Low delity

I Observed data Ĉ may be noisy

Ĉ = C + �I Noise is amplified through spatial gradient and time derivative.I Lower fidelity data is favorable to smooth out the amplified noise.I Higher fidelity data is favorable to capture sharp variations.

ResultsI Candidate operators

Strong form operators Weak form operatorsy

∫Ω w

∂C1∂t dv

∫Ω w

∂C2∂t dv

∇(∗∇C1)

∫Ω∇w∇C1dv

∫Ω∇wC1∇C1dv

∫Ω∇w∇C2dv∫

Ω∇wC21∇C1dv

∫Ω∇wC1C2∇C1dv

∫Ω∇wC

22∇C1dv∫

Ω∇wC31∇C1dv

∫Ω∇wC

21C2∇C1dv

∫Ω∇wC1C

22∇C1dv

∫Ω∇wC

32∇C1dv

∇(∗∇C2)

∫Ω∇w∇C2dv

∫Ω∇wC1∇C2dv

∫Ω∇w∇C2dv∫

Ω∇wC21∇C2dv

∫Ω∇wC1C2∇C2dv

∫Ω∇wC

22∇C2dv∫

Ω∇wC31∇C2dv

∫Ω∇wC

21C2∇C2dv

∫Ω∇wC1C

22∇C2dv

∫Ω∇wC

32∇C2dv

∇2(∗∇2C)∫Ω∇

2w∇2C1dv∫Ω∇

2wC1∇2C1dv∫Ω∇

2w∇2C2dv∫Ω∇

2wC2∇2C2dv

non-gradient−∫Ω w1dv −

∫Ω wC1dv −

∫Ω wC2dv

−∫Ω wC

21dv −

∫Ω wC1C2dv −

∫Ω wC

22dv

−∫Ω wC

31dv −

∫Ω wC

21C2dv −

∫Ω wC1C

22dv −

∫Ω wC

32dv

boundary condition −∫Γ1

w1ds −∫Γ4

w1ds −∫Γ3

w1ds −∫Γ4

w1ds

I Stem-and-leaf plots show active operators selected by stepwise regressionout of a set of 38 possible choices

Diffusion-reaction operators for two fields C1 and C2 using noise-free and noisy data.

Cahn-Hilliard operators for two fields C1 and C2 using noise-free and noisy data.

Key takeawaysI The weak form regularizes higher-order derivatives.I The variational approach naturally delineates and identifies boundary

conditions.We are working on identifying governing equations using real experimentaldata that is:I Sparse, being available at very coarse time steps.I Incomplete, being only available over subdomains of the full field that are

uncorrelated with respect to time.Reference:Z.Wang, X. Huan, K. Garikipati, Variational system identification of the partial differentialequations governing pattern-forming physics: Inference under varying fidelity and noise,Computer Methods in Applied Mechanics and Engineering, 356, 44-74, 2019.doi.org/10.1016/j.cma.2019.07.007