Contributions of Canonical Modeling To Stochastics and ... · Contributions of Canonical Modeling...

Contributions of Canonical ModelingContributions of Canonical ModelingTo To StochasticsStochastics and Statisticsand Statistics

Eberhard O. VoitBiomedical Engineering

Georgia Tech and Emory3 May 2007

0 90 1800.9

1

1.1

Canonical Modeling in a Stochastics Seminar?

Voit talking about Stochastics?

Other Mismatches?

http://imagecache2.allposters.com/images/pic/AWI/NR9512~Target-1974-Posters.jpg

στóχος στοχαστικóς

“able to hit the target”“of keen mind”

“smart”Jasper Johns

Part 1: Canonical ModelingDefinitionPower-law modelsRecasting

Part 2: Deterministic versus StochasticDualityDeterministic chaos

Part 3: S-DistributionPropertiesModeling possibilities

Overview

Canonical Modeling

Definition (Merriam Webster Dictionary):

Model (verb): “to produce a representation or simulation of

<using a computer to model a problem>”

Canonical:“conforming to a general rule

or acceptable procedure”

Canonical Modeling

Examples:

Linear Dynamic Model:

Nonlinear Canonical (Dynamic) Models:Combinations of:

DifferentiationLogarithmSum

UXAX +⋅=&

Canonical Modeling

Examples:

Lotka-Volterra:

∑

∑

∑

=

=

=

+=

+==

+==

n

jjiji

i

n

jjijiiii

i

n

jjiijiii

i

Xbadt

Xd

XbaXXXdt

dX

XXbXaXdt

dX

1

1

1

ln

// &

&

Canonical Modeling

Examples:

Log-Lin Model:

Lin-Log Model:

Simplified: )ln(

ln1/

lnln1/

1

10

00

00

10

00

00

j

mn

jiji

i

mn

j j

jij

i

ii

i

i

i

mn

j j

jij

i

ii

i

i

i

Xbadt

dX

XX

ee

Jdt

dXJv

XX

ee

Jdt

dXJv

∑

∑

∑

+

=

+

=

+

=

+=

⎥⎥⎦

⎤

⎢⎢⎣

⎡⎟⎟⎠

⎞⎜⎜⎝

⎛+==

⎟⎟⎠

⎞⎜⎜⎝

⎛+⎟⎟

⎠

⎞⎜⎜⎝

⎛+==

ε

ε

Canonical Modeling

Examples:

“Half-System”

= “Riccati-System”:

∑

∏

+

=

+

=

+=

=

mn

jjijii

mn

j

fjii

XfX

XX ij

1

1

)ln()ln()ln( γ

γ

&

&

Formulation of a Canonical Model for Complex Systems

Xi

Vi+ Vi

–−+ −== ii

ii VV

dtdX

X&

Approximate Vi+ and Vi

– in a logarithmic coordinate system

Biochemical Systems Theory (BST); Canonical ModelingSavageau, 1969

Log Xi

Log Vi+/-

Why not Use “True” Mechanisms and Rate Functions

E(1)

EAB(3)

EQ(4)

EA(2)

k12

k23

k41

k34

k14

k43

k21

k32 EPQ

A+B P+Q

(B)(P)(Q)AB coef.

coef.BB coef.

coef.BQBQ coef.

coef.BPQ

(A)(B)(P)AB coef.

coef.ABP

(P)(Q)AB coef.A coef.

A coef.constant

constantcoef.Q

coef.Qcoef.PQ

(B)(Q)AB coef.

coef.BB coef.

coef.BQ(A)(P)AB coef.

coef.AA coef.

coef.AP

(Q)AB coef.A coef.

A coef.constant

constantcoef.Q

(P)AB coef.A coef.

A coef.AP coef.

AP coef.coef.P(A)(B)

AB coef.AB coef.

(B)AB coef.B coef.(A)

AB coef.A coef.

AB coef.A coef.

A coef.constant

(P)(Q)num.1num.2

AB coef.num.1 (B)(A)

AB coef.num.1

⎟⎟⎠

⎞⎜⎜⎝

⎛××+

⎟⎠⎞

⎜⎝⎛+

⎟⎟⎠

⎞⎜⎜⎝

⎛×××+

⎟⎠⎞

⎜⎝⎛ ×+⎟

⎠⎞

⎜⎝⎛ ×+

⎟⎠⎞

⎜⎝⎛ ××+

⎟⎠⎞

⎜⎝⎛ ××+⎟

⎠⎞

⎜⎝⎛+

⎟⎠⎞

⎜⎝⎛+⎟

⎠⎞

⎜⎝⎛+⎟

⎠⎞

⎜⎝⎛ ×

⎟⎠⎞

⎜⎝⎛ ×⎟

⎠⎞

⎜⎝⎛

=-

v

from Schultz (1994)

Is an approximation “allowed”?

A+B P+Q

How Bad of an Approximation is this?

On one hand...

“True”process:

Molecular dynamics“Firing” of some individual reaction Ri in ΔtChemical Master EquationMild assumption on medium homogeneityStochastic mass action descriptionAverage rate: Poisson processNo Brownian motion: Langevin equationScale time: Itô formalismLarge numbers of molecules:

deterministic mass action lawQuasi-steady-state assumption:

Michaelis-MentenLocally approximate with power-law

How Bad of an Approximation is this?

On the other hand...

Only two assumptions:Deterministic formulation is o.k.System is (locally) spatially homogeneous

Then:Taylor theory guarantees that power-law is: perfect at operating point;good close to operating point.

Biological experience shows that range ofvalid approximation is often quite wide

Alternative Power-Law Formulations

mniiimniii hmn

hhi

gmn

ggii XXXXXXX ++

++ β−α= ,21,21 ... ... 2121&

S-system Form:

Xi

Vi1+ Vi1

–

Vi,p+ Vi,q

–

−+ ∑∑ −== ijiji

i VVdt

dXX&

Alternative Power-Law Formulations

mniiimniii hmn

hhi

gmn

ggii XXXXXXX ++

++ β−α= ,21,21 ... ... 2121&

S-system Form:

Xi

Vi1+ Vi1

–

Vi,p+ Vi,q

–

−+ ∑∑ −== ijiji

i VVdt

dXX&

Generalized Mass Action Form:

∑ ∏±= ijkfjiki XX γ&

Structure Determined by Parameter Values

X1 X 2 X 3 X 4

X1 X 2 X 3 X 4

2112gXα

4121412gg XXα g41 < 0

Crucial consequence :Identification of structure becomes parameter estimation

g41 = 0

Interesting Feature of S-systems:Steady-State Equations Linear

Define Yi = log(Xi):

mnmniiii

mnmniiii

YhYhYhYgYgYg

++

++

+++β=

+++α

,2211

,2211

loglog

S-system highly nonlinear, but steady-state equations linear.

IIDDD YAAbAY ⋅⋅−⋅= −− 11

0 ... ... ,21,212121 =β−α= ++

++mniiimniii h

mnhh

ig

mngg

ii XXXXXXX&

Characterization of Steady States

IIDDD YAAbAY ⋅⋅−⋅= −− 11

Slight change in an input exclusively affects YI.

ID

D

AA

bY

⋅=

∂∂

−1 Gains (w.r.t. independent variables)

1−= DA

∂∂ D

YI

Y

Sensitivities (w.r.t. rate constants)

Computation of eigenvalues for stability analysis; easy criteria for Hopf bifurcations

Applications of S-systems and BST

Pathways: purines, glycolysis, citric acid, TCA, red blood cell,...metabolic engineering; optimization, general design and operating principles

Genes: circuitry, regulation,…

Genome: explain expression patterns upon stimulus

Growth, immunology, pharmaceutical science, forestry, ...

Math: recasting, function classification, bifurcation analysis,...

Statistics: S-system representation, S-distribution, trends;applied to seafood safety, marine mammals, health economics

Big Question:

Canonical models appear to be very restrictive. So, what is the range of dynamics that can be modeled with them (e.g., S-systems)?

“Recasting”

mniiimniii hmn

hhi

gmn

ggii XXXXXXX ++

++ β−α= ,21,21 ... ... 2121&

S-system Form:

Various phenomena are special cases:

radioactive decaymass-action kineticsallometryexponential growth

Observations

2NKrrNN −=&E.g., logistic growth:

Gompertz growth function:

Auxiliary Variables

)]exp(exp[)( tVtV f αβ −−=

)exp( tk ααβ −=

kk

kVV

α−=

=&

&

(special case of an S-system)

t-Distribution

2/)1(2

1

2

21

1)(+−

⎥⎦

⎤⎢⎣

⎡+

⎟⎠⎞

⎜⎝⎛Γ

⎟⎠⎞

⎜⎝⎛ +

Γ=

ν

νν

ν

πνttf

(special case of an S-system)

)(,, 32

21 tfXtXctX =+=+= ν

31

2131

23

12

1

)1()1(

22

1

XXXXXcX

cXX

X

−− +−+=

−=

=

νν&

&

&

Noncentral t-Distribution

)(

2

21

)2/exp(),,(2/)1(

2

2

tSt

tf+−

⎥⎦⎤

⎢⎣⎡

+⎟⎠⎞

⎜⎝⎛Γ

⎟⎠⎞

⎜⎝⎛ +

Γ−

=ν

νν

ν

ν

πνδδν

j

j tt

j

j

tS ⎥⎦

⎤⎢⎣

⎡

+⎟⎠⎞

⎜⎝⎛ +

Γ

⎟⎠⎞

⎜⎝⎛ ++

Γ= ∑

∞

=2

0

2

!2

12

1

)(ν

δν

ν

Noncentral t-Distribution (cont’d)

j

j tt

j

j

ttf ⎥

⎦

⎤⎢⎣

⎡

+⎟⎠⎞

⎜⎝⎛ +

Γ

⎟⎠⎞

⎜⎝⎛ ++

Γ⋅⎥⎦

⎤⎢⎣⎡

+⎟⎠⎞

⎜⎝⎛Γ

⎟⎠⎞

⎜⎝⎛ +

Γ−

= ∑∞

=

+−

20

2/)1(

2

2 2

!2

12

1

2

21

)2/exp(),,(ν

δν

ν

νν

ν

ν

πνδδν

ν

532

89

532

8

72

22

72

212

7

175

5.126

765.1

25

34

31

2131

23

12

1

)2/exp(),,(

)2/exp(

)1(

)1()1(

221

XXtfX

XXX

XXcXXXX

XXXX

XXXX

XX

XXXXXcX

cXXX

δδν

δ

νδνδ

δνν

νδ

νν

−==

−=

−=

+=

=

=

+−+=

−=

=

−−

−−

−

−−

&

&

&

&

&

&

&

&

central t densitycumulative central t

cumulative noncentral tnoncentral t density

Noncentral t-Distribution (cont’d)

Added Value:

Can compute:DensityCumulativeMean, Variance, MomentsQuantilesPower curves

Can do the same for virtually allprobability density functions!

Theorem (Savageau and Voit, 1987)

Let

be a set of differential equations, wherein fi is composed of sums and products of elementary functions, or nested elementary functions of elementary functions. Then there is a smoothchange of variables that recasts (*) into an S-system of the form

Recasting

),...,,( 21 nii ZZZfZ =&0)0( ii ZZ = ni ,...,2,1= (*)

∏∏==

−=m

j

hji

m

j

gjii

ijij XXX11

βα&0)0( ii XX = mi ,...,2,1=

where αi and βi are non-negative and gij and hij are real.

Recasting (cont’d)

Recasting “embeds” systems of arbitrary differential equationsin a higher-dimensional space.

The structure of the recast set is that of an S-system.

Initial values confine the solution within the higher-dimensional space onto a trajectory that correspondsexactly to the solution of the original system.

Of note: linearity of steady state equations is lost; system matrix AD no longer invertible; some variables 0.

Recasting (cont’d)

Not surprising, recasting is also possible to the GMA form

∏=

=m

j

kjii

ijXX1

γ&

Surprising, recasting can be continued to even simpler forms, such as the Half-system or Riccati system:

∑ ∏±= ijkfjiki XX γ&

or even:}1,0{

1

∈= ∏=

ij

m

j

ejii eXX ijη&

Recasting (cont’d)

It is also possible to recast to the Lotka-Volterra form:

∑=

+⋅=n

jjijiii XbbXX

10 )(& mi ,...,2,1=

(Peschel and Mende, 1986).

It is also possible to recast to the Generalized Lotka-Volterra form: GMA system where each equation contains every power-law term of the system (where necessary with kinetic order 0) (Hernández-Bermejo and Fairén, 1997)

Note: It seems not possible to recast to log-lin and lin-log forms.

All the above “canonical forms” are ultimately combinations of

∂, +, · , and log

Intriguing Consequence

Because of recasting, all differentiable nonlinearities are ultimately combinations of

∂, +, · , and log

Value Added through Recasting

Homogeneous mathematical formatPossibility of classificationCustomized methods, softwareEfficient Taylor algorithm for integration Efficient Taylor algorithm for dynamic sensitivity analysisStreamlined Lie-group analysis for conserved quantitiesStatistics:

quantilespower curvestime-varying distributions

e.g., dynamic confidence intervalsStability analysis of non-polynomial vector fields

(Papachristodoulou and Prajna, 2006)

Deterministic versus Stochastic

Well-known “duality”

Deterministic systems can be intrinsicallyunpredictable and indistinguishable

from (stochastic) chaos.

Stochastic systems have average features that arewell characterized and essentially deterministic.

Ultimate example: Gas laws.

Example: Rössler Oscillator

41

411

4

4131

14

13

13

122

4321

5.29

6.49465,1

5.0

XXXX

XXXXXX

XXX

XXXX

−−

−−−−−−

−=

−=

−=

−=

μμ

λλ μλμλ

μλ

&

&

&

&

μ/14

3

2

1

50)0(

1)0(47)0(25)0(

=

===

X

XXX

0 90 180

0

30

60

X1 X2

10 25 40

40

50

60

X1

X2

Not Chaotic Enough? Coupled Rössler Oscillators

0 90 180

0.9

1

1.1

X10

R1

R2

t

t

Y1

Y2

Y2

Blue-Sky Catastrophe:

Other Example

)sin(25.0 3 tAxxx ⋅=−+ &&&

GMA-form

325.0)2( xxysAyyx

−+−−⋅=

=

&

&

2)cos(2)sin(

+=+=

tcts

stccts

−=−=−==2)sin(

2)cos(&

&

New variables3

2121

21

25.0)2( −+−−⋅=

⋅+⋅=⋅=

xxysA

zzzzyzzy

&&&

S-system form1

13

2

121

21

)(

]25.0)2([−

−

⋅−=

⋅−−⋅=

⋅=

zxxz

zysAz

zzx

&

&

&

GMA form325.0)2( xxysAy

yx−+−−⋅=

=

&

&

Blue-Sky Catastrophe (cont’d)

stccts

−=−=−==2)sin(

2)cos(&

&

0 250 500

-1.5

0

1.5x

A = 0.26498 A = 0.264510 250 500

-1.5

0

1.5x

S-Distribution

Possible to recast essentially all known continuous univariatedistributions.

Problem: Often too many auxiliary variables

Alternative: Approximate cumulatives with one S-system equation, “S-distribution”:

)( hg FFF −⋅= α& 5.0)( =mF

Initial value F(m): location; α ~ 1/σ; g, h: shape

S-Distribution: Many Shapes Possible

0 10 20

0

0.25

0.5f1 f2 f3

001.0,1

)(

)(

)(

53

2.133

5.12

75.022

5.01

25.011

==

−=

−=

−=

iF

FFF

FFF

FFF

α

α

α

α

&

&

&

S-Distribution: Good Fits

Example: noncentral t-distribution

S-Distribution: Classification

S-Distribution: Classification

Distributions classifiable by two shape parameters (g, h):

S-Distribution: Other Features

Approximation of discrete distributionsRelatively unbiased data specification (identification

of appropriate distribution)Various estimation methods (e.g., MLE)Time-dependent confidence intervals (definition of

“normal” in health statistics)Extension to several variatesExtensions to multimodalityExtensions to greater shape flexibility

S-Distribution: Rather General Random Number Generator

Use “quantile form” of desired S-distribution

)(1)( hg

hg

FFdFdXFF

dXdF

−=−=

αα

S-Distribution: Rather General Random Number Generator

1. Generate uniformly distributed random numbers, ui ∈(0,1)2. Solve quantile equation to ui

3. Value is ri.4. Collection of ri are S-distributed.5. Great for Monte Carlo simulations

ui

ri0

1

Poisson Process per S-Distribution:

1. Known that inter-arrival time of Poisson process is exponentially distributed.

2. Thus, generate exponentially distributed random numbers from S-distribution and use them to determine next event.

3. Using dynamic rate for exponential distribution results in a non-homogeneous Poisson process.

Linear-Logistic Risk Modeland Cox’s Proportional Hazard Model

Can show that famous models in epidemiology (including

models for two-by-two tables, odds ratios, relative risks) are

direct consequences of disease models formulated as

S-systems.

S-Distribution: Liouville’s Problem

Can we solve a differential equation simultaneously under many different initial values?

Reformulate: How is a distribution of valuestransformed under the dynamics of a system ofdifferential equations? (Example: S-system)

Generic transformation of a distribution:

))(()()( 11 YfYdYdYf xy

−−= ϕϕ

S-Distribution: Liouville’s Problem

Question: What is the distribution of initial values after τ time units?

Suppose, X1 is of interest in the dynamic system

),...,(

),...,(

1

1

nii

nii

YYY

XXX

Ψ=

Ψ=&

&

),(),(

00,

00,

τ+=

=

tXYtXX

ii

ii

nini

,...,1,...,1

==

S-Distribution: Liouville’s Problem

),...,(),...,(

1

),...,(),...,(

11

111

1

1

11

111

nnii

nnii

YYYYdYdYdYdY

YYXXdYdX

−

−

ΨΨ=

=

ΨΨ= )( 00, tXX ii =

)( 00, τ+= tXY ii

Similar to the quantile equation, we can make Y1 theindependent variable by dividing the entire system by the equation of Y1

S-Distribution: Liouville’s Problem

Define distribution of X1 and express in terms of Y1:

),...,(),...,()()(

)(

11

11111

11

1

1

11

1

nnxxx

xx

YYXXFdYdX

FdYdF

FdXdF

−ΨΨ==

=

ϑϑ

ϑ

Example

25.6

16

2

25.0

11

107

2.0

XXX

XXX−×−=

=&

&

1)0(10)0(

2

61

== −

XX

0 2 4

0

0.45

0.9f1

0 20 40

0

3

6X1

Question: If population is distributed at t = 0 with f, what is its size distribution at time t = τ ? Any guesses?

)(10 8.0 FFfF −==&

Result

Like

lihoo

d

Summary

Canonical power-law models were devised for continuous,deterministic descriptions of dynamic phenomena.

Canonical models seem restrictive, but are in truth very flexible

Recasting shows that canonical models capture virtually alldifferentiable nonlinearities, including probability distributions

S-distributions approximate traditional distributions and allowfor interesting applications