Nonequilibriumstochastic Processes at Single-molecule and Single-cell levels

Hao Ge (葛颢)haoge@pku.edu.cn

1Beijing International Center for Mathematical Research2Biodynamic Optical Imaging Center

Peking University, Chinahttp://bicmr.pku.edu.cn/~gehao/

BICMR: Beijing International Center for Mathematical Research

BIOPIC: Biodynamic Optical Imaging Center

Summary of Ge group

Stochastic Biophysics(Biomath)

Stochastic theory of nonequilibrium statistical mechanics

JSP06,08PRE09,10,13,14JCP12; JSTAT15

Nonequilibrium landscape theory and rate formulas

PRL09,15JRSI11; Chaos12

Stochastic modeling of biophysical systems

Statistical machine learning of single-cell data

JPCB08,13; JPA12Phys. Rep. 12Science13;Cell14;MSB15

CR15

Which kind of physical/chemical processes can be described by stochastic processes?

• Mesoscopic scale (time and space)

• Single-molecule and single-cell (subcellular) dynamics

• Trajectory perspective

Single-molecule experiments

Lu, et al. Science (1998)

Single-molecule enzyme kinetics

E. Neher and B. SakmanNobel Prize in 1991

Single Ion channel

Single-cell dynamics (in vivo)

Eldar, A. and Elowitz, M. Nature (2010)

Choi, et al. Science (2008)

Stochastic theory of nonequilibrium statistical

mechanics

The fundamental equation in nonequilibrium thermodynamics

Ilya Prigogine(1917-2003)Nobel Prize

in 1977

Entropy production

SdSddS ie

0 k

kki XJSdepr

Clausius inequality

T

QdS

Rudolf Clausius(1822-1888)

Second law of thermodynamics

0T

QdSepr

rewrite

More general

Carl Eckart(1902-1973)

P.W. Bridgman(1882-1961)Nobel Prize

in 1946

Lars Onsager(1903-1976)Nobel Prize

in 1968

Mathematical theory of nonequilibrium steady state

Min Qian (1927-)Recipient of Hua Loo-KengMathematics Prize (华罗庚数学奖) in 2013

Time-independent(stationary) Markov process

Ge, H.: Stochastic Theory of Nonequilibrium Statistical Physics (review). Advances in Mathematics(China) 43, 161-174 (2014)

Master equation model for the single-molecule system

No matter starting from any initial distribution, it will finally approach its stationary distribution satisfying

01

N

j

ij

ss

iji

ss

j kpkp

j

ijijiji kpkp

dt

tdp )(

Consider a motor protein with N different conformations R1,R2,…,RN. kij is the first-order or pseudo-first-order rate constants for the reaction Ri→Rj.

Self-assembly or self-organization

ij

eq

iji

eq

j kpkp

Detailed balance (equilibrium state)

NESS thermodynamic force and entropy production rate

0 ji

ss

ij

ss

ij

ness AJeprT

ji

ss

jij

ss

i

ss

ij kpkpJ

ji

ss

j

ij

ss

i

B

ss

ijkp

kplogTkA NESS thermodynamic force

NESS flux

NESS entropy production rate

Time-dependent case

j

ijijiji tkptkp

dt

tdp

Free energy

j

kTtE

BjeTktF

/)(log)(

j

kTtE

kTtEeq

i

ss

ij

i

e

etptp

/)(

/)(

)()(Boltzmann’s law

If {kij(t)} satisfys the detailed balance condition for fixing t

01

N

j

ij

ss

iji

ss

j tktptktpQuasi-stationary distribution

0 tktptktp ij

ss

iji

ss

j

Mathematical equivalence of Jarzynski and Hatano-Sasa equalities

Ge, H. and Qian, M., JMP (2007); Ge, H. and Jiang, D.Q., JSP (2008);

Jarzynski equality: local equilibrium

kTF

pp

kTW eeeq

/

)0()0(

/

.)0()0(

FW eqpp

Hatano-Sasa equality: without local equilibrium

1

)0()0(

/

ss

ex

pp

kTQe

.kT/Q

S

ex

pp ss

00

Same theorem for time-dependent Markov process

Are these inequalities already known in the Second Law of classic thermodynamics? Do they only hold for the whole transition process between two steady states?

The traditional Clausius inequality can be in a differential form.

Using Feynman-Kac formula of the time-dependent case

Decomposition of mesoscopic thermodynamic forces

tAtA

tktp

tktpTktA ij

ss

ij

jij

iji

Bij log

i j

0 ji

ijijp tAtJteT Entropy production

0 ji

ss

ijijhk tAtJtQ )()()(Housekeeping heat

Free energy dissipation 0 ji

ijijd tAtJtf )()()(

t t t dhkp fQeT Ge, H., PRE (2009);

Ge, H. and Qian, H., PRE (2010) (2013)

tktp

tktpTktA

ji

ss

j

ij

ss

i

B

ss

ij log

All the results have also been proved for multidimensional diffusion process.

0tep for any time tIn the absence of external energy input and at steady state.

0tQhk for any time t In the absence of external energy input

0tfd for any time t At steady state

Two origins of nonequilibrium

Ge, H., PRE (2009); Ge, H. and Qian, H., PRE (2010) (2013)

0 ptot e

T

Q

dt

dS

The new Clausius inequality is stronger than the traditional one.

0

d

hktotex fT

QQ

T

Q

dt

dS

Decomposition of entropy production rate

.QfeT

,Q,f

hkdp

hkd

0

00

Ge, H., PRE (2009);

Ge, H. and Qian, H., PRE (2010) (2013)

T

Qe

dt

dS totp

Mathematical problems left

Mathematical proof for some newly recent

developed finite-time fluctuation theorems of

diffusion process with time-dependent diffusion

coefficients;

Mathematical proof for the large deviation

principle of sample entropy production of

diffusion process on Rn;

Stochastic theory of nonequilibrium statistical

mechanics of second-order stochastic process；

……

A first step towards the stochastic theory of nonequilibrium statistical

mechanics for second-order stochastic system:

Time-reversibility and anomalous behavior

tX,XFdt

Xdm

.

2

2

Two different definitions of entropy production rates

dxdvlogm

xDFF

xDk

Ts:rP

Ts:Plog

TlimkEPR

ttvB

PsTts

st

TB

2

01

22

2

0

01

v,xFF

Spinney, R.E., and Ford, I.J., PRL (2012); Lee, H.K., Kwon, C., and Park, H., PRL (2013)

dxdvlog

m

xDvx

xDkEPR ttvB

2

22

2

Kim, K.H. and Qian, H., PRL (2004)

correspond to time-reversibility and Maxwell-Boltzmann distribution for thermodynamic equilibrium respectively

00 21 EPR,EPR

thermodynamic equilibrium Ge, H.: PRE (2014)

When the external forceis only dependent on position

xGvxv,xF

xxTk

xD

B

2

021 EPREPR

thermodynamic equilibrium

xUxG

;TxT

x

Thermal equilibrium

Mechanical equilibrium

Flow of kinetic energy along the spatial coordinate

t,xQt,xWJxEdt

d kinetic

xx

kinetic

t

00 qx JJEPR

x

kinetic

t

kinetic

xq vxEJxJ (measurable) Heat flux

Ge, H.: PRE (2014)

Thermodynamic equivalence between mesoscopic and macroscopic scales

The entropy production rate in the small-noise limit

Celani, et al.: PRL (2012);

Ge, H.: PRE (2014)

dxˆ

ˆ

J

xT

xEPREPR t

t

over

xoverspatial

2

dxxˆxTx

xTk

nt

xB

over

22

6

2

Entropy production rate of the overdamped-limit

Anomalous contribution of EPR

Hence the overdamped approximation only keeps the dynamics rather than the second law of thermodynamics.

spatialEPREPR

Decomposition of entropy production rate

Local reciprocal relation between linear coefficients

Ge, H.: JSTAT (2015)

qqqpqxq

qxqpxx

over

x

XLXLxJ

XLXLJ

xˆx

xTknL t

Bqq

32

6

8

xˆx

xTL txx

x

xTkLL B

xqqx

2

Always hold, even for far-from-equilibrium system.

Reciprocal relation between Soret effect (thermal diffusion) and Dufour effect

Come from the second moment of velocity along each dimension.

Two-state model of central dogma without feedback

Help to uncover the mechanism of transcriptional burst

DNA transcription

Regulation of gene expression

Induced condition

Highly expressed,

i.e. low repression

Transcriptional burst under induced condition

Golding et al. Cell (2005)

DNA topology and transcriptional burst

Levens and Larson: Cell (2014) (preview) Shasha et al. Cell (2014)

Anchored DNA segment

High-throughput in-vitro experiments

Supercoiling accumulation and gyrase activity

Gene OFF mRNA øGene ON

1k

Shasha et al. Cell (2014)

Two-state model without feedback

0 1 n…… n+1 ……

0 1 n…… n+1 ……

Gene ON

Gene OFF

1k 1k 1k

2

2

)1( n

)1( n

Gene OFF mRNA øGene ON

1k

Chemical master equation

Copy number of mRNA

Ge

ne

sta

tes

The mean-field deterministic model has only one stable fixed point!

Poisson distribution with a spike at zero

When α,β<<k1,γ, then

.1,!

)()()(

;)0()0()0(

1

21

21

1

1

nn

k

enpnpnp

eppp

n

k

k

Poisson distribution with a spike (bimodal)

Duty cycle ratio

Two-state model of central dogma with positive feedback

A rate formula for stochastic phenotype transition—in an intermediate region of gene-state switching

Central Dogma

Copy numbers in a single cell

Bacteria Eukaryotic cells

DNA 1 or 2 ~2

mRNA A few 1 - 103

Protein 1 - 104 1 - 106

Not enough attention has been paid to this fact.

Regulation of gene expression

An example of gene circuit with positive feedback:

Lac operon

Bimodal distributions in biology: multiple phenotypic states

Choi, et al., Science (2008) To, T. and Maheshri, N. Science (2010)

Ferrell, J. and Machleder, E. Science (1998)

Two-state model with positive feedback

1k

n max

large

Mean-field deterministic model with positive feedback

0 5 10 15 20 25 300

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Bifurcation diagram for simple example

1/Keq

x*

OFF state

ON state

x

flux

Influx g(x)

Outflux γx

Stable On-state

Stable Off-state

Unstable threshold

Flux-balance plot

Bifurcation diagram

xxgdt

dx )(

Sigmoidal influx

x xg

maxn

nx

**)( xxg

Interconversion of different phenotypic states

Choi, et al., Science (2008) Gupta, et al., Cell (2011)

How to quantify the transition rates between different phenotypic states, provided their existence?

Recall Langevin dynamics and Kramers rate formula

Chemical reaction activated by diffusional fluctuations

H.A. Kramers (1894-1952)

tf

dt

dx

dx

xdU

dt

xdm

2

2

P. Langevin (1872-1946)

.,0

;2

;0

2

tssftf

Tktf

tf

B

Tk

U

a Be2

large is k

ǂǂ

x. around x,xx m2

1xU

;x around x,xxm2

1xU

22

A

2

a

2

a

ǂǂǂ

𝑞‡ = 1 𝑞A =2𝜋𝑘𝐵𝑇

𝜔𝑎ℎ

𝑘+ = 𝜅𝑘𝐵𝑇

ℎ𝑒−Δ𝐺‡

𝑘𝐵𝑇

= κ𝑘𝐵𝑇

ℎ

𝑞‡

𝑞A𝑒−Δ𝑈‡

𝑘𝐵𝑇

𝜅 =𝜔‡

𝛾

𝛾 =𝜂

𝑚

From single chemical reaction to biochemical networks (biology)

Chemical master equation (CME)

Max Delbruck(1906-1981)Nobel Prize in 1969

Physical state of atoms

Molecular copy number Phenotypic state

Conformational state

Single cell: biochemical network

Single chemical reaction

The state of system

Emergent state at a higher level

M

j

j

M

j

jjj tXPXrtXPXrdt

tXdP

11

,,,

Two-state model with positive feedback

1k

n max

largeThe analytical results introduced here can be applied to

any self-regulating module of a single gene, while the

methodology is valid for a much more general context.

Three time scales and three different scenarios

( protein ofrate synthesis:

)(switching state -gene:

)(cycle cell:

))(

)(,)(

)(

1

1

kiii

nhnfii

i

)i( )iii( )ii( Ao, et al. (2004); Huang, et al. (2010)

When stochastic gene-state switching is extremely rapid

)ii( )i( )iii(Qian, et al. (2009); Wolynes, et al. (2005)

When stochastic gene-state switching is extremely slow

When the time scales of (ii) and (iii) are comparableAssaf, et al. (2011); Li, et al. (2014)

)i( )ii( Wolynes, et al. (2005);Ge, et al. (2015)

When stochastic gene-state switching is relatively slow

)iii(

)i( )ii( )iii(

A single-molecule fluctuating-rate model is derived

Ge, H., Qian, H. and Xie, X.S., PRL (2015)

(A) ,,1 hfk (B)hfk ,,1

0 5 10 15 20 25 300

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Bifurcation diagram for simple example

1/Keq

x*

Bifurcation diagram

xxgdt

dx )(

OFF state

ON state

Rescaled dynamics

Continuous Mean-field limit Fluctuating-rate model

maxn

nx

xn

k

dt

dx

max

1

xn

k

dt

dx

max

2

f2xh

Ge, H., Qian, H. and Xie, X.S., PRL (2015)

Stochastic dynamics of fluctuating-rate model

xn

k

dt

dx

max

1

xn

k

dt

dx

max

2

f2xh

Nonequilibrium landscape function emerges

Dynamics in the mean field limit model

xxgdt

dx )(

0

~

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9-12

-10

-8

-6

-4

-2

0

2

4

6

x

Stable(OFF)

Unstable Stable(ON)

0.01 0.0105 0.011 0.0115 0.0120.69

0.695

0.7

0.705

0.71

0.715

0.72

0.725

0.73

x

xss exp 0

0

Landscape function

analog to energy function at equilibrium case

xn

k

xh

xn

k

f

dx

xd

maxmax

2

2

1

0

Ge, H., Qian, H. and Xie, X.S., PRL (2015)

As gene-state switching is much faster than the

cell cycle

Quantify the relative stability of stable fixed points

෪Φ0 = Φ0 ∕ 𝑓

alternative

attractor

2: fluctuating in local

attractor, waiting

1: relaxation

process

3: abrupt transition

via barrier-crossing

The uphill dynamics is the rare event, related to phenotype switching, punctuated transition in evolution, et al.

Dynamics of bistable systems

Intra-attractorial dynamics

Inter-attractorial dynamics

A B

discrete stochastic model

among attractors

ny

nx

chemical master equation cy

cx

A

B

fast nonlinear differential equations

appropriate reaction coordinate

ABp

robabil

ity

emergent slow stochastic dynamics

and landscape

(a) (b)

(c)

(d)

Three time scalesFixed finite molecule numbers

Stochastic

Stochastic

Deterministic

Ge and Qian: PRL (2009), JRSI (2011)

Rate formulae associated with the landscape function

Gene-state switching is extremely slow

k linearly depend on gene-state switching rates

Wolynes, et al. PNAS (2005)

Gene-state switching is relatively slow

Barrier crossing

ǂ0ekk 0

Ge, H., Qian, H. and Xie, X.S., PRL (2015)

maxn

maxn

maxn

maxn

maxn

A recent example: HIV therapy (activator + noise enhancer)

0ekk 0

off

onoff

k

kk 00

~

Activator: increasing kon , lower the barrier

Noise enhancer: Decreasing both kon and koff, further lower the barrier

Significantly increasing the transition rate

Weinberger group, Science (2014)

Gene ON Gene OFFonk

offk

Rigorous analysis: quasi potential in LDP

Local: The Donsker-Varadhan large deviation theory for Markov process

Global: The Freidlin-Wentzell large deviation theory for random perturbed dynamic system

+

LDT of Fluctuating-rate model (Switching ODE)

Two-scale LDT of Switching(Coupled) Diffusion

See Chapter 7 in Freidlin and Wentzell: Random Perturbations of Dynamical Systems (2nd Ed). Springer 1984

Compared to previous rate formulae for bursty dynamics

Eldar, A. and Elowitz, M. Nature (2010) Cai, et al. Science (2006)

Burst sizef

xMax

k

f

1b off

1

b

x x

0off

ekk

ǂ xx ,

b

1

dx

xd 0

ǂIf

Walczak,et al.,PNAS (2005);Choi, et al.,JMB (2010);Ge,H.,Qian,H.and Xie, X.S.,PRL (2015)

Voice on Cell

Multistability and different time scales

k-1

k1

k2

k-2

k3

k-3

Conformation(Phenotype) 1Conformation(Phenotype) 2

Conformation(Phenotype) 3

Local landscapes and Kramers’ rate formula

Constructed locally

Phenotypic subspace

ijk

jikijV

jiV

i j

1 , 0

ijV

ijij ekk

ij

ijT

k1

ji

jiT

k1

Driving force:

ss

i

ss

j

ijji

ji

ijij

p

pVV

k

klog

1log

1

1 , 0

jiV

jiji ekk

0312312

Multistability: local-global conflictions

Global landscape: from stationary distribution

Just cut and glue on the local landscapes (having non-derivative points).

The emergent Markovian jumping process

being nonequilibrium is equivalent to the

discontinuity of the local landscapes (time

symmetry breaking).

Ge, H. and Qian, H.: Chaos (2012)

1k-1

k1

2

3

Summary

Stochastic processes become more and more popular to

model the nonequilibrium mesoscopic biophysical

dynamics, especially in single-cell biology.

Stochastic theory of nonequilibrium statistical mechanics is

rigorously studied and further developed. Recently we are

interested in the second-order stochastic system with

temperature gradient.

Stochastic model of central dogma at single-cell level helps

to uncover the mechanism of transcriptional burst, and we

propose a single-molecule fluctuating-rate model as well as

an associated saddle-crossing rate formula for the

phenotype transition in an intermediate scenario.

Acknowledgement

Prof. Hong QianUniversity of Washington

Prof. Sunney Xiaoliang XieHarvard UniversityPeking University

Fundings: NSFC, MOE of PRC

Thanks for your attention!