1 Transient dynamics of reduced order models of genetic regulatory networks · 2011-12-20 · 1 Transient dynamics of reduced order models of genetic regulatory networks Ranadip Pal,

1

Transient dynamics of reduced order

models of genetic regulatory networksRanadip Pal, Member, IEEE, Sonal Bhattacharya,

Abstract

In systems biology, a number of detailed genetic regulatory networks models have been

proposed that are capable of modeling the fine-scale dynamics of gene expression. However,

limitations on the type and sampling frequency of experimental data often prevent the parameter

estimation of the detailed models. Furthermore, the high computational complexity involved in the

simulation of a detailed model restricts its use. In such a scenario, reduced order models capturing

the coarse scale behavior of the network are frequently applied. In this paper, we analyze the

dynamics of a reduced order Markov Chain model approximating a detailed Stochastic Master

Equation model. Utilizing a reduction mapping that maintains the aggregated steady state probability

distribution of stochastic master equation models, we provide bounds on the deviation of the Markov

Chain transient distribution from the transient aggregated distributions of the stochastic master

equation model.

Index Terms

Genetic Regulatory Network Modeling Robustness, Transient Analysis, Markov Chains

✦

1 INTRODUCTION

Conceptualization of biological regulation as detailed mathematical system models dates back several

decades [1]. The recent technological advances in the measurement of the regulome (genes, proteins

and metabolites involved in gene regulation) have provideda significant impetus to modeling in systems

biology. However, we are still not at the stage where the enormous complexity of a biological system can

be represented by a single detailed composite model explaining all the interactions between RNAs, Protein

• R. Pal and S. Bhattacharya is with the Department of Electrical and Computer Engineering, Texas Tech University, Lubbock,

TX, 79409, USA.

E-mail: [email protected], [email protected]

• This research was supported by NSF Grant CCF0953366.

December 20, 2011 DRAFT

2

and Metabolites. Furthermore, a conceived model capturingall the interactions will be extremely hard to

validate due to existing limitations on expression measurements. Usually only parts of the regulome are

made explicit in a mathematical model. The complexity of regulation in a single level (transcriptome,

proteome or metabolome) can in itself be enormous and it is often not feasible to reliable estimate the

parameters of a detailed model of a single level from experimental data.

Stochastic Master Equation (SME) models are a form of detailed stochastic genetic regulatory net-

work (GRN) models that are presumed to provide a fine scale description of the underlying genomic

regulation process [2], [3], [4]. The inference of the parameters of an SME model preferably require

cell specific measurements whereas most of the available datasets on gene expression measurements are

based on microarray type approaches that provide population averaged data. Availability of cell specific

measurements based on green fluorescent protein type approaches is still extremely limited due to higher

cost and complexity as compared to existing approaches. Theinference of SME model parameters also

lacks preciseness due to low sampling rates in biological experiments and the extrinsic noise in data

extraction. Furthermore, the use of SME type models for representing GRNs are also limited by the

enormous complexity involved in its simulation and reliance on monte-carlo type approaches for design

of any intervention approaches based on such models. To provide an estimate of the complexity of an

SME model, let us consider that we are trying to model the interactions between 10 mRNAs using

an SME model and the number of mRNA molecules that can be created for each mRNA is between

0 and 99. The SME model can be considered as a continuous time Markov Chain with any possible

combination of the number of RNA molecules being a state. Thus, we will have a10010 = 1020 state

continuous time Markov chain with enormous complexity involved in its simulation. In such a scenario,

reduced order models are often used. One of the ways to reducethe complexity of a SME model is

to consider a model that captures moments (expectations or variances) of the distribution. Deterministic

differential equation models are often used to model the average behavior of a stochastic system [5],

[6], [7], [8], [9]. On the other hand, we can also consider a coarse-scale stochastic model to capture

our dynamic behavior. For our earlier example, if we binarize the expression levels of the mRNAs, then

we have to work with only210 = 1024 state Markov Chain. But the natural question arises: whether

such a coarse model is useful? The answer is in-fact yes as fora number of systems medicine scenarios

where we are trying to predict the onset of a genetic disease or effect of application of a drug or gene

silencing, we are often able to predict the phenotypes basedon binarized or ternarized expressions of

molecular markers [10], [11], [12]. In this article, we willconsider a coarse-scale Markov Chain (CSM)

to represent the reduced-order model and an SME model to represent the detailed model. We will use a

mapping that was recently introduced [13] to map the detailed model to the coarse-model. The mapping

was shown to maintain the aggregated steady state probability distribution of the detailed model and

the transition probabilities of the reduced order model canbe independently estimated from coarse-scale


3

steady state data [13]. However, the bounds on the deviationof the transient dynamics (whether the

transient response of the reduced order models is comparable to the coarse-scale transient response of

the finer models) was not explored. The transient dynamics can be highly significant if our purpose is

intervention for systems medicine and we want to maintain the probability of a state within specific

limits. If our design is based on the approximate model, we are aware of the state probabilities of the

approximate model but we would be interested in bounding thedeviation of the state probabilities of

the coarse-model from the aggregated state probabilities of the detailed SME model.

In this paper, we analyze and provide bounds for the difference in the transient behavior of the CSM

and SME models. We also consider the bound on the difference in state probabilities for the CSM and

aggregated SME models when a stationary control policy designed on the coarse-model is applied to the

CSM and SME models. We also derive stochastic bounds for the deviations in the transient dynamics.

The stochastic bounds are computationally inexpensive to compute and they require partial distribution

information of the entries of the transition probability matrix for their calculation.

The paper is organized as follows. The background to the problem including a review of SME, CSM

models and the mapping from SME to CSM is presented in Section2. Section 3 characterizes the

deviations for single and multiple time steps and provides the corresponding upper bounds. Section 4

provides stochastic bounds for the maximum deviation. Transient analysis under the action of stationary

control is provided in Section 5. Simulation results are included in Section 6 and conclusions are provided

in Section 7.

2 BACKGROUND

2.1 SME Models

To explain a SME model, we will consider a system withn molecular species andm different reaction

channels where the state of the system is defined byθ = [χ1, ..., χn] , θ ∈ Nn is a vector of integers

representing a specific population of each of then molecular species. For such a system, given the

probability density vectorp(θ, t) at time t, we can derive the differential equation [4]

p(θ; t) = −p(θ; t)m∑

j=1

aj(θ) +m∑

j=1

p(θ − vj ; t)aj(θ − vj) (1)

whereaj(θ)dt denotes the probability that thejth reaction will happen in a time step of lengthdt and

vj is the stoichiometric transition vector.

By considering all the reactions beginning or ending at state θ, the time derivative of the prob-

ability density of stateθ can be written in the form [14]:P(Θ; t) = P(Θ; t)A whereP(Θ; t) =

[p(θ1, t), p(θ2, t), · · · ] is the complete probability density state vector at timet andA is the state reaction

matrix. For the case of finite number of reachable states, theexact solution to the SME can be computed


4

asP(Θ; t) = P(Θ, 0)eAt [14]. This kind of situation can arise in biological systemswhere the number

of mRNA/Protein molecules that can be generated are bounded. If for systems, the dimension ofΘ is

infinite, we can apply the Finite State Projection approach [14] to arrive at a finite truncation of the state

space.

2.2 Coarse-scale Markov Chain Models

Coarse-scale deterministic models with synchronous timing have been used since the late 1960s in the

form of the Boolean network model [15]. In this model, gene expression is quantized to two levels:

ON and OFF denoted by 1 and 0, respectively. The expression level (state) of a gene is functionally

related via a logical rule to the expression states of other genes. Specifically, a Boolean network (BN)

is composed of a setV = {ν1, ν2, · · · , νn} consisting ofn binary variables, each denoting a gene

expression, and a setF = {f1, f2, · · · , fn} of regulatory functions, such that for discrete time,t =

0, 1, 2, · · · , νk(t+1) = fk(ν1(t), · · · , νn(t)). At any time point, the state of the network is given by an

expression vector(ν1(t), · · · , νn(t)), called the gene activity profile (GAP). However the assumption of

a single transition rule for each gene can be problematic dueto inherent stochasticity of gene expression,

presence of latent variables and noise in the experimental data used for inference. Thus, a probabilistic

Boolean network (PBN) [16] was proposed. The probabilisticstructure of the PBN can be modeled as

a Markov chain. Relative to their Markovian structure, PBNsare related to Bayesian networks [17];

specifically, the transition probability structure of a PBNcan be represented as a dynamic Bayesian

network and every dynamic Bayesian network can be represented as a PBN (actually, a class of PBNs)

[18]. As the reduced order model in this paper, we will consider coarse scale Markov Chain (CSM) which

includes the class of PBNs. The CSM model provides the transition probabilities between states in the

next discrete time duration. The primary difference from the SME model is the significant small number

of states considered and the discrete time dynamics. The state aggregations are based on thresholds

used for binarization. As an example, if we consider an SME model of 2 genes and each producing a

maximum of39 mRNA molecules, then the SME model will have40× 40 = 1600 states. For the CSM

model, if we use a binarization threshold of20, then a state like [30 11] in the SME model will be

mapped to state [1 0] in the CSM model. The CSM model will have atotal of 2× 2 = 4 states.

2.3 SME and its CSM approximation

Based on the state transitions of a SME model, it can be considered as a continuous time Markov Chain

with a huge number of states. To explore mappings from fine-scale SME models to CSM models, let

us considerM andN to denote the number of states of the SME and CSM model respectively. We

consider a sequence 0 =a0, a1, ..., aN = M such thataj−aj−1 for j ∈ {1, 2, ..., 2n} denote the number

of states in the SME model that map to statej in the CSM model. Note that the number of states in


5

the SME model that map to statej in the CSM model can vary withj depending on the thresholds

used for binarization. For instance, if we consider our previous example of 2 genes with 1600 states and

change the binarization thresholds to 10 rather than 20, then 10 × 10 = 100 states in the SME model

will be mapped to state[00] in the CSM model and10 × 30 = 300 states in the SME model will be

mapped to state[01] in the CSM model. LetP∆ = eA∆t represent theM dimensional discrete time

SME model (∆t is a suitable time period) andPr represent the reducedN dimensional CSM model.

For the SME model, letη represent theM dimensional steady state probability vector andζ represent

the N dimensional collapsed steady state vector. Here, collapsing refers to aggregation of states i.e.

ζ(i) =∑ai

i2=ai−1+1 η(i2) for i = 1, ..., N . In [13], a mapping from SME to CSM model is presented

where the steady state probability distribution of the reduced modelPr represented by Eq. 2 is equivalent

to the collapsed steady state probability distribution ofP∆.

Pr(i, j) =

∑aj

j1=aj−1+1

∑ai

i1=ai−1+1 P∆(i1, j1)η(i1)∑ai

i2=ai−1+1 η(i2)(2)

With Eq. 2 denoting the state transition probabilities of the CSM model, the steady state probability

distribution vector,π, of the CSM model is given by [13]:π(i) =∑ai

i2=ai−1+1 η(i2) = ζ(i) for i =

1, ..., N . Eq. 2 provides a mapping from a fine-scale SME model to a CSM model based on aggregation

of states that maintains the collapsed steady state distribution of the detailed model. If we revisit Eq.

2, we notice thatPr(i, j) can be calculated from the transitions of the network once ithas reached the

steady state. Thus, the mapping described by Eq. 2 exists between a network represented by a SME

model and a CSM model when the transition probabilities of the CSM are inferred based on coarse-scale

state transition data at steady-state.

3 MAXIMUM DEVIATION FOR SINGLE TIME STEP

The focus of this paper is analyzing the deviation of the state probability distribution of the CSM model

from the aggregated state probability distribution of the SME model during the transient phase. Let

x(t) = [x1(t), x2(t), ..., xM (t)] 1 denote the SME model’s probability distribution at any timet. Let

γj(t) denote the collapsed probability of statesaj−1 + 1 to aj of the SME model at any timet i.e.

γj(t) =∑aj

i1=aj−1+1 xi1(t). Let τj(t) denote the probability of statej at any timet for the CSM model.

We are interested in finding the difference at time instantt +∆t between the collapsed probability of

the states in the SME model that map to statej of the CSM model (i.e.γj(t+∆t)) and the probability

of statej in the CSM model (i.e.τj(t+∆t)) . Let us define the difference equation as

dj(t+∆t) = |γj(t+∆t)− τj(t+∆t)|. (3)

1. Note thatx(t) representsP (Θ; t) and [x1(t), x2(t), ..., xM (t)] = [p(θ1, t), p(θ2, t), · · · , p(θM , t)]


6

Theorem 3.1. The maximum deviation betweenγj(t+∆t) and τj(t+∆t) starting fromγj(t) = τj(t)

is given by the following equation:

dj(t+∆t) ≤ maxi1∈[1,...,N ]

(

maxi2∈Si1

q(i2, j)− mini2∈Si1

q(i2, j)

)

(4)

whereSi = [ai−1 + 1, ..., ai] and q(i2, j) =∑aj

j1=aj−1+1 P∆(i2, j1) for i2 ∈ [1, 2, ...,M ].

Proof: The value of the aggregated states of the SME model mapping toCSM statej at any time

t+∆t is given by:

γj(t+∆t) =

aj∑

j1=aj−1+1

M∑

i1=1

xi1 (t)P∆(i1, j1) (5)

The value of thejth state of the CSM model at any timet+∆t is given by:

τj(t+∆t) =

N∑

i2=1

τi2 (t)Pr(i2, j)

=

N∑

i2=1

τi2(t)

∑aj

j2=aj−1+1

∑ai2

i3=ai2−1+1 P∆(i3, j2)η(i3)∑ai2

i4=ai2−1+1 η(i4)

=N∑

i2=1

∑aj

j2=aj−1+1

∑ai2

i3=ai2−1+1 P∆(i3, j2)η(i3)τi2 (t)

ζ(i2)(6)

Assumingγj(t) = τj(t), dj(t +∆t) represents the difference betweenγj and τj during time stept to

t+∆t starting from equal distribution and is given by the following equation:

dj(t+∆t) = |γj(t+∆t)− τj(t+∆t)|

=

∣

∣

∣

∣

∣

∣

aj∑

j1=aj−1+1

M∑

i1=1

xi1 (t)P∆(i1, j1)−N∑

i2=1

∑aj

j2=aj−1+1

∑ai2

i3=ai2−1+1 P∆(i3, j2)η(i3)τi2(t)

ζ(i2)

∣

∣

∣

∣

∣

∣

=

∣

∣

∣

∣

∣

∣

aj∑

j1=aj−1+1

N∑

i1=1

ai1∑

i2=ai1−1+1

xi2 (t)P∆(i2, j1)−∑ai1

i3=ai1−1+1 P∆(i3, j1)η(i3)τi1 (t)

ζ(i1)

∣

∣

∣

∣

∣

∣

=

∣

∣

∣

∣

∣

∣

N∑

i1=1

∑

i2∈Si1

xi2 (t)ζ(i1)− τi1(t)η(i2)

ζ(i1)

∑

j1∈Sj

P∆(i2, j1))

∣

∣

∣

∣

∣

∣

(7)

whereSi = [ai−1 + 1, ..., ai]. Let q(i2, j) denote∑aj

j1=aj−1+1 P∆(i2, j1), then:

∑

i2∈Si1

xi2(t)ζ(i1)q(i2, j) ≤ γi1(t)ζ(i1) maxi2∈Si1

q(i2, j), (8)

∑

i2∈Si1

τi1(t)η(i2)q(i2, j) ≥ τi1 (t)ζ(i1) mini2∈Si1

q(i2, j) (9)

Substituting equations 8 and 9 into Eq. 7, and usingγj(t) = τj(t), the upper bound for the difference


7

equation is given by:

dj(t+∆t) ≤∣

∣

∣

∣

∣

N∑

i1=1

γi1(t)ζ(i1)maxi2∈Si1q(i2, j)− τi1(t)ζ(i1)mini2∈Si2

q(i2, j)

ζ(i1)

∣

∣

∣

∣

∣

≤N∑

i1=1

γi1(t)

(

maxi2∈Si1


q(i2, j)

)

≤ maxi1∈[1,...,N ]

(

maxi2∈Si1


q(i2, j)

)

(10)

Thus, theorem 3.1 provides an upper bound for the differenceequationdj for a single time step∆t

in terms of the row sumsq(i2, j) of entries of the state transition probability matrixP∆ . The bounds

on the deviation for multiple time steps is presented next.

3.1 Maximum Deviation for Multiple Time Steps

Theorem 3.2. The maximum deviation betweenγj(T∆t) and τj(T∆t) starting fromγj(0) = τj(0) is

given by the following equation:

dj(T∆t) ≤ β(1− µm

T

1− µm

) (11)

whereβ = maxj∈[1,··· ,N ] βj andβj = maxi1∈[1,...,N ]

(

maxi2∈Si1q(i2, j)−mini2∈Si1

q(i2, j))

is the

single time step upper bound derived in Theorem 3.1 andµm = maxj∈[1,··· ,N ](∑N

i1=1 |R(i1, j) − ρ|)whereρ is any constant andR(i1, j) = mini2∈[ai1−1+1,...,ai1 ]

q(i2, j).

Proof: Equation 4 bounds the deviation for a single time step∆t assuming the distributions

(collapsed probability distribution of the SME model and the probability distribution of the CSM

model) at timet to be equal. To arrive at the deviation of the transient probability distribution at time

t + ∆t without the equality assumption at timet, let us denoteτj(t) as τj(t) = γj(t) + φj(t) where∑N

j=1 φj(t) = 0. From Equations 7, 8 and 9, we have

dj(t+∆t) ≤∣

∣

∣

∣

∣

N∑

i1=1

γi1(t)ζ(ii)maxi2∈Si1q(i2, j)− τi1(t)ζ(ii)mini2∈Si2

q(i2, j)

ζ(ii)

∣

∣

∣

∣

∣

≤∣

∣

∣

∣

∣

N∑

i1=1

γi1(t)

(

maxi2∈Si1


q(i2, j)

)

∣

∣

∣

∣

∣

+

∣

∣

∣

∣

∣

N∑

i1=1

(τi1(t)− γi1(t)) mini2∈Si1

q(i2, j)

∣

∣

∣

∣

∣

(12)

whereSi1 = [ai1−1 + 1, ..., ai1 ].


8

Let us denoteR(i1, j) = mini2∈Si1q(i2, j) for i1, j ∈ S whereS = [1, ..., N ], . For any constantρ,

∑Ni1=1 ρφi1(t) = ρ× 0 = 0. Then,

dj(t+∆t) ≤ maxi1∈S

(

maxi2∈Si1


q(i2, j)

)

+ |N∑

i1=1

(R(i1, j)− ρ)φi1(t)|

≤ maxi1∈S

(

maxi2∈Si1


q(i2, j)

)

+ (

N∑

i1=1

|R(i1, j)− ρ|)maxi1∈S

|φi1(t)| (13)

If we denote the summation∑N

i1=1 |R(i1, j) − ρ| by µ(j), and maxj∈S µ(j) by µm and Φ(t) =

maxi1∈S |φi1 (t)|, then the deviation for statej at t+∆t is upper bounded byβj + µmΦt whereβj is

the maximum deviation in one time step starting from the samedistribution. Furthermore,Φt denotes

the maximum deviation among all the states a timet and henceΦt ≤ β + µmΦt−∆t. Since at time

t = 0, Φ0 = 0 as τj(0) = γj(0) for j ∈ [1, ..., N ], the difference afterT∆t time can be bounded by

dj(T∆t) ≤ βj + µmΦ(T−1)∆t

≤ βj + µm(β + µmΦ(T−2)∆t)

≤ β(1 + µm + µ2m · · ·+ µT−1

m ) + µTmΦ0

≤ β(1− µm

T

1− µm

) (14)

4 STOCHASTIC BOUNDS

The calculation of the upper bound shown in Eq. 4 is specific toeach individualP∆ matrix and is not

straightforward. It involves generation of the entries of theP∆ matrix which has enormous computational

complexity for large M. Furthermore, our purpose is to use a coarse model when data is limited and

requirement on the knowledge of the fine scale probability transition matrix to upper bound our deviation

will defeat our purpose. Thus we would be interested in arriving at an estimate of the bound by exploiting

possible structures in the row sumsq(i, j) without the inference of the individual entries of theP∆

matrix. We will consider derivation of stochastic bounds assuming various distributions of the entries

of the transition probability matrix. If the structure of our SME model provides an estimate on the

distribution of the entries ofP∆, the stochastic bounds will provide us the maximum expecteddeviation.

To estimate the bound in Eq. 4, we need to estimate the range ofthe sum ofLj consecutive entries of

a row of a transition probability matrix whereLj representsaj − aj−1 for j ∈ [1, · · · , N ].

To approach this problem, we will first derive the distribution of the sum ofLj consecutive entries of

a row of a transition probability matrix based on the assumption of uniform distribution and distribution

centered around different means, for the entries ofP∆. Since the aggregation is over a large number of

states (i.e.Lj is large), the distribution of the sum ofLj entries (i.e. distribution ofq(i, j) ) will tend


9

towards a Gaussian distribution based on Central Limit Theorem. Even though, we consider only two

cases of uniform distribution and distribution centered around different means, for largeLj , we can get

an idea of the distribution ofq(i, j) based on the knowledge of the mean and variance of the distribution

of individual entries. Once, we derive the distribution ofq(i, j) in section 4.1, we calculate the bounds

on the expectation of the range ofLj random variables selected from the derived distribution ofq(i, j) in

section 4.2. This provides an estimate for the expected value of |maxi2∈Si1q(i2, j)−mini2∈Si1

q(i2, j)|which is the primary term in the bound derived in Eq. 4.

4.1 Distribution of the sum of L entries of a row of the probability transition matrix

We will consider two cases: firstly, we will assume that all the entries of the probability transition matrix

are independent and identically distributed belonging to auniform distribution. In the second case, we

will assume that the value of the diagonal entries ofP∆ are much higher than the other entries and the

non-diagonal entries are uniform, independent and identically distributed.

4.1.1 Uniform Distribution

If we consider generating a row of a probability transition matrix by selectingM random entries from

a uniform distribution of range[0, 1] and dividing each entry by the sum of theM selected entries, then

the expected value of each entry will be1/M and we can approximate the distribution of an entry in

the probability transition matrix by a uniform distribution with range[0, 2/M ]. The expectation of the

random variableZ is E(Z) = 1M

and varianceσ2 = 13M2 .

Let us consider the summation ofL entries of a row of a probability transition matrix where each

entry has distributionfZ . Then the random variableX = Z1+Z2+ · · ·+ZL has expectationLµ = L/M

and varianceLσ2 = L3M2 assumingZi’s are independent fori = 1, · · · , L.

Based on central limit theorem [19], for sufficiently largeL, the distribution ofX will approach

N ( LM, L3M2 ) ( Henceforth,N (µ, σ2) will denote a Normal distribution with meanµ and varianceσ2).

We should note that when we are reducing from a fine scale modelto a coarse scale model, the value

of L which represents the number of states of the finer model mapping to a single state of the coarser

model is usually large.

4.1.2 Distribution centered around different means

Let us consider the case whereP∆ = IM + C where IM is the M × M identity matrix and the

individual entries ofC are assumed to be small and bounded by|C(i, j)| ≤ ε for i, j ∈ [1, 2, · · · ,M ],

i 6= j and Mε/2 << 1. The mean of the distribution of the summation∑aj

j1=aj−1+1 P∆(i1, j1) for

i1 ∈ [ai−1 + 1, · · · , ai] will be centered around1 − (M − L) ε2 when i = j otherwise centered around

Lε/2 assuming a uniform distribution between0 and ε for P∆(i, j) for i 6= j. The variance of the


10

distribution of the summation∑aj

j1=aj−1+1 P∆(i1, j1) for i1 ∈ [ai−1 + 1, · · · , ai] is Lε2/12 for i 6= j.

For i = j,∑aj

j1=aj−1+1 P∆(i1, j1) = 1−∑

j2∈S2P∆(i1, j2) whereS2 = [1, 2, ...aj−1, aj +1, ...M ] and

thus the variance is(M − L)ε2/12 assuming theM − L to be independent. Therefore, distribution of

the summation∑aj

j1=aj−1+1 P∆(i1, j1) for i1 ∈ [ai−1 + 1, · · · , ai] can be approximated by

N (Lε2 , Lε2

12 ) if i 6= j

N (2−(M−L)ε2 , (M−L)ε2

12 ) if i = j

4.2 Bounds on the Expectation of the range of L random variables

To estimate the bound in Eq. 4, we need to estimate the range ofthe row sums of aL×L block of the

probability transition matrixP∆.

Let us considerL identically distributed random variablesX1, X2, · · ·XL with a common cumulative

distribution functionF , probability distribution functionf , meanµ and varianceσ2. The derivation in

section 4.1 is used to generateF, f, µ andσ. Let ML andmL denotes the maximum and minimum of

X1, X2, · · ·XL respectively. Based on the results of [20], we can arrive at the following theorem:

Theorem 4.1. The upper bound on the expected range is given by the follwingequation:

E(ML)− E(mL) ≤ rL − r1 + L

∫ ∞

rL

[1− F (x)]dx + L

∫ r1

−∞F (x)dx (15)

whererL = F−1(1− 1L) and r1 = F−1( 1

L).

Proof:

For any real numberr, we can show that [20]:

ML ≤ r +

L∑

i=1

(Xi − r)+. (16)

where(x)+ = x for x > 0 and (x)+ = 0 for x ≤ 0. Taking expectations on both sides, we have

E(ML) ≤ r +

L∑

i=1

E((Xi − r)+). (17)

We know that,

E((Xi − r)+) =

∫ ∞

r

(x − r)f(x)dx =

∫ ∞

r

[1− F (x)]dx (18)

Thus,

E(ML) ≤ r + L

∫ ∞

r

[1− F (x)]dx. (19)

To find the minimum of the functionh(r) = r + L∫∞r

[1− F (x)]dx, we equate

dh(r)

dr= 1− L[1− F (rmin)] = 0

=⇒ F (rmin) = 1− 1

L


11

Thush(r) is minimized atr = rL where

rL = F−1(1 − 1

L) (20)

andF−1(t) = inf(x|F (x) ≥ t) for 0 ≤ t ≤ 1. Thus,

E(ML) ≤ rL + L

∫ ∞

rL

[1− F (x)]dx. (21)

Similarly for the minimum, we have

E(mL) ≥ r1 − L

∫ r1

−∞F (x)dx. (22)

wherer1 = F−1( 1L).

Thus

E(ML)− E(mL) ≤ rL − r1 + L

∫ ∞

rL

[1− F (x)]dx + L

∫ r1

−∞F (x)dx (23)

4.3 Expected bound for single time step

Theorem 4.2. The expected bound on the deviationβj for single time step is given by the following

equation:

E(βj) < 2σ(2 logL− log logL)12 (24)

whereσ denotes the standard deviation for the distribution ofq(i, j) andL = maxi=1,··· ,N |ai− ai−1|.

Proof: As discussed in section 4.1, the distribution ofq(i, j) can be approximated by a Gaussian

distribution. For distributions such as Gaussian that are symmetric around the meanµ, we haveF (µ−x)+F (µ+x) = 1. Thus,r1 = F−1( 1

L) = 2µ−F−1(1− 1

L) = 2µ− rL and equation 23 can be written

as

E(ML)− E(mL) ≤ 2rL − 2µ+ 2L

∫ ∞

rL

[1− F (x)]dx (25)

The upper bound onrL + L∫∞rL

[1− F (x)]dx for standard normal distribution is given by [20]:

rL + L

∫ ∞

rL

[1− F (x)]dx < (2 logL− log logL)12 (26)

Thus for normal distribution with meanµ and standard deviationσ

E(βj) = E(ML)− E(mL) < 2σ(2 logL− log logL)12 (27)

As an example of the bound, let us consider the case of selection of transition probabilities from

uniform distribution as considered in section 4.1.1. We have σ = 1M

√

L3 , thus


12

E(ML)− E(mL) < 21

M

√

L

3(2 logL− log logL)

12 (28)

Numerically, forM = 3200 andL = 100 we haveE(ML)− E(mL) < 0.01.

For the case of unequal means in section 4.1.2, the maximum ofE(ML)−E(mL) will be achieved

for i = j and is given by:

E(ML)− E(mL) < 2

√

(M − L)ε2

12(2 logL− log logL)

12 (29)

4.4 Expected bound for multiple time steps

We will approximate the bound for multiple time steps by plugging in the expected values forβj andµm

in equation 11. The bound on the expected deviationE(βj) is given by theorem 4.2. In this subsection,

we will consider generating a stochastic upper-bound forµm = maxj∈S(∑N

i=1 |R(i, j) − ρ|). Let us

consider the case that the diagonal entries are much bigger than the other entries which is mostly true

for small ∆t. Since, eachR(i, j) is summation of L entries where L is large, we can approximatethe

distribution ofR(i, j) by a Gaussian distribution. Let the distribution ofR(i, j) for i 6= j be approximated

by N (ξ1, σ21) and the distribution ofR(i, j) for i = j be approximated byN (ξ2, σ

22).

The following theorem provides an upper bound on the expected value ofµm.

Theorem 4.3.

E(µm) ≤ (N − 1)

√

2

πσ1 + (ξ2 − ξ1)erf(

ξ2 − ξ1√2σ2

) +

√

2

πσ2e

−(ξ2−ξ1) (30)

whereerf(x) = 2√π

∫ x

0 e−t2 dt denotes the error function.

Proof: We haveµm = maxj∈[1,··· ,N ](∑N

i1=1 |R(i1, j) − ρ|) whereρ is any constant.R(i, j) has

distributionN (ξ1, σ21) for i 6= j and has distributionN (ξ2, σ

22) for i = j . For the following proof, we

will select ρ = ξ1 and assumeσ2 > σ1.

For i 6= j, we have

E(|R(i, j)− ρ|) =

∫ 1

0

|x− ρ| 1√2πσ1

e− (x−ξ1)2

2σ21 dx

≤∫ ∞

−∞|x− ρ| 1√

2πσ1

e− (x−ξ1)2

2σ21 dx

= 2

∫ ∞

0

u1√2πσ1

e− u2

2σ21 du

=

√

2

πσ1 (31)


13

Similarly for i = j , we have

E(|R(i, j)− ρ|) =

∫ 1

0

|x− ρ| 1√2πσ2

e− (x−ξ2)2

2σ22 dx

≤∫ ∞

−∞|x− ρ| 1√

2πσ2

e− (x−ξ2)2

2σ22 dx

≤∫ ∞

−∞|u+ ξ2 − ξ1|

1√2πσ2

e− u2

2σ22 du

≤∫ ξ1−ξ2

−∞−(u+ ξ2 − ξ1)

1√2πσ2

e− u2

2σ22 du +

∫ +∞

ξ1−ξ2

(u+ ξ2 − ξ1)1√2πσ2

e− u2

2σ22 du

≤ 2

∫ ξ2−ξ1

0

(ξ2 − ξ1)1√2πσ2

e− u2

2σ22 du + 2

∫ +∞

ξ2−ξ1

u1√2πσ2

e− u2

2σ22 du

≤ (ξ2 − ξ1)erf(ξ2 − ξ1√

2σ2

) +

√

2

πσ2e

−(ξ2−ξ1) (32)

Thus, combining the (N-1) cases ofi 6= j and single case ofi = j over the∑N

i=1 |R(i, j)− ρ|, we have

E(N∑

i=1

|R(i, j)− ρ|) ≤ (N − 1)

√

2

πσ1 + (ξ2 − ξ1)erf(

ξ2 − ξ1√2σ2

) +

√

2

πσ2e

−(ξ2−ξ1)

(33)

Since, we are assuming the same distribution for allj’s,

E(µm) ≤ (N − 1)

√

2

πσ1 + (ξ2 − ξ1)erf(

ξ2 − ξ1√2σ2

) +

√

2

πσ2e

−(ξ2−ξ1)

(34)

5 EFFECT ON THE TRANSIENT BEHAVIOR DURING APPLICATION OF STATION-

ARY CONTROL POLICY

5.1 Background

The motivation behind application of control theory is to devise optimal policies for manipulating

control variables that affect the transition probabilities of the network and can, therefore, be used to

desirably affect its dynamic evolution. In practice, intervention can be achieved by (a)targeted small

molecule kinase inhibitors (Imatinib, Gefitinib, Erlotinib, Sorafenib, Sunitinib etc.[21] ) (b) Monoclonal

antibodies altering the protein concentrations (Cetuximab, Alemtuzumab, Trastuzumab etc.[21]) or (c)

gene knockdowns[22] . The state desirability is determined by the values of genes/proteins associated

with phenotypes of interest. In this section, we will consider infinite-horizon control with the goal of

favorably altering the steady-state distribution of the network via a stationary control policy [11].

To explain infinite-horizon intervention in CSM, we next provide a brief mathematical description of

the control problem. A CSM with control can be modeled as a finite state Markov Chain [23], [11]


14

described by the control-dependent one-step transition probability pij(u) := P (zt+1 = j|zt = i, ut = u)

where, for allt, the statezt is an element of a spaceS and the control inputut is an element of a space

C. The states make transitions according toω := (Pu)u∈C . In this case, once a control input is chosen,

the resulting controlled transition probability matrix isuniquely determined.

Let µ = (u1, u2, ....) represent a generic control policy andΠ represent the set of all possibleµ’s,

i.e., the set of all possible control policies. LetJµ,ω denote the expected total cost for the discounted

cost infinite-horizon problem [11] under control policyµ and transitionsω:

Jµ,ω(z0) = limM→∞

Ezt+1{M−1∑

t=0

αtg(zt, ut, zt+1)}, (35)

where 0 < α < 1 denotes the discount factor andg(zt, ut, zt+1) represents the cost of going from

statezt to zt+1 under the control actionut. g is higher for undesirable destination states. For the same

destination states,g is higher when the control is active versus when it is not.

The control problem here corresponds to minimizing the costin Eq. 35. Consequently, the optimal

infinite-horizon discounted cost is given by:

Φ(Π, ω, z0) := minµ∈Π

Jµ,ω(z0). (36)

The interest in infinite-horizon control is based on the consideration that such policies allow us to

optimally alter the steady-state probability distribution of the network by driving its probability mass

into desirable states. Furthermore, they result in stationary control policies (i.e. independent of time and

dependent on state) which are easier to implement.

5.2 Transient Analysis

To analyze the effect on the transient dynamics of the stationary control policy designed on the reduced

order model when applied to the detailed SME model, we will consider a case of two control actions at

each time step. In reality, the two control actions will refer to whether to use a drug or not to apply the

drug. We will represent the SME models corresponding to the two control actions asP∆1 andP∆2 . As

mentioned,P∆1 might represent the model of the network with no drug delivery andP∆2 represent the

model of the network following a drug delivery. Let,Pr1 andPr2 represent the transition probability

matrices of the reduced order CSM models generated from models P∆1 andP∆2 respectively using Eq.

2.

Since, we will design the control policy based on the reducedorder model, our control design will be

based onPr1 andPr2 . Let the goal of our control action be to alter the steady state probability distribution

of the network and we design a stationary control policy using dynamic programming approaches based

on Pr1 andPr2 by minimizing Eq. 36. The steady state distribution of a GRN can reflect the phenotype

and thus alteration of it will be one of our primary control goals. However, we would also be interested


15

in the state probability distribution during the transientphase so as to reduce the probabilities of being

in a highly undesirable state during the transient phase of control policy application. The application

of the designed control policy on the reduced order model canprovide an estimate of the transient

dynamics during application of the control policy. In this section, we are interested in deriving the

maximum deviation in the transient phase of the aggregated state probability distribution of the SME

model from the state probability distribution of the reduced order model after application of the control

policy designed on the reduced order model.

A stationary control policy for the case of 2 control actionsfor the CSM will be a binary vector of

lengthN where0 at locationi denotes no control action when at statei (the model of the network for

statei will then be based onPr1 ) whereas a1 at locationi denotes control action when at statei (the

model of the network for statei will then be based onPr2 ). To illustrate it further, let us consider a

simple example from [13] whereN = 2 ,

Pr1 =

∣

∣

∣

∣

∣

∣

0.2 0.8

0.65 0.35

∣

∣

∣

∣

∣

∣

Pr2 =

∣

∣

∣

∣

∣

∣

0.45 0.55

0.25 0.75

∣

∣

∣

∣

∣

∣

and stationary control policy =[0 1]. Let Prd denote the controlled CSM transition probability matrix

for a specific stationary control policy. Thus for the stationary control policy of [0 1],Prd will have

its first row fromPr1 and second row fromPr2 and the controlled modelPrd can be represented as

Prd = TPr1 + (IN − T )Pr2 whereIN represent the identity matrix of sizeN ×N andT is aN ×N

matrix with all entries zero except the diagonal entries corresponding to no control action being equal

to 1. ThePrd andT matrices for stationary control policy[0 1] is as follows:

Prd =

∣

∣

∣

∣

∣

∣

0.2 0.8

0.25 0.75

∣

∣

∣

∣

∣

∣

T =

∣

∣

∣

∣

∣

∣

1 0

0 0

∣

∣

∣

∣

∣

∣

When a control policy designed using the CSM is applied to thefine-scale stochastic network model,

the control policy corresponding to statesai−1 + 1, · · · , ai of the SME model will be the same as the

control policy for statei of the CSM. Thus, the controlled fine scale model will beP∆c= T∆P∆1 +

(IM − T∆)P∆2 whereT∆ is a M × M matrix of all zeros except the states that map to CSM states

with no control action, equal to1. For the example before, if we considerM = 4 and the states1 and

2 of the fine-scale model map to state1 of the CSM, then

T∆ =

∣

∣

∣

∣

∣

∣

∣

∣

∣

∣

∣

∣

1 0 0 0

0 1 0 0

0 0 0 0

0 0 0 0

∣

∣

∣

∣

∣

∣

∣

∣

∣

∣

∣

∣

.

For the controlled SME modelP∆c, let ηc represent theM dimensional steady state probability

vector andζc represent theN dimensional collapsed steady state probability vector. Here, collapsing


16

refers to aggregation of probability of states i.e.ζc(i) =∑ai

i2=ai−1+1 ηc(i2) for i = 1, ..., N . Let xc(t) =

[xc1(t), x

c2(t), ..., x

cM (t)] denote the controlled SME model’s probability distribution at any timet. Let

γcj (t) denote the collapsed probability of statesaj−1 +1 to aj of the controlled SME model at any time

t. Let τcj (t) denote the probability of statej at any timet for the controlled CSM modelPrd . We are

interested in finding the difference between the collapsed probability of the states in the controlled SME

model that map to statej of the controlled CSM model and the probability of statej in the controlled

CSM model at time instantt+∆t.

Let us define the difference equation of interest as

dcj(t+∆t) = |γcj (t+∆t)− τcj (t+∆t)|. (37)

The following theorem provides an upper-bound on the deviation dcj(t + ∆t) starting fromγcj (t) =

τcj (t).

Theorem 5.1.

dcj(t+∆t) ≤ maxi1∈[1,...,N ]

(

maxi2∈Si1

qu(i1)(i2, j)− mini2∈Si1

qu(i1)(i2, j)

)

(38)

whereSi1 = [ai1−1 + 1, ..., ai1 ] and

u(i) =

1 if T (i, i) = 1

2 if T (i, i) = 0

Proof: The value of the collapsed states of the SME model mapping to CSM statej at any time

t+∆t is given by:

γcj (t+∆t) =

aj∑

j1=aj−1+1

M∑

i1=1

xci1(t)P∆c

(i1, j1) (39)

The transition probabilities of controlled CSM modelPrd can be represented as follows:

Prd(i, j) =

∑aj

j1=aj−1+1

∑ai

i1=ai−1+1 P∆u(i)(i1, j1)ηu(i)(i1)

∑ai

i2=ai−1+1 ηu(i)(i2)

=

∑aj

j1=aj−1+1

∑ai

i1=ai−1+1 P∆u(i)(i1, j1)ηu(i)(i1)

ζu(i)(i)(40)

Thus, the value of thejth state of the controlled CSM model at any timet+∆t is given by:

τcj (t+∆t) =

N∑

i2=1

τci2 (t)Prd(i2, j)

=N∑

i2=1

τci2(t)

∑aj

j2=aj−1+1

∑ai2

i3=ai2−1+1 P∆u(i2)(i3, j2)ηu(i2)(i3)

∑ai2

i4=ai2−1+1 ηu(i2)(i4)

=

N∑

i2=1

∑aj

j2=aj−1+1

∑ai2

i3=ai2−1+1 P∆u(i2)(i3, j2)ηu(i2)(i3)τ

ci2(t)

ζu(i2)(i2)(41)


17

If we consider the caseγcj (t) = τcj (t), d

cj(t+∆t) represents the difference betweenγc

j andτcj during

time stept to t+∆t starting from equal distribution at timet and is given by the following equation:

dcj(t+∆t)

= |γcj (t+∆t)− τcj (t+∆t)|

=

∣

∣

∣

∣

∣

∣

aj∑

j1=aj−1+1

M∑

i1=1

xci1(t)P∆c

(i1, j1)−N∑

i2=1

∑aj

j2=aj−1+1

∑ai2

i3=ai2−1+1 P∆u(i2)(i3, j2)ηu(i2)(i3)τ

ci2(t)

ζu(i2)(i2)

∣

∣

∣

∣

∣

∣

=

∣

∣

∣

∣

∣

∣

aj∑

j1=aj−1+1

N∑

i1=1

ai1∑

i2=ai1−1+1

xci2(t)P∆u(i1)

(i2, j1)−∑ai1

i3=ai1−1+1 P∆u(i1)(i3, j1)ηu(i1)(i3)τ

ci1(t)

ζu(i1)(i1)

∣

∣

∣

∣

∣

∣

=

∣

∣

∣

∣

∣

∣

N∑

i1=1

ai1∑

i2=ai1−1+1

xci2(t)

aj∑

j1=aj−1+1

P∆u(i1)(i2, j1)−

ηu(i1)(i2)τci1(t)

ζu(i1)(i1)

aj∑

j1=aj−1+1

P∆u(i1)(i2, j1)

∣

∣

∣

∣

∣

∣

=

∣

∣

∣

∣

∣

∣

N∑

i1=1

∑

i2∈Si1

xci2(t)ζu(i1)(i1)− τci1 (t)ηu(i1)(i2)

ζu(i1)(i1)

∑

j1∈Sj

P∆u(i1)(i2, j1))

∣

∣

∣

∣

∣

∣

(42)

whereSi = [ai−1 + 1, ..., ai]. If we let∑aj

j1=aj−1+1 P∆u(i1)(i2, j1) = qu(i1)(i2, j), we know that:

∑

i2∈Si1

xci2(t)ζu(i1)(ii)qu(i1)(i2, j) ≤ γc

i1(t)ζu(i1)(ii) max

i2∈Si1

qu(i1)(i2, j), (43)

∑

i2∈Si1

τci1 (t)ηu(i1)(i2)qu(i1)(i2, j) ≥ τci1(t)ζu(i1)(ii) mini2∈Si1

qu(i1)(i2, j) (44)

The upper bound for the controlled difference equation is then given by:

dcj(t+∆t) ≤∣

∣

∣

∣

∣

N∑

i1=1

γci1(t)ζu(i1)(i1)maxi2∈Si1

qu(i1)(i2, j)− τci1 (t)ζu(i1)(i1)mini2∈Si2qu(i1)(i2, j)

ζ(ii)

∣

∣

∣

∣

∣

≤N∑

i1=1

γci1(t)

(

maxi2∈Si1


qu(i1)(i2, j)

)

≤ maxi1∈[1,...,N ]

(

maxi2∈Si1


qu(i1)(i2, j)

)

(45)

Based on the results of theorem 5.1, we note that the single step deviation of the controlled system

is upper-bounded by the maximum of the single-step bounds ofthe individual SME models. Similarly,

the multi-step upper bound of the controlled system is the maximum of the multi-step bounds of the

individual systems.

6 RESULTS

In this section, we apply our analysis to randomly generatedSME models (section 6.1) , oscillating SME

models (section 6.2) and biological model of Genetic ToggleSwitch [24] (section 6.3).


18

6.1 Randomly Generated Models

In this sub-section, we present the simulation results of the shift in transient behavior of the SME model

and the reduced CSM model based on various random SME models.We generated random transition

probability matrices to represent the SME models and using the reduction mapping in Eq. 2 generated

the reduced CSM model. At first, we consider the cases of transition probability matrix for which the

diagonal entry is larger than all the non-diagonal entries.Let d ∈ [0, 1] represent the average value of the

diagonal entry. The non-diagonal entries of a row are selected from a uniform distributionU(0, 2(1−d)M

).

The diagonal entry is selected as1 − ∑

(non-diagonal entries). The generation of the diagonal entry

in this fashion guarantees that the variance of the diagonalentry is (M − 1) times the variance of each

individual non-diagonal entry. In actual biological transition probability matrices, the diagonal entry

represents the probability of remaining in the same state and that is equal to1 − ∑

( probabilities

of leaving the state). The results of the simulation are shown in Table 1 whereM and N represent

the number of states of the SME and CSM model respectively andd represents the average value of

the diagonal entry. We usedT = 100 as the number of time steps by which the system has reached

the steady state. For each set of parameters we created100 random transition probability matrices and

calculated the expected deviations and bounds. For the actual deviation calculations, the initial probability

distribution is taken to be[1, 0, 0, · · · , 0]. In Table 1,Ds represents the mean of the maximum difference

between the collapsed SME state probabilities and the CSM state probabilities in a single time step i.e.

Ds = E(maxi∈[1,...,N ],t∈[1,..T ] ||τj(t) − γj(t)| − |τj(t − 1) − γj(t − 1)||). Dm represents the mean of

the maximum difference between the collapsed SME state probabilities and the CSM state probabilities

in multiple time steps i.e.Ds = E(maxi∈[1,...,N ],t∈[0,1,..T ] |τj(t) − γj(t)|). Bs represents the mean of

the single step bounds calculated from the transition probability matrices using the bound in equation 4.

The mean is with respect to the100 randomly generated transition probability matrices.Bm represents

the mean of the multi-step bounds calculated from the transition probability matrices using the bound in

equation 11.SBs represents the mean of the single step stochastic bounds calculated using equation 29.

The ε used in Eq. 29 is2(1−d)/(M −1). SBm represents the mean of the multi-step stochastic bounds

calculated using equations 29, 11 and 30. For Eq. 30, theσ1 andσ2 used are2(1− d)/(M − 1)√

L/12

and2(1−d)/(M−1)√

(M − L)/12 respectively. The means of the distributionsξ1 andξ2 are(1−d)/N

and(1− d)/N + d respectively. The results of Table 1 shows that the generated bounds are quite close

to the actual maximum deviations. The bound for single time step Bs is quite close toDs, the actual

maximum deviation in a single time step. Similarly,Bm is of the same order asDm. Furthermore,

we note that the stochastic boundSBs is very close to the expectation of the boundBs, similarly the

stochastic bound for multiple stepSBm is very similar to the actual bound for multiple stepBm. This

shows that we can use the stochastic bounds for generating the deviation bounds when the distribution of


19

the entries of the transition probability matrix is known. The stochastic bounds can be calculated without

generating the actual transition probability matrices andhas extremely low computational complexity

involved in its calculation.

M N d Ds Dm Bs Bm SBs SBm

400 10 .5 0.035 0.036 0.059 0.119 0.068 0.147

400 4 .5 0.033 0.034 0.063 0.125 0.069 0.146

200 10 .5 0.049 0.049 0 .072 0.149 .086 0.194

200 10 .1 0.084 0.084 0.131 0.150 0.155 0.196

400 10 .1 0.060 0.060 0.107 0.120 0.122 0.148

400 10 .8 0.013 0.026 0.024 0.119 0.027 0.146

400 4 .8 0.014 0.027 0.025 0.124 0.028 0.145

800 10 .8 0.011 0.022 0.0185 0.094 0.021 0.111

TABLE 1

Transient response bounds starting from random SME models with diagonal entries greater than

non-diagonal entries

Table 2 shows the simulation results for transition probability matrices with all entries belonging to

a uniform distributionU(0, 2M). As before,M andN represent the number of states of the SME and

CSM model respectively. We usedT = 100 as the number of time steps to reach the steady state. For

each set of parameters we created100 random transition probability matrices and calculated theaverage

deviations and bounds. For the actual deviation calculations, the initial probability distribution is taken

to be [1, 0, 0, · · · , 0]. Ds represents the mean of the maximum difference between the collapsed SME

state probabilities and the CSM state probabilities in a single time step.Dm represents the mean of

the maximum difference between the collapsed SME state probabilities and the CSM state probabilities

in multiple time steps.Bs represents the mean of the single step bounds calculated from the transition

probability matrices using the bound in equation 4.Bm represents the mean of the multi-step bounds

calculated from the transition probability matrices usingthe bound in equation 11.SBs represents the

mean of the single step stochastic bounds calculated using equation 28.SBm represents the mean of

the multi-step stochastic bounds calculated using equations 28, 11 and 30. For Eq. 30, the standard

deviationsσ2 = σ1 = 2M

√

L12 and the means of the distributions areξ2 = ξ1 = 1/N .

From Table 2, we note that the proposed bounds are close to theactual deviations. The entries in

columnBs andBm are close to the entries of columnsDs andDm respectively. We also note that for

transition probability matrices with all entries selectedfrom same uniform distribution, the single step

deviation is quite close to multi-step deviation. The stochastic bounds indicated in columnsSBs and

SBm are very close to the actual boundsBs andBm and these stochastic bounds can be calculated in


20

M N Ds Dm Bs Bm SBs SBm

200 4 0.047 0.047 0.092 0.095 0.104 0.111

400 4 0.044 0.044 0.072 0.074 0.080 0.084

500 5 0.035 0.035 0.059 0.061 0.064 0.067

600 6 0.029 0.029 0.051 0.052 0.053 0.056

800 8 0.022 0.022 0.040 0.041 0.040 0.042

1000 4 0.023 0.023 0.049 0.050 0.056 0.057

TABLE 2

Transient response bounds starting from random SME models with uniform transition

probabilities

a computationally inexpensive manner. Thus for uniform distribution of entries of probability transition

matrix, the stochastic bounds provide a very good approximation of the upper bounds.

6.2 Oscillating SME models

In this subsection, we consider models with periodic behavior. We consider a two protein systemP1

andP2 and truncate the maximum number of protein molecules forP1 andP2 to 19 resulting in a total

number of20× 20 = 400 possible states for the SME. The CSM model contains2× 2 = 4 states and

the threshold used for binarization is10. To generate a SME model with oscillation, we considered a

transition probability matrix structure as shown in Fig 1(a) where the states in setS2i−1 andS2i of the

SME model map to statei of the CSM model fori = {1, 2, 3, 4}. Each setSi contain the same number

of 400/8 = 50 states. The blue squares contain non-zero transition probabilities and all blank squares

contain zeros. The non-zero transition probabilities are selected from a uniform random generator of

range[0, 1] and finally normalized to make the sum of rows equal to 1. For the figure 1, we note that

the state transitions can be grouped into the following transitions between the set of statesS1 to S8 :

S1 → S4 → S5 → S3 → S6 → S4

S1 → S8 → S4 → S5 → S3 → S6 → S4

S2 → S3 → S6 → S4 → S5 → S3

S2 → S7 → S4 → S5 → S3 → S6 → S4

The oscillating behavior of generated protein molecules isevident from Fig 1(b) that shows the

corresponding expected number of protein molecules forP1 and P2 vs time. As the SME transition

probability matrix is not ergodic in this example, we added asmall perturbation to calculate the steady

state probability to generate the reduced order CSM model using Eq 2. In this example, we had the sum

of transition probabilities from a state inS1 to any state inS4 or a state inS2 to any state inS3 fixed

at 0.5. The bound generated using Eq. 4 is zero as for alli1 and j ∈ [1, 2, 3, 4], maxi2∈Si1q(i2, j) =


21

0 2 4 6 8 10 12 14 16 18 208

8.5

9

9.5

10

10.5

11

time −>

Ava

rage

exp

ress

ion

Protein 1Protein 2

Fig. 1. (a) Transition Probability matrix description for 1st oscillating model (b) Expected protein

molecule generation vs time

mini2∈Si1q(i2, j). Thus a value of zero for the bound assures us that the transient probabilities of the

CSM and the collapsed SME model should match exactly. The time evolution of the CSM and the

collapsed SME model for the four states starting from a uniform distribution is shown in Fig. 2. From

Fig. 2, it is clear that the transient behavior of the CSM and the collapsed SME model matches exactly.

2 4 6 8 10 12 14 16 18 200

0.05

0.1

0.15

0.2

0.25

Time−>

prob

abili

ty

CSMcollapsed SME

(a) Probability vs time for state 00

2 4 6 8 10 12 14 16 18 200.25

0.3

0.35

0.4

0.45

0.5

0.55

0.6

0.65

Time−>

prob

abili

ty

CSMcollapsed SME

(b) Probability vs time for state 01

2 4 6 8 10 12 14 16 18 200.25

0.3

0.35

0.4

0.45

0.5

0.55

0.6

0.65

Time−>

prob

abili

ty

CSMcollapsed SME

(c) Probability vs time for state 10

2 4 6 8 10 12 14 16 18 200

0.05

0.1

0.15

0.2

0.25

Time−>

prob

abili

ty

CSMcollapsed SME

(d) Probability vs time for state 11

Fig. 2. Time evolution of probability starting from a uniform distribution for an oscillating model

described by Fig 1. Red * denotes the coarse scale model evolution and blue + denotes the

evolution of the collapsed SME model.

We next consider changes such that we have transient oscillations and the transition probabilities are

arranged in a manner to arrive at a non-zero value for the bound in Eq. 4. The transition probabilities

now are similar to Fig 1(a) except that we allow new transitions from states inS6 to states inS2 and


22

the sum of transition probabilities from a state inS6 to a state inS2 is fixed at0.25. The time evolution

of the CSM and the collapsed SME model for the four states starting from a uniform distribution is

shown in 3. TheDs for this model is0.03 andDm is 0.033. Based on the configuration of the transition

probability matrix, the single step bound using equation 4 is 0.25. We note that the bound is much higher

than the actual deviations. If we analyze the derivation of the bound, we will note that there are only

few blocks that havemaxi2∈Si1q(i2, j) 6= mini2∈Si1

q(i2, j). In this example, we can use the reduced

single step bound of0.25× γ3(t) if we have an estimate ofγ3(t).

5 10 15 20 25 30 35 40 45 500

0.05

0.1

0.15

0.2

0.25

Time−>

prob

abili

ty

CSMcollapsed SME


5 10 15 20 25 30 35 40 45 500.25

0.3

0.35

0.4

0.45

0.5

0.55

0.6

0.65

Time−>

prob

abili

ty

CSMcollapsed SME


5 10 15 20 25 30 35 40 45 500.2

0.25

0.3

0.35

0.4

0.45

0.5

0.55

0.6

0.65

Time−>

prob

abili

ty

CSMcollapsed SME


5 10 15 20 25 30 35 40 45 500

0.05

0.1

0.15

0.2

0.25

Time−>

prob

abili

ty

CSMcollapsed SME


Fig. 3. Time evolution of probability starting from a uniform distribution for the modified model

with transient oscillations. Red * denotes the coarse scale model evolution and blue + denotes

the evolution of the collapsed SME model.

6.3 Genetic Toggle Switch

We next apply our transient analysis on a genetic toggle switch where there are two competing proteins

U1 andU2 , each of which inhibits the transcription of the other [24],[25]. The decay reactionsU1 → ∅andU2 → ∅ have propensitiescΨ1 andcΨ2 respectively. The transcription of new copies of the proteins

are guided by the reactions∅ → U1 and∅ → U2 with propensitiesb/(b+Ψ2) andb/(b+Ψ1) respectively.

We considerb = 0.8 andc = 0.035 as the starting system. We truncate the maximum number of protein

molecules forU1 andU2 to 39 resulting in a total number of40 × 40 = 1600 possible states for the

SME. Based on the propensity functions and parameter valuesb = 0.8 andc = 0.035, the state reaction

matrix,A1, is generated and the continuous Markov Chain is approximated by a discrete chain using the


23

following equationP∆1 = eA1∆t with a time step of∆t = 0.5 seconds. The thresholds for binarization

of the proteinsU1 andU2 are selected to beT1 = T2 = 20 molecules2. After binarization, the two

protein reduced network has2× 2 = 4 states. We calculated the transition probabilities of the reduced

model (CSMPr1) using Equation 2. In our case, the Markov ChainP∆1 is ergodic as all the states are

communicating, recurrent and aperiodic. Due to the ergodicity, P∆1 has a unique steady state probability

distribution. The time evolution of the probability of the four states starting from a uniform distribution

is shown in Fig 4.

0 500 1000 1500 20000.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Time−>

prob

abili

ty

CSMcollapsed SME


0 500 1000 1500 20000

0.05

0.1

0.15

0.2

0.25

0.3

0.35

Time−>

prob

abili

ty

CSMcollapsed SME


0 500 1000 1500 20000

0.05

0.1

0.15

0.2

0.25

0.3

0.35

Time−>

prob

abili

ty

CSMcollapsed SME


0 500 1000 1500 20000

0.05

0.1

0.15

0.2

0.25

Time−>

prob

abili

ty

CSMcollapsed SME


Fig. 4. Time evolution of probability starting from a uniform distribution for the genetic switch with

parameters b = 0.8 and c = 0.035. Red * denotes the coarse scale model evolution and blue +

denotes the evolution of the collapsed SME model.

The actual deviation characteristics of the CSM probability and the SME model are shown in Table

2. The threshold levels selected here are arbitrary but in real systems medicine application, it will be decided based onthe

modes of the distribution corresponding to cancerous and non-cancerous states.


24

3. State denotes the4 states (00, 01, 10 and11) of the reduced model.Ds andDm denotes the actual

maximum deviation in a single time step and multiple time steps respectively.SBs andSBm denotes

the stochastic bounds for single time step and multiple timesteps respectively. Theε used to calculate

the stochastic bound is2 ∗ (1 − d)/(M − 1) with d = 0.4 whered is an approximation of a diagonal

entry. We are assuming that each diagonal entry is greater than 0.4 which is reasonable as we have a

small δt of 0.25 seconds. Theµm used to calculate the multiple step bound is estimated as follows.

The transition probability matrix is given byeA∗δt whereA is the reaction matrix. We approximate the

transition probability matrix byI +Aδt using the first two terms of the series expansion. Based on this

approximation, we generated the approximate transition probability matrix and calculatedµ’s from the

approximate matrix. For our case,δt = 0.25, c = 0.035, b = 0.8 and t1 = 20, t2 = 20. To calculate

the multi-step bound, we used the maximum ofµ’s 0.8154 as theµm. From Table 3, we note that the

stochastic bound for multi-steps (0.2596) is very close to the maximum actual deviations for each state

(0.2257, 0.0895, 0.0917, 0.0784).

State Ds Dm SBs SBm

00 0.0054 0.2257 0.0479 0.2596

01 0.0027 0.0895 0.0479 0.2596

10 0.0027 0.0917 0.0479 0.2596

11 0.0016 0.0784 0.0479 0.2596

TABLE 3

Transient response bounds for the Genetic Switch Model with parameters b = 0.8 and c = 0.035

To study the effect on the transient behavior of the control policy designed on the reduced network

when applied to the original network, we consider a control problem of reducing the probability of state

00. The control input is assumed to decrease the decay of the proteins by decreasing the value of the

parameterc from 0.035 to 0.025. Thus we have two systems, the original withb = 0.8 and c = 0.035

and the other corresponding to when control is switched on with parametersb = 0.4 and c = 0.025.

The time evolution of the probability of the four states starting from a uniform distribution for the new

system is shown in Fig 5.

The actual deviation characteristics of the CSM probability and the SME model for the new system

are shown in Table 4. As before,Ds andDm denotes the actual maximum deviation in a single time

step and multiple time steps respectively;SBs andSBm denotes the stochastic bounds for single time

step and multiple time steps respectively. Theε used to calculate the stochastic bound is2(1−d)M−1 with

d = 0.4. As in the previous case, theµm is considered to be the maximum of the approximateµ’s which

is 0.8654 for b = 0.8, c = 0.025. From Table 4, we note that the expected bound for multi-steps (0.356)


25

0 500 1000 1500 20000.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Time−>

prob

abili

ty

CSMcollapsed SME


0 500 1000 1500 20000.05

0.1

0.15

0.2

0.25

0.3

Time−>

prob

abili

ty

CSMcollapsed SME


0 500 1000 1500 20000.05

0.1

0.15

0.2

0.25

0.3

Time−>

prob

abili

ty

CSMcollapsed SME


0 500 1000 1500 20000

0.05

0.1

0.15

0.2

0.25

Time−>

prob

abili

ty

CSMcollapsed SME


Fig. 5. Time evolution of probability starting from a uniform distribution for the genetic switch with

parameters b = 0.8 and c = 0.025. Red * denotes the coarse scale model evolution and blue +

denotes the evolution of the collapsed SME model.

is very close to the maximum actual deviations for each state(0.2886, 0.1242, 0.1242, 0.0756).

The optimal stationary policy to reduce the steady state probability mass for state00 is 0, 1, 1, 0 i.e.

apply no control while in states00 and 11 and apply control while in states01 and 10. The optimal

control policy is derived using dynamic programming principles and is an example of sporadic control

based on state changes. The probability evolution for the controlled model is shown in Figure 6. Without

application of control, the steady state probability of state 00 is 0.795 for both the CSM and SME models

as shown in Figure 4(a). The equality of the steady-state probabilities for CSM and collapsed SME is

because of the steady state behavior maintaining property of the mapping [13]. With application of control

at every step, the steady state probability of state00 is 0.7043 for both the CSM and SME models as

shown in Figure 5(a). After application of the stationary control 0, 1, 1, 0, the steady state probability for


26

State Ds Dm SBs SBm

00 .0039 .2886 .0479 .356

01 .0019 .1242 .0479 .356

10 .0019 .1242 .0479 .356

11 .0011 .0756 .0479 .356

TABLE 4

Transient response bounds for the Genetic Switch Model with parameters b = 0.8 and c = 0.025

state00 is 0.6586 for the coarse-scale controlled model and0.6790 for the collapsed controlled SME

model as shown in Figure 6(a). We note that we have been able toachieve our objective of lowering

the steady state probability of state00. The maximum deviations for single step and multi-steps forthe

controlled model are shown in Table 5. The expected bounds for single time step and multi-time steps

shown in Table 5 are the maximum of the bounds for the uncontrolled system (b = 0.8, c = 0.035) and

the system with control always on (b = 0.8, c = 0.025). We note that the expected bounds are a good

indicator of the maximum deviation in the transient response.

State Ds Dm SBs SBm

00 .0043 .3155 .0479 .356

01 .0024 .1451 .0479 .356

10 .0024 .1482 .0479 .356

11 .0016 .0782 .0479 .356

TABLE 5

Transient response bounds for the controlled Genetic Switch Model with stationary policy 0110

7 CONCLUSIONS

In this paper, we provided bounds for the deviation of the transient state probability distribution of a

reduced order Markovian model as compared to the collapsed state probability distribution of the fine

scale SME model. Analysis of the predictive capability of a reduced order model is highly important

in systems biology to understand the effects of limited samples and data extraction noise. We showed

by simulations that our derived analytical bounds are of thesame order and quite close to the actual

differences in state probabilities. Our transient analysis in case of stationary control policies showed that

the deviations are bounded by the maximum of the bounds for the individual systems. In this article, we

also derived stochastic bounds for the transient deviations that are easier to calculate. Our simulations


27

0 500 1000 1500 20000.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Time−>

prob

abili

ty

CSMcollapsed SME


0 500 1000 1500 20000

0.05

0.1

0.15

0.2

0.25

0.3

0.35

Time−>

prob

abili

ty

CSMcollapsed SME


0 500 1000 1500 20000

0.05

0.1

0.15

0.2

0.25

0.3

0.35

Time−>

prob

abili

ty

CSMcollapsed SME


0 500 1000 1500 20000

0.05

0.1

0.15

0.2

0.25

Time−>

prob

abili

ty

CSMcollapsed SME


Fig. 6. Time evolution of probability starting from a uniform distribution for the controlled genetic

switch with stationary control of 0 1 1 0. Red * denotes the coarse scale model evolution and blue

+ denotes the evolution of the collapsed SME model.

illustrate that the stochastic bounds are a good measure of the maximum deviation in the transient

behavior.

REFERENCES

[1] N. Wiener,Cybernetics: Or the Control and Communication in the Animaland the Machine. MIT Press, Cambridge, MA,

1948.

[2] A. Arkin, J. Ross, and H. H. McAdams, “Stochastic kineticanalysis of developmental pathway bifurcation in phage-infected

escherichia coli cells,”Genetics, vol. 149, pp. 1633–1648, 1998.

[3] H. H. McAdams and A. Arkin, “Stochastic mechanisms in gene expression,”Proc. Natl. Acad. Sci, vol. 94, pp. 814–819,

1997.

[4] D. T. Gillespie, “A rigorous derivation of the chemical master equation,”Physica A, vol. 188, pp. 404–425, 1992.

[5] J. Goutsias, “A hidden markov model for transcriptionalregulation in single cells,”IEEE/ACM Transactions on Computational

Biology and Bioinformatics, vol. 3, pp. 57–71, 2006.


28

[6] Goutsias, John, “Classical versus Stochastic KineticsModeling of Biochemical Reaction Systems,”Biophys. J., vol. 92, no. 7,

pp. 2350–2365, April 2007.

[7] C. A. Gomez-Uribe and G. C. Verghese, “Mass fluctuation kinetics: capturing stochastic effects in systems of chemical

reactions through coupled mean-variance computations.”J Chem Phys, vol. 126, no. 2, p. 024109, 2007.

[8] Lee, Chang H., Kim, Kyeong H., and Kim, Pilwon, “A moment closure method for stochastic reaction networks,”The Journal

of Chemical Physics, vol. 130, no. 13, pp. 134 107+, 2009.

[9] R. Pal and M. U. Caglar, “Control of stochastic master equation models of genetic regulatory networks by approximating

their average behavior,” inIEEE International Workshop on Genomic Signal Processing and Statistics, 2010.

[10] A. T. Weeraratna, Y. Jiang, G. Hostetter, K. Rosenblatt, P. Duray, M. Bittner, and T. J. M., “Wnt5a signalling directly affects

cell motility and invasion of metastatic melanoma,”Cancer Cell, vol. 1, pp. 279–288, 2002.

[11] R. Pal, A. Datta, and E. R. Dougherty, “Optimal infinite horizon control for probabilistic boolean networks,”IEEE

Transactions on Signal Processing, vol. 54, pp. 2375– 2387, 2006.

[12] E. Yzerman, J. Den Boer, M. Caspers, A. Almal, B. Worzel,W. Van der Meer, R. Montijn, and F. Schuren, “Comparative

genome analysis of a large Dutch Legionella pneumophila strain collection identifies five markers highly correlated with

clinical strains,”BMC Genomics, vol. 11, pp. 433+, 2010.

[13] R. Pal and S. Bhattacharya, “Characterizing the effectof coarse-scale pbn modeling on dynamics and intervention performance

of genetic regulatory networks represented by stochastic master equation models,”IEEE Transactions on Signal Processing,

vol. 58, pp. 3341–3351, 2010.

[14] B. Munsky and M. Khammash, “The finite state projection algorithm for the solution of the chemical master equation,”The

Journal of Chemical Physics, vol. 124, no. 044104, 2006.

[15] S. Kauffman,The Origins of Order: Self-Organization and Selection in Evolution. New York: Oxford Univ. Press, 1993.

[16] I. Shmulevich, E. R. Dougherty, S. Kim, and W. Zhang, “Probabilistic boolean networks: A rule-based uncertainty model

for gene regulatory networks,”Bioinformatics, vol. 18, pp. 261–274, 2002.

[17] N. Friedman, M. Linial, I. Nachman, and D. Pe’er, “Bayesian networks to analyze expression data,”Proceedings of the

Fourth Annual International Conference on Computational Molecular Biology, pp. 127–135, 2000.

[18] H. Lahdesmaki, S. Hautaniemi, I. Shmulevich, and O. Yli-Harja, “Relationships between probabilistic boolean networks and

dynamic bayesian networks as models of gene regulatory networks,” Signal Processing, vol. 86, pp. 814–834, 2006.

[19] D. P. Bertsekas and J. N. Tsitsikilis,Introduction to Probability, 2nd ed. Athena Scientific, 2008.

[20] T. L. Lai and H. Robbins, “Maximally dependent random variables,” Proc. Nat. Acad. Sci. USA, vol. 73, pp. 286–288, 1976.

[21] J. W. Clark,Harrison’s Manual of Oncology. McGraw-Hill Professional, 2007, ch. 1.10 Molecular Targeted Drugs, pp.

67–75.

[22] R. A. Science, “Integrated solutions for gene knockdown,” Biochemica, no. 4, 2004.

[23] R. Pal, A. Datta, M. L. Bittner, and E. R. Dougherty, “Intervention in context-sensitive probabilistic boolean networks,”

Bioinformatics, vol. 21, pp. 1211–1218, 2005.

[24] M. Hegland, C. Burden, L. Santoso, S. MacNamara, and H. Booth, “A solver for the stochastic master equation applied to

gene regulatory networks,”Journal of Computational and Applied Mathematics, vol. 205, no. 2, pp. 708 – 724, 2007.

[25] T. Jahnke and W. Huisinga, “A dynamical low-rank approach to the chemical master equation,”Bull. Math. Biol., vol. 70,

pp. 2283–2302, 2008.


29

AUTHOR BIOS

Ranadip Pal received the B. Tech. degree in Electronics and ECE from the Indian Institute of Technology,

Kharagpur, India in 2002 and the M.S. and PhD degrees in Electrical Engineering from Texas A & M

University, College Station in 2004 and 2007 respectively.In August 2007, he joined the Department of

Electrical and Computer Engineering at Texas Tech University, Lubbock as an Assistant Professor. His

research areas are computational biology, genomic signal processing and control of genetic regulatory

networks. Pal is the recipient of NSF CAREER Award, 2010. While at Texas A&M, Pal was the recipient

of the Ebensberger/Fouraker Fellowship, a Distinguished Graduate Student Masters Research Award and

a National Instruments Fellowship. Pal was also an Indian National Math Olympiad Awardee.

Sonal Bhattacharya received the B.Sc. and M.Sc. degrees in physics from Calcutta University, Calcutta,

India, in 1998 and 2001, respectively, and the post M.S. diploma in physics from Saha Institute of

Nuclear Physics, Calcutta, in 2002. She is currently pursuing the Ph.D. degree in Electrical and Computer

Engineering at Texas Tech University, Lubbock, TX. Her research areas are genomic signal processing,

control of genetic regulatory networks, image processing,solid-state physics, and fluid mechanics.


Documents

1 Transient dynamics of reduced order models of genetic regulatory networks · 2011-12-20 · 1 Transient dynamics of reduced order models of genetic regulatory networks Ranadip Pal,