Planning for Gene Regulatory Network Intervention

Daniel BryceArizona State University

Seungchan KimArizona State University &

Translational Genomics Research Institute

1/10/07 Bryce & Kim -- IJCAI-07 2

Prior Work Planning for Finding Pathways

S. Khan, K. Decker, W. Gillis, and C. Schmidt. “A multi-agent system-driven AI planning approach to biological pathway discovery.” In Proceedings of ICAPS’03, 2003.

Fifth International Planning Competition, 2006. Reasoning about change in cellular processes

N. Tran and C. Baral. “Issues in reasoning about interaction networks in cells: necessity of event ordering knowledge.” In Proceedings of AAAI’05, 2005.

Extracting and Expressing Transition Functions from Micro-array experiments, Markov chain analysis.

S. Kim, H. Li, E. Dougherty, N. Cao, Y. Chen, M. Bittner, and E. Suh. “Can Markov chain models mimic biological regulation?” Journal of Biological Systems, 10(4):337–357, 2002.I. Shmulevich, E. Dougherty, S. Kim, and W. Zhang.”Probabilistic boolean networks: a rule-based uncertainty model for gene regulatory networks.” Bioinformatics 18(2):261–274, 2002.

Non-AI work on planning interventions.A. Datta, A. Choudhary, M. Bittner, and E. Dougherty. “External control in Markovian genetic regulatory networks: the imperfect information case.” Bioinformatics, 20(6):924–930, 2004.

1/10/07 Bryce & Kim -- IJCAI-07 3

Gene Regulatory Networks (GRNs)

g1 g2

g3 g4

Gene Correlations

g1 g2 g3 g4

g1’ g2’ g3’ g4’

Dynamics Model

From: [Wuensche, PSB-98]

Cell Type(Phenotype, e.g., liver cell)

Tissue

Micro arrayData

Questions of interest: How does cancer occur? How can we prevent cancer? How do we kill specific cells? Can we control Differentiation?

e.g., Program stem cell to become Liver Cell

Can we change Phenotype? e.g., Revert liver cell to back to

stem cell, then differentiate to heart cell

1/10/07 Bryce & Kim -- IJCAI-07 4

Gene Regulatory Network Behavior

Steady States (normal)

Transient States(intermediate)

UndesirableState

Edge Thickness == Pr(s | s’)

CancerPhenotype

Extra cellular signals can effect the cell state transitions (e.g., Chemotherapy, Pharmaceuticals, and Stress)

Partial Observations of molecular components or physiology are available

1/10/07 Bryce & Kim -- IJCAI-07 5

GRN Intervention Planning

Datta et. al. Assumptions Synchronous Events Exact Representation Optimal Bounded Length

Plans Datta et. al. Approach

Enumerate Reachable Belief States

Dynamic Programming Our Approach

AI Planning AO* Search

Non-Intervention

Observation

Intervention

Observation

1/10/07 Bryce & Kim -- IJCAI-07 6

Evaluation

WNT5A GRN Highly active WNT5A

indicates proliferation of cancer

2 (non)interventions 2 variations: direct and

indirect control 2 observations 7 genes (binary valued)

Randomly Generated GRN 4 (non)interventions 2 observations 7 genes (binary valued)

Compare AI Planning with Datta et. al. Scaling horizon Sensitivity to

Reward Function Metric: Total Time

1/10/07 Bryce & Kim -- IJCAI-07 7

WNT5A GRN (from TGEN dataset)

Indirect Control Intevene Pirin gene Observe WNT5A gene

Direct Control Intervene WNT5A

gene Observe Pirin gene

Total Time

0

200

400

600

800

1000

1200

2 4 6 8 10 12

Horizon

Tim

e(s)

AO*

Datta

Total Time

0

1000

2000

2 4 6 8 10 12 14 16Horizon

Tim

e(s

)

AO*

Datta

1/10/07 Bryce & Kim -- IJCAI-07 8

Total Time

0

1000

2000

2 4 6 8 10 12

Horizon

Tim

e(s

)

-10-1110Datta

Random GRN (4 acts)

GoalReward(AO*)

Enumeration

AO* exploitsReward

Function forPruning

(ImprovedScalabilityIn Some Cases)

1/10/07 Bryce & Kim -- IJCAI-07 9

Assumptions Revisited

Finite Horizon Not all treatments require same length

Synchronous Change Actions overloaded to include GRN change

7 Genes and 1 intervention Within human comprehension

1/10/07 Bryce & Kim -- IJCAI-07 10

Indefinite or Finite Horizon?

Indefinite Horizon: If goal state is a steady state, then no need to plan more actions to meet a given horizon

1/10/07 Bryce & Kim -- IJCAI-07 11

Asynchronous Change

Decouple Intervention from Gene Regulatory Network Simulation Triggers (Tran and Baral, AAAI’05) Probabilistic Exogenous Events (Blythe,

UAI’94)

1/10/07 Bryce & Kim -- IJCAI-07 12

Larger GRNs

50-5000 genes More Interventions and Observations Representation:

ADD for transition relation blows up DBN is better, but exact inference can be costly Extensions of Thrun’s MC-POMDP’s, sample based

representation, is in the right direction

Search Heuristics: McLUG: Planning Graphs with Probabilistic Actions

1/10/07 Bryce & Kim -- IJCAI-07 13

Conclusion

Off-the-shelf AI planning improves upon state of the art in Intervention Problems

Future Research Needed: Scaling

Indefinite Horizon Extra Actions and Observations Sample-based Representation Search Heuristics

Modeling Asynchronous Probabilistic Change

Plan Explanation

1/10/07 Bryce & Kim -- IJCAI-07 14

Extra Slides

1/10/07 Bryce & Kim -- IJCAI-07 15

Empirical Comparison

Total Time and Expanded Nodes

Better in all Cases

With no heuristicsSearch performance

Correlates with RewardFunction

Datta Enumeration

AO*

1/10/07 Bryce & Kim -- IJCAI-07 18

The Network

1/10/07 Bryce & Kim -- IJCAI-07 19

The Parameters and Functions

1/10/07 Bryce & Kim -- IJCAI-07 20

Computational Biology

Bioinformatics Knowledge Discovery & Data-mining Manage and Analyze Biological Data

Systems Biology Simulation Model Dynamic Systems

1/10/07 Bryce & Kim -- IJCAI-07 21

Representing State Distributions

2.

2.

35.

25.

21

2

_

1

21

_2

_

1

_

gg

gg

gg

gg

Explicit Vectorg1

g2 g2

.2 .25 .35

Algebraic Decision Diagram

1/10/07 Bryce & Kim -- IJCAI-07 22

Representing State Distributions

2.

2.

35.

25.

21

2

_

1

21

_2

_

1

_

gg

gg

gg

gg

Explicit Vectorg1

g2

.2 .25 .35

Algebraic Decision Diagram

1/10/07 Bryce & Kim -- IJCAI-07 23

Representing Probabilistic Actions

0001

08.2.0

3.06.1.

05.3.2.

21

2

_

1

21

_2

_

1

_

2'1'2'_

1'2'1

_'2

_'1'

_

gg

gg

gg

gg

gggggggg g1

g’1 g’1

g’2 g’2 g’2 g’2

g2 g2 g2 g2

g’2 g’2 g’2 g’2

0 .1 .2 .3 .5 .6 .8 1

Explicit Transition Matrix Algebraic Decision Diagram

1/10/07 Bryce & Kim -- IJCAI-07 24

Representing Probabilistic Actions

g1

g’1 g’1

g’2 g’2 g’2 g’2

g2 g2 g2 g2

g’2 g’2 g’2

0 .1 .2 .3 .5 .6 .8 1

0001

08.2.0

3.06.1.

05.3.2.

21

2

_

1

21

_2

_

1

_

2'1'2'_

1'2'1

_'2

_'1'

_

gg

gg

gg

gg

gggggggg

Explicit Transition Matrix Algebraic Decision Diagram

1/10/07 Bryce & Kim -- IJCAI-07 25

Modeling Network Dynamics

?g

?g1 ?g2 ?g3 ?g4 0 1

(- (influence1 ?g1 ?g2 ?g) (noise))

(- (influence2 ?g3 ?g4 ?g) (noise))

(noise) (noise)

?g1 ?g2 ?g(not (up-regulated ?g1)) (not (up-regulated ?g2)) (pred-fn ?g1 ?g2 ?g zz)(not (up-regulated ?g1)) (up-regulated ?g2) (pred-fn ?g1 ?g2 ?g zo) (up-regulated ?g1) (not (up-regulated ?g2)) (pred-fn ?g1 ?g2 ?g oz) (up-regulated ?g1) (up-regulated ?g2) (pred-fn ?g1 ?g2 ?g oo)

?g1 ?g2 ?g(not (up-regulated ?g1)) (not (up-regulated ?g2)) (pred-fn ?g1 ?g2 ?g zz)(not (up-regulated ?g1)) (up-regulated ?g2) (pred-fn ?g1 ?g2 ?g zo) (up-regulated ?g1) (not (up-regulated ?g2)) (pred-fn ?g1 ?g2 ?g oz) (up-regulated ?g1) (up-regulated ?g2) (pred-fn ?g1 ?g2 ?g oo)

(predicts ?g1 ?g2 ?g) (predicts ?g3 ?g4 ?g)

1/10/07 Bryce & Kim -- IJCAI-07 26

Network Dynamics Encoding <dynamics>(forall (?g ?g1 ?g2 ?g3 ?g4 - gene) ;;constraint for grounding that binds only those genes ?g1 - ?g4 that ;;predict ?g. External control actions add predicates to the ;;antecedent below so that ?g does not bind to controlled genes. (when (and (predicts1 ?g1 ?g2 ?g) (predicts2 ?g3 ?g4 ?g)) (probabilistic (- (influence1 ?g1 ?g2 ?g) (noise)) ;;predictor 1 probability (and (when (or ;;conditions to set ?g up (and (not (up-regulated ?g1)) (not (up-regulated ?g2)) (pred-fn ?g1 ?g2 ?g zz)) (and (not (up-regulated ?g1)) (up-regulated ?g2) (pred-fn ?g1 ?g2 ?g zo)) (and (up-regulated ?g1) (not (up-regulated ?g2)) (pred-fn ?g1 ?g2 ?g oz)) (and (up-regulated ?g1) (up-regulated ?g2) (pred-fn ?g1 ?g2 ?g oo)) ) (up-regulated ?g)) ;;set ?g up (when (or ;;conditions to set ?g down (and (not (up-regulated ?g1)) (not (up-regulated ?g2)) (not (pred-fn ?g1 ?g2 ?g zz))) (and (not (up-regulated ?g1)) (up-regulated ?g2) (not (pred-fn ?g1 ?g2 ?g zo))) (and (up-regulated ?g1) (not (up-regulated ?g2)) (not (pred-fn ?g1 ?g2 ?g oz))) (and (up-regulated ?g1) (up-regulated ?g2) (not (pred-fn ?g1 ?g2 ?g oo))) ) (not (up-regulated ?g))) ;;set ?g down ) (- (influence2 ?g3 ?g4 ?g) (noise)) ;;predictor 2 probability (and [...]) ;;predictor 2, similar to predictor 1 (noise) (up-regulated ?g) ;;noise to set ?g up (noise) (not (up-regulated ?g)) ;;noise to set ?g down ) ))

conditions to up-regulate

with predictor1

Conditions to down-regulate

up-regulatewith predictor1

down-regulatewith predictor1

probabilityOf using

predictor1

Binding constraints

Rules for predictor2

and noise

Bind all genes to variables

1/10/07 Bryce & Kim -- IJCAI-07 27

Network Parameters

s100p

wnt5a ret2 stc2 ret2 0 1

.68 .30 .01 .01

?g1 ?g2 ?g(not (up-regulated ?g1)) (not (up-regulated ?g2)) (pred-fn ?g1 ?g2 ?g zz)(not (up-regulated ?g1)) (up-regulated ?g2) (pred-fn ?g1 ?g2 ?g zo) (up-regulated ?g1) (not (up-regulated ?g2)) (pred-fn ?g1 ?g2 ?g oz) (up-regulated ?g1) (up-regulated ?g2) (pred-fn ?g1 ?g2 ?g oo)

(predicts wnt5a ret2 s100p) (predicts stc2 ret2 s100p)

wnt5a ret2 s100p(not (up-regulated wnt5a)) (not (up-regulated ret2)) (not (up-regulated s100p))(not (up-regulated wnt5a)) (up-regulated ret2) (up-regulated s100p) (up-regulated wnt5a) (not (up-regulated ret2)) (up-regulated s100p) (up-regulated wnt5a) (up-regulated ret2) (up-regulated s100p)

1/10/07 Bryce & Kim -- IJCAI-07 28

Predictor Encoding

(= (noise) .01)

;s100p predictor1(predicts1 wnt5a ret2 s100p)

(pred-fn wnt5a ret2 s100p zo) ;1 (pred-fn wnt5a ret2 s100p oz) ;1 (pred-fn wnt5a ret2 s100p oo) ;1

(= (influence1 wnt5a ret2 s100p) .69)

;s100p predictor2(predicts2 stc2 ret2 s100p)

(pred-fn stc2 ret2 s100p oz) ;1 (pred-fn stc2 ret2 s100p oo) ;1

(= (influence2 stc2 ret2 s100p) .31)

stc2 ret2 s100p(not (up-regulated stc2)) (not (up-regulated ret2)) (not (up-regulated s100p))(not (up-regulated stc2)) (up-regulated ret2) (not (up-regulated s100p)) (up-regulated stc2) (not (up-regulated ret2)) (up-regulated s100p) (up-regulated stc2) (up-regulated ret2) (up-regulated s100p)

wnt5a ret2 s100p(not (up-regulated wnt5a)) (not (up-regulated ret2)) (not (up-regulated s100p))(not (up-regulated wnt5a)) (up-regulated ret2) (up-regulated s100p) (up-regulated wnt5a) (not (up-regulated ret2)) (up-regulated s100p) (up-regulated wnt5a) (up-regulated ret2) (up-regulated s100p)

1/10/07 Bryce & Kim -- IJCAI-07 29

Control Encoding <control>, perfect/partial obeservation

(:action down-regulate :parameters (?gr ?go - gene) :precondition (and (observed ?go)

(controlled ?gr) (started)) :effect (and (decrease (reward) 1)

(when (up-regulated ?gr) (not (up-regulated ?gr))) <dynamics + (not (= ?g ?gr))> ) :observation (

((up-regulated ?go) (up-regulated ?go) 1) ((not (up-regulated ?go)) (not (up-regulated ?go)) 1)

))

(when (up-regulated ?gr) (probabilistic .75 (not (up-regulated ?gr))))

Could be better model!?