Geometric Methods in Optimal Control Theory Applied to Problems

Geometric Methods

in Optimal Control Theory Applied

to Problems in Biomedicine

Urszula Ledzewicz

Southern Illinois University

Edwardsville, USA

Miedzy Teoria a

Zastosowaniami-

Matematyka w Dzialaniu,

Bedlewo,16-22 czerwca 2013

Co-author and Support Heinz Schättler

Washington University St. Louis, MO USA

Research supported by collaborative research

NSF grants DMS 0405827/0405848

DMS 0707404/0707410

DMS 1008209/1008221

• Heinz Schättler and Urszula Ledzewicz,

Geometric Optimal Control – Theory, Methods, Examples

Springer Verlag, July 2012

• Urszula Ledzewicz and Heinz Schättler,

Geometric Optimal Control Applied to Biomedical Models

Springer Verlag, 2014

• Mathematical Methods and Models in Biomedicine

Urszula Ledzewicz, Heinz Schättler, Avner Friedman and Eugene Kashdan, Eds.

Springer Verlag, October 2012

Books

Main Collaborators

Alberto d’Onofrio

European Institute for Oncology, Milano, Italy

Helmut Maurer

Rheinisch Westfälische Wilhelms-Universität Münster, Münster, Germany

Andrzej Swierniak

Silesian University of Technology, Gliwice, Poland

Recent Collaborators

Eddy Pasquier

Children Cancer Institute Australia,

University of New South Wales, Sydney

Yangjin Kim

University of Michigan Dearborn, MI USA

Graduate Student Collaborators

John Marriott

University of Hawaii

Mohammad Naghnaeian, Mostafa Reisi

University of Illinois at Urbana-Champaign

Mozdeh Sadat Faraji

Georgia Tech

Sia Mahmoudian Dehkordi

North Carolina State University

Omeiza Olumoye

Southern Illinois University Edwardsville

Optimal Control Problems

dynamics

(model)

min or max

objective

control

response

disturbance

(unmodelled dynamics)

• a model for chemotherapy for heterogeneous tumors

• a model for antiangiogenic therapy

(alone and in combination with chemotherapy)

• a model for chemotherapy and immune boost

• future directions metronomic chemotherapy

Outline – An Optimal Control Approach to …

• a model for growth and invasion in glioblastoma

Optimal Drug Treatment Protocols

Main Questions

HOW MUCH? (dosage)

HOW OFTEN? (timing)

IN WHAT ORDER? (sequencing)

A Model for Growth and

Invasion in Glioblastoma

• a particularly aggressive form of

brain cancer characterized by

alternating phases of rapid growth

and tissue invasion with a mean

survival time of just about one

year

• treatment: surgery

• problem: distant tumor satellites

• normal glucose levels up-regulate the microRNA miR-451

level leading to cell proliferation and decreased cell

migration

• low glucose levels induce a down-regulation of miR-451,

which, in turn, promotes cell motility and invasion,

but inhibits proliferation.

Glioblastoma

Dynamics of the miR-451 AMPK Complex

[Kim, Roh, Lawler and Friedman, PLoS One, (2011)

M

A M – concentration of miR-451

A – concentration of AMPK complex

G – glucose level

S – source of AMPK complex

miR 451 - micro RNA’s regulate the expression levels

of genes

AMPK – adenosine monophosphate-activated protein

kinase

an enzyme that plays a role in cellular

energy homeostasis including glucose uptake

Dynamics of the miR-451 AMPK Complex

[Kim and Roh, Discr. and Cont. Dyn. Syst., (2013)

α – inhibition of miR-451 by AMPK (Hill-type, scaled)

β - inhibition of AMPK by miR-451 (Hill-type, scaled)

κ1 - autocatalytic strength of miR-451 (scaled)

κ3 - autocatalytic strength of AMPK (scaled)

ε - scaling factor related to the degradation rates of miR-451 and AMPK,

non-dimensionalized,

minimally parameterized

version of the model

A degrades much faster than M (half-life of miR 451 is in

the range of 100-200 hours, for AMPK about 6 hours)

Differential-Algebraic Model

low glucose level

high glucose

level

multi-stability for

intermediate

glucose level

blue curve = slow manifold

red curve = equilibrium manifold

G constant

Hysteresis for a constant glucose level, G=const, the

following hysteresis picture for the equilibria

and their stability arises

growth

(over-expression)

invasion

(under-expression)

cycle for differential

algebraic model

Goal

• maintain the miR-451 level M above a threshold Mth so that

glioma cells remain in the proliferation phase and do not

switch into their invasive migratory phase

• effective control: the level of glucose

we allow for both bolus injections or continuous infusions and

do not limit the control variable in its size

• use as little glucose as possible in order to limit the cancer growth

minimize the overall amount of glucose given

Optimal Control Problem

[M] for a fixed terminal time T, minimize the objective

over all times , bolus dosages , and all

Lebesgue measurable functions

subject to the dynamics

and state-space constraint for all

State-Space Constraint

• the state-space constraint is of order 2:

• this makes transitions with the constraint difficult

(possibly chattering, … )

• consider a modified problem with an order 1 state-space constraint

Optimal Control Problem II

[G] for a fixed terminal time T, minimize the objective

over all times , bolus dosages , and all

Lebesgue measurable functions

subject to the dynamics

and state-space constraint for all

Solution for [G]

• for problem [M] to be well-posed, we assume that

• according to the fast dynamics for A we also assume that

Theorem [SchKimLed, 52nd, CDC 2013]

For these initial conditions the solution to the

optimal control problem [G] is given by

administering an initial bolus dose G1 of glucose

that brings the system to the threshold level Gth

at time 0 (if it lies below) followed by constant

infusion at rate u* ≡ λGth. This control maintains

the level of M above its lower threshold Mth.

continuous administration does better than spaced boli

Periodic Bolus Injections

Heterogeneous Tumor

Cell Populations

Mathematical Model

[Hahnfeldt, Folkman and Hlatky, JTB, 2003]

for simplicity, just consider two populations of different chemotherapeutic

sensitivity and call them ‘sensitive’ and ‘resistant’

S – sensitive cell population

R – resistant cell population

α1 – growth rate of sensitive population

α2 – growth rate of resistant population

γ1 – transfer rate from sensitive to resistant population

γ2 – transfer rate from resistant to sensitive population

φ1 – linear log-kill parameter for sensitive population

φ2 – linear log-kill parameter for resistant population

β – pharmacokinetic parameter related to half-life of chemotherapeutic agent

S

R

c

N=(S,R)

Mathematical Model: Objective

minimize the number of cancer cells N = (S,R)

left without causing too much harm to the healthy cells

Weighted average

of number of

cancer cells at

end of therapy

Weighted average

of cancer cells

during therapy

Toxicity of the

drug

(side effects on

healthy cells)

For a fixed therapy horizon minimize

over all functions subject to the dynamics

where

As Optimal Control Problem

(LSch, JBS, 2013, submitted)

Candidates for Optimal Protocols

• bang-bang controls

• singular controls

treatment protocols of

maximum dose therapy

periods with rest periods

in between

continuous infusions of

varying lower doses

umax

T T

MTD BOD

Φ(t) > 0

Φ(t) > 0 Φ(t) > 0 Φ(t) ≡ 0

switching function Φ(t)

Singular Controls

• is singular on an open interval

switching function on

• all time derivatives must vanish as well

• “allows” to compute the singular control

• order : the control appears for the first time in

the derivative

• Legendre-Clebsch condition (minimize)

• in the region BB,

the Legendre-Clebsch condition is violated and optimal controls are bang-

bang; in particular, this holds if

•outside the region BB,

the Legendre-Clebsch condition is satisfied, but singular controls are of

order 2 and concatenations with bang controls are through chattering arcs

singular controls become a viable option as the sensitive cells

get depleted through chemotherapy and the resistant population

becomes dominant

Bang-bang vs. Singular Solutions

Numerical Simulations

Numerical Simulations

Numerical Simulations

Numerical Simulations

0 20 40 60 80 100 120 140 1600

1

2

3

4

5

6

7

8

9

10x 10

4

time

popu

latio

ns R

and

S

Tumor stimulating myeloid cell

Surveillance T-cell

Fibroblast

Endothelia Chemo-resistant tumor cell

Chemo-sensitive tumor cell

Tumor Microenvironment – Other Treatments

Tumor Antiangiogenesis

avascular

growth angiogenesis

metastasis

http://www.gene.com/gene/research/focusareas/oncology/angiogenesis.html

Tumor Anti-Angiogenesis

• suppress tumor growth by

preventing the recruitment of new

blood vessels that supply the

tumor with nutrients

indirect approach

• done by inhibiting the growth of

the endothelial cells that form the

lining of the new blood vessels

therapy “resistant to resistance”

Judah Folkman, 1972

• anti-angiogenic agents are

biological drugs (enzyme inhibitors

like endostatin) – very expensive

and with side effects

Model [Hahnfeldt,Panigrahy,Folkman,

Hlatky],Cancer Research, 1999

p,q – volumes in mm3

Lewis lung

carcinoma

implanted

in mice

- tumor growth parameter

- endogenous stimulation (birth)

- endogenous inhibition (death)

- anti-angiogenic inhibition parameter

- natural death

p – tumor volume

q – carrying capacity

u – anti-angiogenic

dose rate

For a free terminal time minimize

over all functions that satisfy

subject to the dynamics

Optimal Control Problem

Singular Control

0 0.5 1 1.5 2 2.5 3-20

0

20

40

60

80

100

120

x

psi

feedback control

0 0.5 1 1.5 2 2.5 3 3.5

x 104

0

2000

4000

6000

8000

10000

12000

14000

16000

18000

carrying capacity of the vasculature, q

tum

or

volu

me,p

Admissible Singular Arc

q

p

Synthesis of Optimal Controls

[LSch, SICON, 2007]

0 2000 4000 6000 8000 10000 12000 14000 16000 180000

2000

4000

6000

8000

10000

12000

14000

16000

18000

endothelial cells

tum

or c

ells

an optimal trajectory begin of

therapy

final point – minimum of p

end of “therapy”

p

q

u=a

u=0

typical synthesis: umax→s→0

Some Practical Aspects

An Optimal Controlled Trajectory

for [Hahnfeldt et al.]

Initial condition: p0 = 12,000 q0 = 15,000, umax=75

0 1 2 3 4 5 6 7

0

10

20

30

40

50

60

70

time

opti

mal

con

tro

l u

maximum dose rate

no dose

lower dose rate - singular

averaged optimal dose

u

q0

robust with respect to q0

Minimum Tumor Volumes under Suboptimal

Constant Dose Protocols [LSch, JTB, 2008]

Values of the minimum tumor volume for a fixed initial tumor

volume as functions of the initial endothelial support

full dose

optimal control

averaged optimal dose

half dose

pmin

q0

Response to Two Dose Protocols,

[LMMSch, Math. Med. &Biology, 2010]

Optimal Daily Dosages

Combination Therapy: Antiangiogenic

Treatment with Chemotherapy

Minimize subject to

A Model for a Combination Therapy

[d’OLMSch, Mathematical Biosciences, 2009]

with d’Onofrio and H. Maurer

angiogenic inhibitors

cytotoxic agent or other killing term

Optimal Protocols: what comes first?

4000 6000 8000 10000 12000 14000 16000

7000

8000

9000

10000

11000

12000

13000

carrying capacity of the vasculature, q

tum

or

volu

me,

p

optimal

angiogenic

monotherapy

Controls and Trajectory [for dynamics from Hahnfeldt et al.]

0 1 2 3 4 5 6 7

0

10

20

30

40

50

60

70

time (in days)

dosag

e a

ng

io

4000 6000 8000 10000 12000 14000 16000

7000

8000

9000

10000

11000

12000

13000

carrying capacity of the vasculature, q

tum

or

vo

lum

e,

p

0 1 2 3 4 5 6 7-0.2

0

0.2

0.4

0.6

0.8

1

dosag

e c

he

mo

time (in days)

Medical Aspect

Rakesh Jain,

Steele Lab, Harvard Medical School,

“there exists a therapeutic window

when changes in the tumor in response to

anti-angiogenic treatment may allow

chemotherapy to be particularly effective”

Connection between mathematical results

and experimental data ??

Tumor Immune Interactions

• Stepanova, Biophysics, 1980

Kuznetsov, Makalkin, Taylor and Perelson,

Bull. Math. Biology, 1994

de Vladar and Gonzalez, J. Theo. Biology, 2004,

d’Onofrio, Physica D, 2005

A Model for Tumor-Immune Interactions

renewed interest in the topic also in connection with

immune-dynamics and immuno-therapy

Stepanova-Type Mathematical Models

for Tumor-Immune Dynamics

• STATE:

- primary tumor volume

- immunocompetent cell-density

(related to various types of T-cells)

- tumor growth parameter - growth function

- rate at which cancer cells are eliminated through the activity of T-cells

- constant rate of influx of T-cells generated by primary organs

- natural death of T-cells

- calibrate the interactions between immune system and tumor

- threshold beyond which immune reaction becomes suppressed

by the tumor

Dynamical Model

[Stepanova]

Growth Models on the Tumor Volume x

exponential growth

logistic growth

Gompertzian

Stepanova, 1980

Kuznetsov et al., 1994

de Vladar and

Gonzalez, 2004

d’Onofrio, 2005

L Sch Olumoye, 2013

F is positive, twice

continuously differentiable

Phaseportrait for Gompertzian Model

asymptotically stable

focus – “good”,

benign equilibrium

[Kuznetsov et al., 1994

de Vladar et al., 2004]

saddle point

asymptotically stable

node – “bad”,

malignant equilibrium

multiple stable equilibrium points

Phaseportrait of uncontrolled dynamics

• we want to move the state of the system into the region of

attraction of the benign equilibrium

minimize

For a free terminal time T minimize

over all measurable functions and

subject to the dynamics

Optimal Control Problem (LSch, CDC 2012)

Chemotherapy – log-kill hypothesis

Immune boost

Immunotherapy only

malignant region persists

Immunotherapy alone

is not successful in

this region

Chemotherapy is needed 0 100 200 300 400 500 600 700 800 900 1000

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

x

y

Phase Portrait

Dynamics revisited Write the system as

with

drift vector field

Lie bracket

control vector fields

g1 and g2 commute:

Legendre-Clebsch Condition

for u (Chemotherapy)

The Legendre-Clebsch condition is satisfied if and only if

singular controls are locally optimal

Suppose the control v is singular on an interval I . For optimality

we need that

But in this case

singular controls are maximizing, so not

optimal

the control v, i.e., immune boost, should

be bang-bang

Legendre-Clebsch Condition for Control v

(Immunotherapy)

0 200 400 600 8000

0.5

1

1.5

2

2.5

3

0 2 4 6 8 10 12

0

0.2

0.4

0.6

0.8

1

Chemotherapy with Immune Boost [DCDSB, 2013]

• trajectory follows the optimal chemo monotherapy and provides final boosts

to the immune system and chemo at the end

• “cost” of immune boost is high and effects are low compared to chemo

* *

*

“free pass”

1s01 010

- chemo

- immune boost

* *

Metronomics and Other

Alternatives to MTD

with Eddy Pasquier, CCIA, University of New South Wales

The frequent administration of chemotherapy drugs at relatively low, non-toxic doses, without prolonged drug-free breaks (Hanahan et al., JCI 2000)

METRONOMICS =

Metronomic Chemotherapy + Drug Repositioning

Metronomic Chemotherapy

Adapted from Pasquier et al., Nature Reviews Clinical Oncology, 2010

Metronomic Chemotherapy: modeling challenge

• treatment at lower doses

( between 10% and 80% of the MTD)

• constant or not?

How is it administered?

Advantages (to be modelled):

1. lower, but continuous cytotoxic effects on tumor cells

• lower toxicity (in many cases, none)

• lower drug resistance and even resensitization effect

2. antiangiogenic effects

3. boost to the immune system

How to optimize the anti-tumor, anti-angiogenic and pro-immune

effects of chemotherapy by modulating dose and administration

schedule?

Different therapeutic approaches:

- “Pure” metronomic / Metronomics

J Clin Oncol 2010

-Weekly VLB -Daily CPA -2x weekly MTX -Daily CLX

How to optimize the anti-tumour, anti-angiogenic and pro-immune

effects of chemotherapy by modulating dose and administration

schedule?

Different therapeutic approaches:

- “Pure” metronomic / Metronomics

- MTD / Metronomic sequencing

(Bang-Bang-Metro, Metro-Bang-Bang…)

J Clin Oncol 2005

Lancet Oncol 2010

How to optimize the anti-tumour, anti-angiogenic and pro-immune

effects of chemotherapy by modulating dose and administration

schedule?

Different therapeutic approaches:

- “Pure” metronomic / Metronomics

- MTD / Metronomic sequencing (Bang-Bang-

Metro, Metro-Bang-Bang…)

- Adaptive therapy

Cancer Research 2009

Nature 2009

Nature 2009

How to optimize the anti-tumour, anti-angiogenic and pro-immune

effects of chemotherapy by modulating dose and administration

schedule?

Different therapeutic approaches:

- “Pure” metronomic / Metronomics

- MTD / Metronomic sequencing (Bang-Bang-Metro,

Metro-Bang-Bang…)

- Adaptive therapy

- Chaos therapy

Metronomics Global Health Initiative (MGHI)

http://metronomics.newethicalbusiness.org/

Nicolas Andre

Children’s Hospital of

Timone, France

Giannoula Klement

Tufts University School of

Medicine, Boston, USA

Eddy Pasquier

Children Cancer Institute

Australia Sydney, Australia