Embed Size (px)
Citation preview
Predictive Control Based Therapy of Bone Marrow Cancer
Hugo Miguel Grancho Silva
Thesis to obtain the Master of Science Degree in
Electrical and Computer Engineering
Supervisor: Professor João Manuel Lage de Miranda Lemos
Examination Committee
Chairperson: Professor João Fernando Cardoso Silva SequeiraSupervisor: Professor João Manuel Lage de Miranda Lemos
Member of the Committee: Professor Susana de Almeida Mendes Vinga Martins
September 2015
ii
Without mathematics, there’s nothing you can do.
Everything around you is mathematics.
Everything around you are numbers.
Shakuntala Devi
iii
iv
Acknowledgments
I would like to start this section by thanking to my supervisor professor Joao Miranda Lemos. His wide
knowledge on different topics, his sympathy and huge availability were crucial to the success of this work.
Then thanking to all the people who make part of my life, specially those I met during my course
and with whom I shared adventures, concerns and emotions during the last five years. I want to thank
Diogo Santos for his valuable help in reviewing this thesis, in the discussion of themes about everything,
and for his friendship. Also, I would like to do a special thank to Claudia Barrocas, for all the friendship,
understanding, and infinite patience.
Finally, the most important thank is to my family: my godmother Dulce, my grandparents Lurdes,
Fatima and Jose and especially to my parents Luısa and Vıtor, who have always supported me.
To the memory of my grandfather Francisco, with love.
The work in this dissertation was performed within the framework of project CancerSys, Multiscale
modeling for personalized therapy of bone methastasis, financed by FCT under contract EXPL/EMS-
SIS/1954/2013.
v
vi
Resumo
O osso e um complexo tecido, que e remodelado durante a vida. A remodelacao e um processo que esta
a cargo de dois tipos de celulas: os osteoclastos, que destroem e removem o osso, e os osteoblastos,
que o reconstroem. O microambiente do osso e rico em nutrientes, tornando-se um local habitual para
o surgimento de tumores. Estes interferem na relacao entre os osteoclastos e osteoblastos, acelerando
o processo de remocao e reduzindo a densidade de massa ossea, criando assim condicoes para se
metastizarem.
Este trabalho comeca por rever modelos de crescimento de tumores. A relacao entre os farmacos
e o corpo humano bem como o fenomeno de resistencia ao farmaco sao modelados, para que a ter-
apia seja a mais real possıvel. O metodo de controlo predictivo com base em modelos nao-lineares e
utilizado, para determinar a dose optima de farmaco a administrar ao paciente. E usada uma referencia
exponencial para diminuir o erro, e desenvolvida uma terapia de multiplos farmacos para obter difer-
entes efeitos e e usado o metodo dos mınimos quadrados recursivos para estimar os parametros do
modelo do crescimento do tumor, obtendo assim um controlador adaptativo.
Por fim, a ferramenta desenvolvida e usada num trabalho recente sobre modelacao matematica do
cancro da medula ossea, para reduzir o tumor. A densidade ossea e recuperada, usando um contro-
lador PID discreto. As simulacoes em MATLAB mostram que o tumor e eliminado em dois anos e a
densidade ossea e recuperada em cinco anos, na hipotese de os modelos serem validos.
Palavras-chave: Controlo preditivo nao-linear, Modelos nao-lineares, Modelos de farma-
cologia, Controlo adaptativo, Cancro da medula ossea, Sistemas biomedicos.
vii
viii
Abstract
Bone is a complex tissue that is remodelled continuously. Bone remodelling is a delicate process done
by two types of cells: osteoclasts, that digest and remove old bone, and osteoblasts, that create new
bone. The bone microenvironment is rich in nutrients, hence it is not surprising that is an usual place for
tumor appearance. From the moment in which a tumor is established, it starts to deregulate the balance
between osteoclasts and osteoblasts, accelerating the bone resorption and decreasing the bone mass
density, creating conditions to metastasize.
This work starts by reviewing the mathematical models for tumor growth. The relation between drugs
and the human body as well as the drug resistance phenomena are modelled, so that the therapy can
be as close as possible to reality. The Nonlinear Model Predictive Control algorithm (NMPC) is used
to find the optimal drug dose, to reduce the tumor density. An exponential reference signal is used in
NMPC to produce smaller tracking errors, a multi-drug therapy is developed to perform diferente effects,
and the recursive least squares method is used to learn the parameters of the tumor growth model, to
obtain an adaptive NMPC strategy.
The framework is applied to a state-of-the-art bone microenvironment model used in cancer re-
search, to schedule a therapy for reducing tumor density. The bone mass density will be recovered with
a discrete PID algorithm. Simulations in MATLAB show that the tumor is eliminated in two years and the
bone mass is recovered in five years, assuming model validity.
Keywords: Nonlinear Model Predictive Control, Nonlinear models, Pharmacodynamical mod-
els, Adaptive control, Bone marrow cancer, Biomedical systems.
ix
x
Contents
Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . v
Resumo . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii
Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix
List of Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xv
List of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xviii
Nomenclature . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xx
Glossary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxi
1 Introduction 1
1.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Literature review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.3 Objectives and main contribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.4 Thesis outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
2 Tumor Growth Models 7
2.1 Exponential-linear model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2.2 Logistic model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2.3 Gompertz model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2.4 Gompertz model with input control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2.5 Model discretization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2.5.1 1st order Euler method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2.5.2 2nd order Runge Kutta methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2.5.3 4th order Runge Kutta method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2.5.4 Methods comparison . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2.5.5 4th order Runge Kutta method for a system of equations . . . . . . . . . . . . . . . 14
2.6 Discrete Gompertz model - summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
3 Pharmacodynamical Models 19
3.1 Pharmacokinetics model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
3.1.1 PK controller . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
3.2 Pharmacodynamics model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
3.2.1 PD inverse model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
xi
3.3 Drug resistance model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
3.4 Pharmacodynamical models - summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
4 Model Predictive Control 29
4.1 Phase 1 - Model definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
4.1.1 Linear model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
4.1.2 Nonlinear model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
4.2 Phase 2 - Optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
4.3 Phase 3 - Receding horizon strategy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
4.4 Model predictive control - summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
5 Adaptive control 37
5.1 Recursive least squares method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
5.2 Gompertz model parameters estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
6 MPC Results Analysis and Features 41
6.1 Simulation A . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
6.1.1 Effect of umax . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
6.1.2 Effect of ρ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
6.1.3 Effect of N . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
6.1.4 MPC with RLS for parameters estimation . . . . . . . . . . . . . . . . . . . . . . . 46
6.2 Simulation B . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
6.2.1 Effect of umax and ρ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
6.2.2 Effect of N . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
6.2.3 Effect of the output prediction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
6.2.4 Effect of the optimization problem initialization . . . . . . . . . . . . . . . . . . . . 50
6.3 Multi-drug therapy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
7 Bone Microenvironment 53
7.1 Bone remodelling model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
7.2 Bone remodelling model with tumor interaction . . . . . . . . . . . . . . . . . . . . . . . . 54
7.2.1 PID discrete controller . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
7.3 Treatment - Killing tumor and recovering bone mass density . . . . . . . . . . . . . . . . . 57
8 Conclusions 61
8.1 Work and results analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
8.2 Future work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
Bibliography 65
xii
A Matrix algebra 67
A.1 Operations with matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
A.2 Matrix inversion lemma . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
xiii
xiv
List of Tables
2.1 Exponential-linear model first and second derivative signal table . . . . . . . . . . . . . . 8
2.2 Logistic model first derivative signal table . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2.3 Logistic model second derivative signal table . . . . . . . . . . . . . . . . . . . . . . . . . 9
2.4 Gompertz model first derivative signal table . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2.5 Gompertz model second derivative signal table . . . . . . . . . . . . . . . . . . . . . . . . 11
2.6 2nd order Runge Kutta methods parameters . . . . . . . . . . . . . . . . . . . . . . . . . . 14
xv
xvi
List of Figures
1.1 Osteoclasts, osteoblasts and osteocytes cells illustration . . . . . . . . . . . . . . . . . . . 2
1.2 System block diagram . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
2.1 Exponential-linear model time evolution for six different initial conditions T0 . . . . . . . . 8
2.2 Logistic model time evolution for six different initial conditions T0 . . . . . . . . . . . . . . 10
2.3 Gompertz model time evolution for five different initial conditions T0 . . . . . . . . . . . . . 11
2.4 Gompertz model equilibrium points . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2.5 Discretization of the Gompertz model with four different methods . . . . . . . . . . . . . . 15
2.6 Influence of the step size parameter h . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
2.7 Discrete tumor growth model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
2.8 Discrete tumor dynamics model (T) - block diagram . . . . . . . . . . . . . . . . . . . . . . 17
3.1 Pharmacokinetics time evolution - theory . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
3.2 Pharmacokinetics model impulse response. . . . . . . . . . . . . . . . . . . . . . . . . . . 20
3.3 Pharmacokinetics control model (PK controller) - block diagram . . . . . . . . . . . . . . . 22
3.4 Observer - block diagram . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
3.5 PK controller and observer performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
3.6 PD model: drug effect as a function of drug concentration . . . . . . . . . . . . . . . . . . 24
3.7 Pharmacodynamics model (PD) - block diagram . . . . . . . . . . . . . . . . . . . . . . . 25
3.8 Inverse pharmacodynamics model (PD−1) - block diagram . . . . . . . . . . . . . . . . . 25
3.9 Illustration of drug resistance phenomenon . . . . . . . . . . . . . . . . . . . . . . . . . . 26
3.10 Drug resistance model (R) - block diagram . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
3.11 Pharmacodynamical model (PC) - block diagram . . . . . . . . . . . . . . . . . . . . . . . 27
4.1 Predictive and non-predictive strategies . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
4.2 Example of MPC application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
4.3 Model Predictive Controller - block diagram . . . . . . . . . . . . . . . . . . . . . . . . . . 35
5.1 Recursive Least Squares (RLS) - block diagram . . . . . . . . . . . . . . . . . . . . . . . . 39
6.1 Illustration of 5% rise time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
6.2 Simulation A - block diagram . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
6.3 Unconstrained and constrained MPC phase 2 results . . . . . . . . . . . . . . . . . . . . . 42
xvii
6.4 System input and output with exponential reference signal . . . . . . . . . . . . . . . . . . 43
6.5 5% rise time as a function of umax . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
6.6 5% rise time as a function of ρ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
6.7 Cost and simulation time of having a prediction horizon of size N . . . . . . . . . . . . . . 45
6.8 MPC with RLS for parameters estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
6.9 Simulation A results with model parameters estimation . . . . . . . . . . . . . . . . . . . . 46
6.10 MPC for therapy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
6.11 Tumor density, reference signal and input signal. Drug concentration, c50 and drug resis-
tance variation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
6.12 Drug dose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
6.13 5% rise time as a function of umax and ρ . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
6.14 Cost and simulation time of having a prediction horizon of size N . . . . . . . . . . . . . . 49
6.15 Measurements available two in two days . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
6.16 Measurements available weekly . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
6.17 Measurements available weekly, slower reference signal and higher therapeutic time . . . 51
6.18 System input and output for different optimization problem initializations . . . . . . . . . . 51
6.19 Multi-drug therapy simulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
7.1 Osteoclasts, osteoblasts and bone mass density discretization . . . . . . . . . . . . . . . 54
7.2 Vicious cycle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
7.3 Osteoclasts and osteoblasts populations in the presence of tumor stimulated by an in-
crease in the initial quantity of osteoclasts . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
7.4 MPC for bone marrow cancer therapy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
7.5 Osteoclasts, Osteoblasts, bone mass density and tumor density behaviour when both
drugs d1 and d2 are applied between k1 and k2 . . . . . . . . . . . . . . . . . . . . . . . . 58
7.6 Drug effects u1 and u2, drug dose d2, drug concentration, drug resistance and c50 variation 59
7.7 Tumor density model parameters estimation . . . . . . . . . . . . . . . . . . . . . . . . . . 59
xviii
Nomenclature
Greek symbols
α1,2 Osteoclast and osteoblast production rate
β1,2 Osteoclast and osteoblast removal rate
∆ Pharmacokinetics sample time
δ Drug resistance model sampling time
η Tumor plateau level
θ Tumor growth model parameters estimation
λ Exponential reference signal parameter
φ Dependent variables vector
ρ Optimization parameter
Roman symbols
B Osteoblast steady-state value
C Osteoclast steady-state value
Z Bone mass steady-state density
a Tumor growth rate
b Tumor sensibility to drug
C Osteoclast quantity
c Drug concentration
c∗ Optimal drug concentration
c50 Drug concentration value for which the drug effect is 50% the maximum drug level
clim Drug concentration limit for drug resistance development
d Drug dose
xix
g11, g22 autocrine factors
g12, g21 paracrine factors
h Discretization step size
J Cost Function
Kg Kalman Gain
k1 Bone resorption activity
k2 Bone formation activity
Kd Derivative gain
Ki Integral gain
Kp Proportional gain
Kr Ability of the disease to develop drug resistance
N Prediction Horizon
O Osteoblast quantity
P Covariance Matrix
r Drug resistance value
T Tumor density
T ∗ Reference signal vector
T0 Initial tumor density
Tref Reference signal
u Drug effect
U∗ Optimal input vector
u∗ Optimal drug effect
umax Upper limit of drug effect constraints
umin lower limit of drug effect constraints
Y Virtual system output vector
Z Bone mass density
Subscripts
k Discrete time
t Continuous time
xx
Glossary
LS Leasts Squares
MPC Model Predictive Control
NMPC Nonlinear Model Predictive Control
OPG Osteoprotegerin
PC Pharmacodynamical model
PD−1 Inverse pharmacodynamics model
PD Pharmacodynamics model
PID Proportional, Integral and Derivative
PK Pharmacokinetics model
PTH Parathyroid Hormone
RANKL Receptor Activator of Nuclear Factor KB Ligand
RANK Receptor Activator of Nuclear Factor KB
RLS Recursive Least Squares
R Drug resistance model
xxi
xxii
Chapter 1
Introduction
1.1 Motivation
Despite its simple appearance, bone is a complex and dynamic tissue that is remodelled continuously.
The remodelling process consists of two phases: new bone is built while old bone is eliminated. The
human being has approximately 200 bones and each one is an extreme complex organ composed by
several different tissues working together: bone, cartilage, connective tissues, various blood tissues,
adipose tissue and nervous tissue. The microscopic structure of bone is also very complex. The bone
matrix is composed approximately by 25% water, 25% collagen fibres, and 50% crystallized mineral
salts. As minerals salts are deposited in the bone matrix they crystallize and the tissues hardens. This
process of calcification is initiated by osteoblasts, the bone-building cells [1]. There are three major
types of cells in the bone microenvironment:
• Osteoclasts are huge cells derived from a type of white blood cell. They release powerful enzymes
and acids that digests the protein and mineral components of the bone matrix. This breakdown
of bone matrix, termed resorption, is part of the normal development, growth, maintenance, and
repair of the bone;
• Osteoblasts are the bone-building cells. They synthesize and secrete collagen fibres and other or-
ganic components needed to build the matrix of bone tissue. As osteoblasts surround themselves
in the matrix, they become trapped in their secretions and become osteocytes;
• Osteocytes are mature bone cells. Among other functions, they are responsible for maintain the
daily metabolism, by exchanging nutrients and wastes products with the blood.
A delicate balance exists between the actions of osteoclasts in removing minerals and collagen and
osteoblasts in depositing them. Although bone hardness depends on the crystallized inorganic minerals
salts, bone flexibility depends on its collagen fibres. Collagen fibres and other organic molecules provide
tensile strength, resistance to being stretched or torn apart. If the balance is deregulated, for instance,
if too much mineral is deposited in the bone, the bone tissue may form thick bumps, that can interfere
1
Figure 1.1: Osteoclasts, osteoblasts and osteocytes cells illustration, adapted from [2].
with joints movement. On the other hand, an excessive loss of calcium or inadequate formation of new
tissues, weakens the bones, making them overly flexible or vulnerable to fracture [1, 3].
The genetic material of a cell can suffer changes or be damaged, developing mutations that affect
the normal growth of cellular structures and consequently their division process [3]. A wide range of
diseases that have in common an unusual and unnecessary cell reproduction beyond the organism
needs are called cancer. This uncontrolled proliferation provokes the formation of a cellular mass called
tumor. The tumor, in order to grow, needs to develop the capability of creating new blood vessels to get
the nutrients and oxygenation, called the angiongenesis process [4].
The bone microenvironment provides a fertile soil for cancer cells. The reciprocal interaction be-
tween tumor and bone cells, known as vicious cycle, supports the establishment and orchestrates the
expansion of malignant tumors in the bone. By interfering with the balance between osteoclasts and
osteoblasts, the bone mass density will decrease and the patient will suffer, as well as from other asso-
ciated bone pathologies [5].
The therapy to repress cancer growth has collateral toxic effects that affect the patient. There is thus
a strong motivation to employ methods from system theory and control in order to optimize treatment
doses.
1.2 Literature review
The increase of cancer threat to human health has led to huge advances in the treatment of this type of
diseases such as surgery, radio, chemo and immunotherapy. Those therapies must be well scheduled
to have an effective therapeutic result. In other words, there must be a right therapy which is going to
2
give the right drug dose to be administered to the patient at a specific time. The need for optimiza-
tion processes is obvious and that is one factor which explains the increasing number of mathematical
optimization studies for cancer treatments [3, 4, 6–13].
The purpose of a mathematical model for tumor growth is to predict and control the course of the
disease when a treatment is being applied. Having a mathematical model to represent the tumor growth
is very important [8]. In an animal, the disease may take months to run its course, in humans it may take
years. It is often faster and cheaper to formulate a mathematical model and simulate it on a computer
instead of performing a laboratory or clinical experiment [8]. In addiction, there are ethical impeachments
that prevent certain clinical experiments in human beings.
Model based dynamics optimization or optimal control has demonstrated to be an attractive research
area [6, 9, 13] and recently several studies have used Model Predictive Control (MPC) to compute a
therapeutic schedule. In [11] an optimal chemotherapy dose schedule, solving a convex optimization
problem based on linear matrix inequalities is computed; in [10] it is shown that even when the system
states are not fully directly measurable and there are mismatches in the model parameters, MPC still
provides an useful schedule for cancer treatment; in [12], MPC is used to provide a chemotherapy
schedule for mice with breast cancer; and for instance in another type of disease, MPC is used in the
treatment of HIV-1 to minimize the toxicity in a multi-drug therapy [14].
MPC is not only applied to the treatment of diseases. In fact, MPC has been used mostly in industrial
processes such as chemical plants or oil refineries, since the 80s. For instance, [15] uses MPC to steer a
boat along a desired path, in a path following problem; in [16], MPC is used to control a nonlinear coupled
tank system composed of two vertical tanks and a flow channel, causing the levels of the two tanks to
interact with each other, and finally in [17] MPC is used to control a DC/DC converter in ultracapacitors,
whose charge and discharge is a challenging process. Many other examples could be provided in this
realm.
It has been assumed in many researches related with diseases, that is possible to control directly
the effect of a drug on the target [6, 11, 13, 18–20] . The effect that the body has on the drug and the
effect that the drug has on the body is called pharmacokinetics (PK ) and pharmacodynamics (PD), re-
spectively. After the administration of a drug there are natural processes, such as solubility, distribution,
metabolism and elimination, that affect the amount of drug concentration in the target organ [21]. Stud-
ies [21] and [14] suggest mathematical models to represents those interactions. Resistance to drugs is
a natural process of the human body and is a major problem in cancer therapies [22]. In [14] a drug
resistance model is presented, based on the amount of drug concentration present in the bloodstream.
Bone marrow cancer is a common type of cancer that may be the result of metastasis from prostate
and breast cancers [5]. A very low percentage of patients (∼20%) survive for more than five years after
bone marrow cancer is diagnosed, because this type of disease establish a vicious cycle which not only
promotes the expansion of the tumor but also causes the decrease of the bone mass density, letting
the patient vulnerable to develop other type of bone diseases [5]. Since the bone microenvironment is
a fertile soil to the development of this type of cancer, it is crucial to better understand the interactions
between osteoclasts, osteoblasts, bone mass density and the tumor. In [23] a model to represent the
3
microenvironment interactions between osteoclasts, osteoblasts and the bone mass density was devel-
oped. A tumor growth model is proposed in [7] and the model of [23] was adapted to show the relations
that the tumor has with the bone microenvironment. A recent research [6] employs continuous optimal
control to deal with this disease and uses a classical PI controller to recover the bone mass density.
While some models take as the only state variable the tumor size, other approaches consider the
interactions of the tumor with other cancer cells, spread in different body parts such as the bloodstream
[24]. A recent and much promising direction of research concerns the exploitation of the interactions
with the immune system to boost cancer therapy. An example of this still rather unexplored approach is
provided in [25].
1.3 Objectives and main contribution
The main goal of this work is to develop a control based framework to schedule a therapy to reduce
the density of a cancer tumor. Since the algorithms are going to run on a computer, the project is
done in discrete time. The time evolution of the tumor density T , is represented by a discrete nonlinear
function, that depends on the tumor density itself and on the drug effect, u. To discover which drug
effect u should be applied, the Model Predictive Control (MPC) algorithm is used in order to solve an
optimization problem. For that sake, a quadratic cost function that weights the drug effect u and the error
between the tumor density T and a reference signal Tref , is used. An exponential reference signal is
used to generate smaller errors between T and Tref . Although some studies consider it possible, it is
not usual to have full control on the effect of a drug on the target organ. Therefore, the pharmacokinetics
(PK ), representing the variation of drug concentration as a function of time, and the pharmacodynamics
(PD), representing the variation of the drug effect as a function of the drug concentration, are modelled
and included on the system, to better traduce the reality. Like this, it is possible to directly manipulate
the drug dose.
Summarizing, a drug dose d (an impulse signal) will generate a drug concentration c given by its
PK model, that in turn is going to produce a drug effect u, given its PD model. Since MPC computes
an optimal drug effect u∗, but only the drug dose d can be manipulated, it is necessary to convert the
optimal drug effect u∗ to the optimal drug concentration c∗. To find the optimal drug concentration c∗,
the inverse PD model is defined and used. To discover which drug dose d is going to generate a drug
concentration c as close as possible to the optimal drug concentration c∗, a controller with an asymptotic
observer is designed (Figure 1.2).
Controller PD−1 PK control PD Systemu∗ c∗ c u
Figure 1.2: System block diagram.
It is often assumed that the system state, i.e., the tumor density T , is always available for mea-
surement. To overcome this dependence, new measurements are predicted by using the tumor growth
model and old measurements of the tumor density.
4
MPC only works when a system model is well known. Thereafter, the accuracy of this model should
be the maximum possible. To find the model for tumor density that best fits for a patient, the Recursive
Least Squares (RLS) method is applied to learn the model parameters from data in real time, yielding
an adaptive MPC algorithm.
Drug resistance is a phenomenon that is usually present on cancer therapies. When the drug con-
centration c is low, only the weak cells are killed and the strong cells survive, being able to reproduce
themselves. Therefore, all cells will be drug resistant and an higher drug concentration will be needed to
perform the same drug effect. To traduce this phenomenon, a drug resistance model is used. The MPC
optimization problem was also changed to combine multiple drugs allowing the framework to overcome
drug resistance.
The developed framework is used in a state-of-the-art bone marrow cancer research to eliminate the
tumor and to break the vicious cycle, which is formed in this type of disease. The bone mass density is
recovered using a classical discrete PI algorithm.
To sum up, the main contribution of this dissertation consists of a viability study of the design of
therapies to repress tumor size that are based on MPC. Both a non-adaptive and an adaptive version
are considered.
1.4 Thesis outline
In this chapter an introduction to the work performed in this dissertation has been done by explaining
the motivation behind the study of this theme, followed by a literature review and an explanation of the
main contributions. Thereafter, the work is divided into the following chapters:
• In Chapter 2 a mathematical revision on the tumor growth models used in the literature is done.
An introduction of the model with input control, that will be used to traduce the tumor growth
evolution in time, is given. To obtain a discrete model, four different numerical methods are studied;
• In Chapter 3 a mathematical definition of the pharmacokinetics, pharmacodynamics and drug
resistance models is done. A pole placement control algorithm with an asymptotic observer to
control the pharmacokinetic response is also included;
• In Chapter 4 a mathematical formulation of the Model Predictive Control algorithm is done,
dividing it into phases to better understand this algorithm;
• In Chapter 5 the Recursive Leasts Squares method is presented, used to estimate the model
parameters in a on-line mode;
5
• In Chapter 6 the simulations performed are explained. The results obtained are then presented
and discussed;
• In Chapter 7, the developed framework is used to control the tumor growth in a bone microenvi-
ronment situation, regarding the relations with the osteoclasts and osteoblasts. An algorithm to
enhance the recovery of the bone mass density is also purposed;
• In Chapter 8 the results obtained are summarized and topics that may be interesting to study in
the future are discussed.
6
Chapter 2
Tumor Growth Models
A differential equation can be used to model the growth of a tumor, with an initial tumor cell population.
According to [8], tumor cells population can be calculated from tumor density measurements, since there
is an approximately linear relationship between the tumor density and the cells number.
Let T be a function of time t, where T (t) describes the density of the tumor cell population per unit
time. This function is positive, continuous and monotonically increasing in the respective domain. The
tumor growth model is given by dT (t)
dt= f(T (t))
T (0) = T0
, (2.1)
where T0 is the initial tumor density. For notation simplicity, time dependence of the functions is omitted.
Let T (t) ≡ T anddT
dt= T . Thus, growth models are given by
T = f(T )
T (0) = T0
. (2.2)
On the following sub chapters, three tumor growth models are studied regarding their mathematical
properties.
2.1 Exponential-linear model
The exponential-linear model for the tumor growth is given by
f(T ) =
aT, T < η
0, T = η
, (2.3)
where a ∈ <+ is a constant equal to the proportional increase of the tumor cell population per unit time
and η ∈ <+ is the maximum value for tumor density [26]. By studying the first and second derivative, T
7
and T , with a signal table, some conclusions can be made. The second derivative is given by
T = a2T, (2.4)
and this model has the signal table presented in Table 2.1. For values of T ∈ [0, η[, the tumor density is
T 0 → η
LTL 0 + 0T sp ↗
lT l 0 + 0T ip ∪
Table 2.1: Exponential-linear model first and second derivative signal table. sp - stationary point — ip -inflection point.
increasing and the function T is concave upwards. The function has a stationary and an inflexion point
at T = 0. Studying the stability of the stationary point, linearising the system around it
df
dT= a > 0, (2.5)
is proved that T = 0 is an unstable stationary point of the system, in the sense of Lyapunov. When
T = 0, a slight change in T will cause the system to leave this point and never return to it. The time
evolution of this model, for different initial conditions T0 is shown in Figure 2.1. Although this model
0 10 20 30 40 500
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Time, k [week]
Tum
or d
ensi
ty, T
T0=10−5
T0=10−2
T0=0.25
T0=0.3
T0=0.5
T0=0.75
Figure 2.1: Exponential-linear model time evolution for six different initial conditions T0 with a = 0.3 andη = 1.
fits clinical data for some period of time, it does not model the truth. It was believed that growth tumor
under ideal conditions was a simple exponential process terminated by the exhaustion of the nutritional
support provided by the host [27]. Considering the growth of a tumor as a function of time, it was found
that tumors grow more and more slowly as they get larger, achieving no appreciable growth after a given
time [27]. Therefore a new model must be established to detail a more accurate tumor growth.
8
2.2 Logistic model
The logistic model is given by
f(T ) = aT (η − T ), (2.6)
where a ∈ <+ is the tumor growth rate and η ∈ <+ is the plateau population, an horizontal asymptote
such that
limt→+∞
η − T = 0. (2.7)
In order to evaluate the tumor growth monotony, Table 2.2 contains the T signal variation and its relation
with T . For values of T ∈ [0, η[, the tumor density is increasing and the function has stationary points at
T 0 → η
LTL 0 + 0T sp ↗ sp
Table 2.2: Logistic model first derivative signal table. sp - stationary point.
T = 0 and T = η. The stability of the stationary points is studied by linearising the system around them
df
dT= a(η − 2T ),
df
dT
∣∣∣∣T=0
= aη > 0,
df
dT
∣∣∣∣T=η
= −aη < 0.
(2.8)
This result proves that T = 0 is an unstable stationary point of the system, in the sense of Lyapunov.
When T = 0, a slight change in T will cause the system to leave this point and never to return to it.
However, T = η is an asymptotically stable point, in the sense of Lyapunov, meaning that when time
increases, the system tends to that point. The second derivative, T , satisfies now
T = a2T (η − T )(η − 2T ). (2.9)
The inflection points are discovered as follows
T = 0⇔ T = 0 ∨ T =η
2∨ T = η, (2.10)
Thus, Table 2.3 shows the relation between T and T . For values of T ∈ [0, η/2[, T is concave upwards
T 0 → η/2 → η
LTL 0 + 0 - 0T ip ∪ ip ∩ ip
Table 2.3: Logistic model second derivative signal table. ip - inflection point.
until it reaches the inflection point in half of its maximum size. Then, the function becomes concave
downwards, leading to a slow increase of the tumor density, T . Next, Logistic model is simulated for
9
different initial conditions T0. Results are shown in Figure 2.2.
0 10 20 30 40 500
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Time, k [week]
Tum
or d
ensi
ty, T
T0=0
T0=10−2
T0=0.2
T0=0.5
T0=0.75
T0=1
Figure 2.2: Logistic model time evolution for six different initial conditions T0 with a = 0.2 and η = 1.
Figure 2.2 shows that T = {0, η} are indeed stationary points of the model, as expected. The
remaining initial conditions show that the increase rate is bigger for small values of T . The concavity
changes when half of the maximum tumor size is reached, making the growth slower for values of T
near the plateau level, η. This properties makes T to be a S-shaped function.
2.3 Gompertz model
A more realistic model that is widely used in literature is the Gompertz model (for instance [4, 8, 13]).
Let the model be given by
f(T ) = aT log( ηT
), (2.11)
where a ∈ <+ is the tumor growth rate and η ∈ <+ is the plateau population, an horizontal asymptote
such that
limt→+∞
η − T = 0. (2.12)
By studying the first derivative T , some conclusions can be made. For values of T ∈]0, η[, T is increas-
T 0 → η
LTL nd + 0T nd ↗ sp
Table 2.4: Gompertz model first derivative signal table. nd - not defined, sp - stationary point.
ing and the function has a stationary point at T = η. By studying the stability of the stationary point
10
linearising the system around it
df
dT= a
(log( ηT
)− 1),
df
dT
∣∣∣∣T=η
= −a < 0.(2.13)
This result proves that T = η is an asymptotically stable point, in the sense of Lyapunov, meaning that
when time increases, the system tends to that point. By computing T
T = aT log( ηT
)(log( ηT
)− 1), (2.14)
the inflection points are discovered as follows
T = 0⇔ T = η ∨ T =η
e. (2.15)
Analysing now the second derivative T , some conclusions can also be made. The function T is concave
T 0 → η/e → η
LTL nd + 0 - 0T nd ∪ ip ∩ ip
Table 2.5: Gompertz model second derivative signal table. nd - not defined — ip - inflection point.
upwards when T ∈]0, η/e[, and is concave downwards when T ∈]η/e, η[. The concavity changes when
T is approximately 36.8% of η. Next, Gompertz model is simulated, for different initial conditions T0.
Analysing Figure 2.3, the model is concave upwards for T values below η/e. Thereafter, the concav-
0 10 20 30 40 500
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Time, k [week]
Tum
or d
ensi
ty, T
T0=10−5
T0=10−2
T0=1/e
T0=0.75
T0=1
Figure 2.3: Gompertz model time evolution for five different initial conditions T0 with a = 0.15 and η = 1.
ity changes and the growth is reduced until T reaches asymptotically the stationary point, η. Those
properties make the Gompertz model to be an S-shaped function.
11
2.4 Gompertz model with input control
On the previous sub chapters, several models were discussed, given their mathematical properties. The
Gompertz model is widely used in literature to represent tumor growth and to simulate its behaviour.
The models seen until here are autonomous. Thereafter it is necessary to change them, so they can be
actuated through an input variable. Let the tumor growth model be now defined by
T = f(T, u)
T (0) = T0
. (2.16)
For instance, in [7] a change in the Gompertz model, to introduce the control variable, is proposed,
resulting in
f(T, u) = (a− bu)T log( ηT
), (2.17)
where b ∈ <+0 is a parameter which traduces the disease sensibility to the drug and u ∈ <+ is the drug
effect. Although this model allows to drive the tumor density to low values, it does not reflect the reality.
Cell-kill drugs do not decrease the tumor growth rate (parameter a), instead, they kill cells after they are
established. Thus, consider the following model
f(T, u) = aT log( ηT
)− buT, (2.18)
with {a, η, u} ∈ <+ and b ∈ <+0 and equal biological meaning to all the parameters. Note that when the
tumor density T , becomes smaller, the effect of u also decreases and there is no danger that T is driven
to meaningless negative values.
When u reaches an equilibrium point u, the model (2.18) equilibrium points are given by
T = 0⇔
aT (log(η)− log(T ))− buT = 0.(2.19)
To solve the nonlinear equation (2.19), the Newton’s Method was used [28]. Let
g(T ) = aT (log(η)− log(T ))− buT . (2.20)
Solutions of (2.20) are iteratively given by
Tk+1 = Tk −g(T )
g′(T ), k ∈ N0, (2.21)
for an initial T0 and where g′ is the derivative of g in order to T . The method stops when the error,
Tk+1− Tk , is less then a threshold, ε. Figure 2.4 shows the solution of the method for different u values.
As the control equilibrium value u, increases, the tumor density equilibrium T decreases. Biologically
this means that the higher the drug effect is, the lower the tumor density will remain.
12
0 2 4 6 8 100
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
u
T
a=2ba=ba=0.5b
Figure 2.4: Gompertz model equilibrium points. Parameters: T0 = 0.5, ε = 10−3, η = 1.
2.5 Model discretization
Computers do not have infinite memory. Thus, continuous functions cannot be represented digitally. On
the previous sub chapter, the Gompertz model with input control was defined, but it traduces the tumor
growth as a function of continuous time t. To overcome this issue, model (2.18) must be discretized, so
that it can be implemented on a machine. Thereafter, four numerical methods for initial value problems
are tested: Euler, Medium point, Heun and Runge-Kutta method.
Let the initial value problem be defined as
T = f(t, T, u)
T (0) = T0
, (2.22)
where T is a function of continuos time t and the input variable u, with an initial condition T0.
2.5.1 1st order Euler method
The 1st order Euler method is given by
T (k + 1) = T (k) + hf(t(k), T (k), u(k)), (2.23)
where h ∈ <+ is the discretization step size and k is the discrete time. The Euler method is defined
considering the 1st order Taylor’s series of the exact solution of T . To obtain methods with higher order,
more terms in the Taylor’s series must be considered.
13
2.5.2 2nd order Runge Kutta methods
Let 2nd order Runge Kutta methods, Medium point and Heun method, be defined as
T (k + 1) = T (k) + h(c1V1 + c2V2),
V1 = f(t(k), T (k), u(k)),
V2 = f(t(k) + a2h, T (k) + b2hV1, u(k)),
(2.24)
where h ∈ <+ is the discretization step size and k the discrete time.
c1 c2 a2 b2Medium point meth. 0 1 1/2 1/2
Heun meth. 1/2 1/2 1 1
Table 2.6: 2nd order Runge Kutta methods parameters
2.5.3 4th order Runge Kutta method
The 4th order Runge Kutta method is defined
T (k + 1) = T (k) + h(V1 + 2V2 + 2V3 + V4)/6,
V1 = f(t(k), T (k), u(k)),
V2 = f(t(k) +h
2, T (k) +
h
2V1, u(k)),
V3 = f(t(k) +h
2, T (k) +
h
2V2, u(k)),
V4 = f(t(k) + h, T (k) + hV3, u(k)),
(2.25)
where h ∈ <+ is the discretization step size and k the discrete time.
2.5.4 Methods comparison
These four methods were used to discretize Gompertz model with input variable (2.18). The results are
represented in the Figure 2.5.
Figure 2.5 shows that the 4th order Runge Kutta method achieves the best results when the complete
domain of the function is considered (left figure). The figure on the right zooms the discretization to verify
the error of all the methods compared to the continuous one. The variation of the step size h will also
introduce an error to the discrete solution. Figure 2.6 shows how h influences the discretization, for 4th
Runge Kutta method. As Figure 2.6 shows, the bigger the step size h is, the bigger the error will be.
2.5.5 4th order Runge Kutta method for a system of equations
It was detailed previously how to discretize a system with only one continuous differential equation. For
a system of two continuous differential equations it is presented the 4th order Runge Kutta method, that
14
0 2 4 6 8 100
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Discrete time
Tum
or d
ensi
ty, T
ContinuosEuler meth.Medium Point meth.Heun meth.Runge−kutta 4th meth.
2.6 2.7 2.8 2.9 3 3.1 3.20.7
0.71
0.72
0.73
0.74
0.75
0.76
0.77
0.78
Discrete time
Tum
or d
ensi
ty, T
ContinuosEuler meth.Medium Point meth.Heun meth.Runge−kutta 4th meth.
Figure 2.5: Discretization of the Gompertz model with four different methods. Discretization step size:h = 0.2. Gompertz model parameters: a = 1, η = 1, b = 0.
0 1 2 3 4 5 6 7 80
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Discrete time
Tum
or d
ensi
ty, T
h=0.2h=0.6h=1.2h=1.6h=2.0
Figure 2.6: Influence of the step size parameter h. Gompertz model parameters: a = 1, η = 1, b = 0.
will be needed lately on this work. Let the continuous model be given by
B = g(B,C), B(0) = B0
C = h(B,C), C(0) = C0
. (2.26)
15
The equivalent Runge Kutta discretization is then
C(k + 1) = C(k) + h(K1 + 2K2 + 2K3 +K4)/6,
B(k + 1) = B(k) + h(L1 + 2L2 + 2L3 + L4)/6,
L1 = g(C(k), B(k)),
K1 = h(C(k), B(k)),
L2 = g(C(k) + h ·K1/2, B(k) + h · L1/2),
K2 = h(C(k) + h ·K1/2, B(k) + h · L1/2),
L3 = g(C(k) + h ·K2/2, B(k) + h · L2/2),
K3 = h(C(k) + h ·K2/2, B(k) + h · L2/2),
L4 = g(C(k) + h ·K3, B(k) + h · L3),
K4 = h(C(k) + h ·K3, B(k) + h · L3),
(2.27)
where h ∈ <+ is the discretization step size and k is the discrete time.
2.6 Discrete Gompertz model - summary
Through the models studied in this chapter, the Gompertz model is the one chosen to represent the
discrete time growth evolution of a tumor. A tumor dynamics discrete model is obtained from Gompertz
model with input variable (2.18) with 4th order Runge Kutta method discretization (2.25). Figure 2.7
shows the weekly time evolution in the absence of control, u = 0.
0 10 20 30 40 50 600
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Discrete time
Tum
or d
ensi
ty, T
Figure 2.7: Discrete tumor growth model. Gompertz model parameter: a = 0.15, b = 0, η = 1, T0 = 0.05.Discretization step size: h = 1/7.
This model represents a tumor with a relatively fast growth, since it reaches the plateau level in about
1 year (52 weeks). The discretization step size h is 1/7, which means that one sample per day is used.
16
For future chapters, it is important to define an easier nomenclature for this procedure. Let Procedure
1 be define as the discretization operation of the Gompertz model with input variable (2.18), with an
initial condition T0, using 4th order Runge Kutta method (2.25) with step size h. Procedure 1 will be
represented by the block in Figure 2.8.
Procedure 1 Tu(k), T (k) T (k + 1)
Tumor dynamics model (T)
u(k), T (k) T (k + 1)
Figure 2.8: Discrete tumor dynamics model (T) - block diagram.
17
18
Chapter 3
Pharmacodynamical Models
Drug dosing can be done by using pharmacokinetics and pharmacodynamics models. Pharmacokinet-
ics (PK ) is a branch of pharmacology dedicated to the study of how the body affects a specific drug
after the administration. Figure 3.1 shows how plasma drug concentration changes in time after drug
administration.
Figure 3.1: Pharmacokinetics time evolution - theory.
The first phase, absorption, starts in choosing the drug administration method and finishes when the
drug arrives to the bloodstream. Many of the features that make the body resistant to damage from
foreign invaders and toxic substances also limit the drugs ability to combat diseases in the patient. The
transformation is made primarily by the liver through enzymatic degradation that inactivates the drug.
Both oral and rectal way are strongly affected by this effect. Intravenous administration way skips this
step since the drug is directly delivered to the bloodstream. The second phase consists of the distribution
of the drug to the body through the bloodstream. The most vascularized organs will be the first to be
affected by the drug, subsequently redistributed over the remaining organs. This is the phase where
the drug will reach its target organ and begin to actuate. The last phase, elimination, can be divided in
19
metabolism and excretion: metabolism phase is where the organism recognizes that a foreign substance
is present in the body, and starts metabolic processes to eliminate it; excretion phase is responsible for
removing the drug from the body. It could be made by the respiratory system for gaseous drugs although
renal excretion through urine is the main form of elimination [21, 29].
Pharmacodynamics (PD) is the study of the relationship between drug concentration and drug effect,
i.e. is the study of how a specific drug affects the body. For instance, the ageing process may induce
more or less sensitivity to a particular medications because the affinity that some medications have at a
particular receptor sites may change. Thus, it is important to consider the pharmacodynamics of each
patient, to achieve the most effective therapy [21].
It is also very important to maintain the drug concentration in the therapeutic range. Huge values
of drug concentration can intoxicate the patient and low values can make the patient’s body to develop
resistance to that drug.
On the following section, the PK, PD and the drug resistance quantitative models are presented.
3.1 Pharmacokinetics model
To represent pharmacokinetics model, a transfer function C(s), with two poles and unitary static gain is
considered
C(s) =Y (s)
U(s)=
ab
(s− a)(s− b),
=ab
s2 − (a+ b)s+ ab,
(3.1)
where s is the complex frequency in rads−1. The impulse response of system (3.1) is shown in Figure
3.2. After the drug administration, the drug concentration starts to increase until it reaches a maximum
0 0.5 1 1.5 2 2.5
x 104
0
1
x 10−4
Impulse Response
Time (seconds)
Dru
g co
ncen
trat
ion,
c
Figure 3.2: Pharmacokinetics model impulse response. PK model parameters: a = b = −0.5 · 10−3.
20
value, during the so called absorption phase. After that, the drug concentration starts to decrease with
an high slope, due to distribution phase. At last, the plasma concentration tends to zero, that represents
the elimination phase. With a double negative real pole, the drug is fully eliminated in approximately 7
hours.
This system has an equivalent continuous state space representation. Applying the Inverse Laplace
Transform with null initial conditions
s2Y (s) + s(a+ b)Y (s) + abY (s) = U(s)ab→L −1
y(t)− (a+ b)y(t) + aby(t) = abu(t)⇔
y(t) = (a+ b)y(t)− aby(t) + abu(t),
(3.2)
and considering the following state vector
x =
x1
x2
=
y(t)
y(t)
, (3.3)
the state space representations is then
x =
0 1
−ab a+ b
︸ ︷︷ ︸
A
x +
0
1
︸︷︷︸B
u,
y =[ab 0
]︸ ︷︷ ︸
C
x.
(3.4)
3.1.1 PK controller
This sub-chapter discusses one method to control a state space system, so that it can follow a reference
signal. This goal is achieved by designing a state variable linear feedback project. Due to technological
limitations, or sensors cost, the system state is usually inaccessible. Hence, one option is to estimate
the system state with an asymptotic observer [30]. The PK system is represented in state space form
and is characterized by matrices in model (3.4). The system is fully controllable and observable if and
only if the controllability C and observability O matrices have rank equal to the dimension of the state
vector x [30]. Thus, let the controllability and observability matrices be given by
C = [B AB] =
0 1
1 a+ b
,O = [C CA]T =
ab 0
0 ab
.(3.5)
Those matrices indeed have rank equal to the dimension of the state vector x.
Once it is proved that the PK system is fully controllable and observable, let the controller system
with asymptotic observer be designed as in Figure 3.3 and the asymptotic observer designed as in
21
+ Observer −K PKc∗ e x d c
-
PK control
PK controlc∗ c
Figure 3.3: Pharmacokinetics control model (PK control) - block diagram. The PK block represents thetransfer function (3.1).
Figure 3.4.
−L
B
+
+
∫
A
−L C
e
d
˙x x
Observer
Figure 3.4: Observer - block diagram.
It is desired that c follows the reference c∗ through state variable linear feedback. This is achieved by
designing gain vectors K and L, so that the controller and the observer have their poles in a specified
location on the complex plan. Let the PK system input be written as
d = −Kx. (3.6)
Note that d represents the drug dose that has to be administered to the patient so that the drug con-
centration c is equal to the desired drug concentration c∗. Indeed, d(k) is the amplitude of a Dirac delta
signal. Analysing the observer design in Figure 3.4, the state estimations have the following dynamics
˙x = (A−BK − LC)x− Le, (3.7)
and the closed loop system can be defined in an equivalent way with the following matrices
ACL = A−BK − LC,
BCL = −L,
CCL = C.
(3.8)
22
According to the superposition theorem, the characteristic polynomial of the global system (open loop
system and observer with state estimation feedback) is the product of the characteristic polynomial of
A − BK and A − LC. The gain vector K, can be designed considering that it is actually the state that
is in the feedback and not its estimative. The gain vector L, can be designed considering that the state
is estimated in open loop. Thus, the controller and the observer can be designed separately. Let the
controller αc(s) and observer αo(s) characteristic polynomials be given by
αc(s) = (s− p1)(s− p2), αo(s) = (s− p3)(s− p4), (3.9)
where poles p1 and p2 traduce the closed loop desired system dynamics and poles p3 and p4 represent
the dynamics of the asymptotic observer error, and will be designed so that the error vanishes asymp-
totically to zero. Applying the superposition theorem, vectors K and L are found, matching the system
characteristic polynomials αK(s) and αL(s) with the characteristic polynomials defined by the respective
poles
αK(s) = det(sI −A+BK), αL(s) = det(sI −A+ LC), (3.10)
and the procedure is finished by matching (3.9) with (3.10). The system response will be faster if the
poles p1 and p2 have a small time constant. These poles are chosen to be a negative real double pole
to avoid overshooting. This has to be an accurate design, because small time constants will produce
huge input values d, i.e. huge drug doses. Furthermore p3 and p4 time constants cannot be too small,
otherwise the error will vanish to zero very slowly and they cannot be to big otherwise the observer
will be deceived by the modeling errors. Figure 3.5 shows the performance of the PK controller. The
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.2
0.4
0.6
0.8
1
Time [s]
Drug concentrationReference
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1−3
−2.5
−2
−1.5
−1
−0.5
0
0.5x 10
−5
Time [s]
Obs
erve
r er
ror
Figure 3.5: PK controller and observer performance. Poles location on the complex plan: p1,2 = −50,p3,4 = −100. Controller and observer initial state, respectively: [0 0]T and [1 1]T .
controller designed for the PK system, makes the output to follow the input with an admissible quickness
as well as the observer error which vanishes to zero.
The discrete response of the PK model with the controller and state observer is given by the response
23
for non-homogeneous systems, which is composed by the solution of the homogeneous equation and
the input, using the superposition theorem [30]. Consider the state equation defined by matrices ACL,
BCL and CCL. The state evolution in discrete time is given by
x(k + ∆) = e∆·ACLx(k) +
∫ k+∆
k
eACL(k+∆−τ)BCLd(τ)dτ, (3.11)
where ∆ ∈ <+ is the discretization step size. For Dirac delta input signals, d(τ) = δ(τ − k), the state
equations is thus
x(k + ∆) = e∆·ACLx(k) + e∆·ACLBCLd(k), (3.12)
and the drug concentration is
c(k + ∆) = CCLx(k + ∆). (3.13)
3.2 Pharmacodynamics model
Pharmacodynamics (PD) is the study of the drug effect on the body. This can be represented by the
function which is given by the Hill equation [14], a static and non-linear relation between the drug effect
u, and the drug concentration c
u(k) =c(k)
c50 + c(k), (3.14)
where c50 ∈ <+ is the concentration value for which the effect is half of the maximum effect, u = 0.5.
Note that this model has an horizontal asymptote when the drug concentration tends to infinity meaning
that when the drug concentration increase, the drug effect tends to a maximum value, saturating. The
model evolution as a function of the drug concentration c is shown in Figure 3.6. Pharmacodynamics
10−2
10−1
100
101
102
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
X: 1Y: 0.5
Drug concentration, c
Dru
g ef
fect
, u
Figure 3.6: PD model: drug effect as a function of drug concentration (drug concentration in logarithmicscale). PD model parameter: c50 = 1.
model is normalized in order to vary between umin = 0 (no drug effect) and umax = 1 (maximum drug
effect) and it is represented by the block diagram presented in Figure 3.7.
24
PDc u
Figure 3.7: Pharmacodynamics model (PD) - block diagram. The relation between c and u is given byequation 3.14.
3.2.1 PD inverse model
It is called pharmacodynamics inverse model PD−1, to the model that expresses drug concentration
values as a function of drug effect. Starting from the PD model (3.14) and isolating c, the PD−1 model
is
c(k) = c50u(k)
1− u(k). (3.15)
This model is represented by the block diagram of Figure 3.8. The transpose variation of the drug
concentration as a function of the drug effect is shown in Figure 3.6.
PD−1u c
Figure 3.8: Inverse pharmacodynamics model (PD−1) - block diagram. The relation between u and c isgiven by equation 3.15.
.
3.3 Drug resistance model
Although the design of cancer treatment has become increasingly sophisticated, yet there is no treat-
ment that is fully effective against disseminated cancer. Even the most promising pharmacological
therapies will fail if the drug is unable to reach its target organ with a sufficient concentration to have
a therapeutic effect. Resistance to treatment with anticancer drugs results from a variety of factors in-
cluding individual variations in patients and cell genetic differences in tumors, even those from the same
origin tissue [31]. Frequently, resistance is intrinsic to the cancer, but as the therapy becomes more
and more effective, acquired resistance has also become common [31, 32]. It is assumed in this work
that drug resistance is only developed during treatment, which means that, initially, all tumor cells are
sensitive to the drug.
Resistance to the drug is developed when the drug concentration is below a given threshold, clim. If
the plasma concentration is low, weak cells are killed and only strong cells are going to be reproduced.
Given that, all the future cells are going to be drug resistant and an higher drug concentration is needed
to perform the same drug effect. Let r be defined, at discrete time k, as
r(k) = r(k − 1) + δ ·max(0, clim − c(k)), (3.16)
where δ is the sampling time and clim the limit above which no resistance is developed (Figure 3.9).
Assuming that c50 is proportional to r [14]
c50(k) = c50(0) · (1 +Kr · r(k)), (3.17)
25
Figure 3.9: Illustration of drug resistance phenomenon.
where c50(0) is the initial value of c50 parameter and Kr ∈ <+0 is the proportionality constant which
traduces the ability of the disease to develop resistance to the drug, it is created a model to represent
drug resistance. Figure 3.10 represents the resistance model diagram block.
Rc c50
Figure 3.10: Drug resistance model (R) - block diagram. The relation between c and c50 is given byequations 3.16 and 3.17. Drug resistance model parameters: δ = h = 1/7, clim = 5, Kr = 3. The drugresistance block has memory of the previous drug resistance level r(k − 1).
3.4 Pharmacodynamical models - summary
The pharmacodynamical models study the interactions between the human body and drugs and are
essential to obtain a successful therapy. As seen, the drug resistance phenomenon have implications
on the pharmacodynamics of the drug. As the body develop more resistance to a drug, the PD curve
starts to moving to the right side, meaning that more drug concentration is needed to perform the same
drug effect. It is necessary to adapt model (3.14) to the changes in the c50 parameter. Thus, consider
the PD model to be
u(k) =c(k)
c50(k) + c(k), (3.18)
where c50(k) is given by equation (3.17).
To conclude, let the pharmacodynamical model block diagram be defined as in Figure 3.11.
From now on, pharmacodynamical models will be represented by the PC block diagram, which is
composed by the PK, PD and drug resistance models, studied previously. The aim of this block is to
give the drug effect u, as close as possible to the optimal drug effect u∗. The PK controller output d
represents the drug dose that should be administered to the patient.
26
PD−1 PK control PD
R
cc∗
c50
u∗ u
Pharmacodynamical model (PC)
PCu∗ u
Figure 3.11: Pharmacodynamical model (PC) - block diagram.
27
28
Chapter 4
Model Predictive Control
Model Predictive Control (MPC) is an optimal control strategy based on numerical optimization. The
expression predictive control arises from a forecast of the variables: future control inputs and future
system responses are predicted using a system model. Then, an optimization problem at regular in-
tervals with respect to a performance function cost is solved. Despite being conceptually very simple
to design and implement, MPC algorithm can control large scale systems with many control variables,
and, most importantly, MPC provides a method for dealing with constraints on inputs and system states
[33]. Such constraints are present in all control engineering applications and represent limitations on
actuators and/or in system states. In MPC, these constraints are taken into account by explicitly solving,
in real-time, a constrained optimization problem.
There is a fundamental difference between predictive control and classical control (also known as PID
control): while classical controllers only observe the current variables (and remember the past) predictive
controllers observe the current and use also estimates of the future variables (and also remember the
past). Predictive thinking is more natural in everyday life. For example, during car driving, the driver
observes the future shape of the road, brakes when something is approaching, pushes gas if no one is
near, and decreases the speed if a slower car appears in the field of vision. Continuing this comparison:
classical control in the driving world would mean a driving style where the car driver looks only through
the side and back windows. In a curve, the driver would correct the position only after an error has
occurred [33]. Any real driver is a predictive controller, because he drives according to the curvature
of the road and what is in front of the car. It is easy to understand that the greater the preview, the
Figure 4.1: left: non predictive strategy — right: predictive strategy. Adapted from [33].
better the input control will be, but the calculations will be more time consuming. Furthermore, a small
sampling time is necessary, otherwise the driver will not be able to control the car in due time.
29
Although MPC is simple to understand, it becomes simpler if the algorithm is divided into three
phases. In general, MPC is formulated as solving online a finite horizon open-loop optimal control prob-
lem, subject to system dynamics and constraints, involving states and controls. Based on measurements
obtained at time k, the controller predicts the future dynamics behaviour of the system over a prediction
horizon N (phase 1) and determines the input such that a determined open-loop performance objective
function is optimized (phase 2). If there are no disturbances and no mismatch between the system and
the model, and also, if the optimization problem could be solved for infinite predictive horizon, then it will
be feasible to apply the input solution found for time k. However this is not possible in general. Due to
disturbances and system model mismatches, the real system behaviour is different from the predicted
behaviour. In order to incorporate a feedback mechanism, the open-loop input function obtained will
be implemented only until the next measurement becomes available (phase 3). Imagine, for instance,
that in a chess game, a piece must be moved. All the moves must be analysed including the opponent
moves. Following a criteria (which may be to win the game with the fewest possible moves) a sequence
of moves is chosen and the first decision of that sequence is applied. If the opponent’s move is as
expected, the sequence can continue, otherwise the process will be reviewed and a new sequence of
moves will be generated.
The time difference between the measurements and recalculations can vary, however it is often
assumed to be fixed, i.e. the measurements will take place every sampling time, h. Using the new
measurement at time k + h, the whole procedure - prediction, optimization and receding horizon - is
repeated to find a new input with the control and prediction horizons moving forward.
In the next sub chapters, a major focus is given on the MPC phases, briefly introduced here.
4.1 Phase 1 - Model definition
4.1.1 Linear model
When a linear model is used to simulate the real system, it becomes possible to compute explicitly, the
virtual output vector (also called predicted output vector). Consider the following discrete state space
model
x(k + 1) = Ax(k) +Bu(k), x(0) = x0,
y(k) = Cx(k),(4.1)
where x(k) is the system state, u(k) is the system input and y(k) is the system output, at a discrete
time k. The system state predicted one step ahead, i.e. x(k + 1|k)1, is given by (4.1). The system state
predicted two steps ahead, x(k + 2|k), is given by
x(k + 2|k) = Ax(k + 1|k) +Bu(k + 1|k), (4.2)
1x(b|a) ≡ system state at time b predicted at time a, with a ∈ N0, b ∈ N and b > a. For notation simplicity let x(a|a) = x(a).The same notation is used with the system input u and output y.
30
and replacing (4.1) in (4.2)
x(k + 2|k) = A2x(k) +ABu(k) +Bu(k + 1|k). (4.3)
Generalizing, the state predictor x(k + i|k) is given by
x(k + i|k) = Aix(k) + ΛUi, (4.4)
where i = 1, ..., N , N ∈ Z+ is the prediction horizon and Λ and U are given by
Λ = [ Ai−1B Ai−2B · · · A0B ], A0 = I,
Ui = [ u(k) u(k + 1|k) · · · u(k + i− 1|k) ]T ,(4.5)
where I is the identity matrix and Ui the virtual input vector. Thus, the virtual state vector X, is given by
X = Πx(k) +WUi
X =
x(k + 1|k)
x(k + 2|k)...
x(k + i|k)...
x(k +N |k)
Π =
A
A2
...
Ai
...
AN
W =
B 0 · · · · · · · · · · · · 0
AB B 0 · · · · · · · · · 0...
.... . . . . . . . . . . .
...
Ai−1 Ai−2B · · · B 0 · · · 0...
.... . . . . . . . . . . .
...
AN−1 AN−2B · · · · · · · · · · · · B
. (4.6)
Given the system output equation (4.1), the virtual output vector is easy to compute
Y = CX. (4.7)
4.1.2 Nonlinear model
In the case that the system behaviour is represented by a nonlinear model, it is not possible to do the
previous explicit computation. Consider the following nonlinear state model
x(k + 1) = f(x(k), u(k)), x(0) = x0,
y(k) = g(x(k)).(4.8)
The predicted output one step ahead, i.e. y(k + 1|k), is given by
y(k + 1|k) = g(x(k + 1|k)) = g(f(x(k), u(k))). (4.9)
31
The predicted system output, two steps ahead y(k + 2|k), is given by
y(k + 2|k) = g(x(k + 2|k)) = g(f(x(k + 1|k), u(k + 1|k)))
= g(f(f(x(k), u(k)), u(k + 1|k))).(4.10)
Generalizing, the predicted output y(k + i|k) is given by
y(k + i|k) = g(f(f(f...f︸ ︷︷ ︸i times
(x(k), u(k)), u(k + 1|k)), ...), u(k + i− 1|k))). (4.11)
Thus, the virtual output vector is
Y = [y(k + 1|k) y(k + 2|k) · · · y(k + i|k) · · · y(k +N |k)]T . (4.12)
4.2 Phase 2 - Optimization
The virtual input vector is computed by minimizing a cost function. This function can be defined, for in-
stance, in terms of future system states, system outputs, future system inputs or errors between system
outputs and references. Consider for now, that the cost function is called J . The virtual input vector is
found by solving the following optimization problem
U∗ = argminU
J. (4.13)
Constraints to the input variable like
umin ≤ u ≤ umax, (4.14)
can also be met by solving the following optimization problem
U∗ = argminU
J (4.15)
s.t. Ac · U ≤ bc,
where Ac is a constant matrix and bc is a vector, depending on (4.14). The inequality is to be taken
elementwise.
In this work, the cost function J will be the quadratic cost function, which can be written in matrix
notation as follows
J(x(k), U) = (Y − Y ∗)T (Y − Y ∗) + ρUTU, (4.16)
where Y ∗ is the reference signal vector that Y should follow and has the same size as Y and ρ ∈ <+ is
a tuning parameter. Function J will weight the input signal and the error between the system output and
the reference signal.
If the system is represented by a linear model (4.1), an analytic solution can be computed for the
32
optimization problem. Replacing (4.7) in (4.16) and neglecting the constant terms
J = (CΠx(k) + CWU − Y ∗)T (CΠx(k) + CWU − Y ∗) + ρUTU
= (x(k)ΠTCT + UTWTCT − Y ∗T
)(CΠx(k) + CWU − Y ∗) + ρUTU
= UT (WTCTCW + ρI︸ ︷︷ ︸M
)U + 2UTWTCT (CΠx(k)− Y ∗) +(((((((
((x(k)ΠTCTCΠx(k) +��
��Y ∗
T
Y ∗
= UTMU + 2UTWTCT (CΠx(k)− Y ∗).
. (4.17)
One method to find the function (4.17) minima consists of computing the respective gradient and finding
its zero. Another method is to match function (4.17) with a generic quadratic function where the minima
is well known. The generic expression of a quadratic function with a minima at U∗ is given by
K =(U − U∗)TM(U − U∗)
=UTMU − 2UTMU∗ +�����
U∗T
MU∗.(4.18)
Neglecting the constant term and matching (4.17) with (4.18), the predicted input vector is found to be
MU∗ = −WTCT (CΠx(k)− Y ∗)
U∗ = −M−1WTCT (CΠx(k)− Y ∗).(4.19)
�
If a nonlinear prediction model is employed, the optimization problem is significantly harder than for the
linear model case. Generally there is no guarantee that a solver will converge to a global minima, and
the time to find even a local solution is greater that the linear model case [33].
4.3 Phase 3 - Receding horizon strategy
MPC phase 2 ended with the computation of the optimal input vector U∗. Consider now Figure 4.2,
where MPC is applied. At discrete time k, it is intended to find which is going to be the system input
Figure 4.2: Example of MPC application (adapted from [34]).
33
u(k), by using MPC with a prediction horizon N . Thus, consider that the system output is known (pink
signal) at time k. The MPC output (marine blue signal), is the vector U∗, and only the first element of U∗,
u∗(k), is going to be applied to the system. This procedure is repeated for k + 1, in a receding horizon
strategy. By applying only the first element, and observing the system output, a degree of robustness is
introduced into the controller through the system output feedback, which makes the controller to be able
to deal with errors and/or uncertainties. To summarize, MPC Algorithm has the following pseudo code:
Algorithm 1 MPC algorithm - pseudo code
1. set time k ← 12. obtain measurements/estimates of the state(s) of the system for time k3. create virtual state vector using the system model (phase 1)4. obtain an optimal input vector by minimizing a given cost function over a certain prediction horizon(phase 2)5. implement the first element of the optimal input vector and wait until a new measurement of thestate is available (or compute state estimation) (phase 3)6. do k ← k + 1 and go to 2. until the simulation is finished
4.4 Model predictive control - summary
In the previous sub chapters, MPC was mathematically formulated. The algorithm is here summarized
and adapted to this work mathematical notation. As seen, MPC is used to compute the input variable u.
In a receding horizon strategy, it solves the following constrained optimization problem
U∗ = argminU
J
s.t. Umin ≤ U ≤ Umax,(4.20)
where J is the quadratic cost function
J(T (k), U) = (Yk+1 − T ∗k+1)T (Yk+1 − T ∗k+1) + ρUTk Uk, (4.21)
where:
Yk+1 = [T (k + 1|k) T (k + 2|k) · · · T (k +N |k)]T , (4.22)
is the vector of the virtual system outputs with length equal to the number of predictors N and starting at
k + 1 (the system outputs are iteratively computed using the procedure 1, previously defined in chapter
2.6); T ∗ is the reference vector with the signal that the system should follow
T ∗k+1 = [Tref (k + 1) Tref (k + 2) · · · Tref (k +N)]T , (4.23)
34
where Tref (k) is the reference signal, function of discrete time k; and finally U is the vector of the virtual
inputs, with length N and starting at k
Uk = [u(k) u(k + 1|k) · · · u(k +N − 1|k)]T . (4.24)
The solution of (4.20) will be the optimal input vector
U∗k = [u∗(k) u∗(k + 1|k) · · · u∗(k +N − 1|k)]T . (4.25)
The constraint vector Umin is a vector of zeros with length N and Umax is given by
Umax = 0.99 · [umax umax · · · umax︸ ︷︷ ︸N
]T . (4.26)
Although it has been concluded in chapter 3.2 that the drug effect u will vary between umin = 0 and
umax = 1, the inverse pharmacodynamics model is not defined when u = 1 (the corresponding drug
concentration value would be infinity) and for that reason, Umax is limited to 99% of the maximum drug
effect.
To solve the optimization problem (4.20) the MATLAB functions fminunc and fmincon are used for the
unconstrained and constrained problem, respectively. These MATLAB functions use the quasi-Newton
algorithm, starting at a initial solution U0 provided by the user. Every discrete time k, it will be necessary
to provide an U0 estimation, therefore, let the initial estimates of U0 for the discrete time k be named
U0(k). The first initial estimative of U0, U0(0) is a vector of ones with length N . Therapeutically, this
estimate is equivalent to the worst case scenario where the drug effect is set to the maximum admissible
value in all the prediction horizon. Although the choice of U0(0) influence the solution of (4.20), the
initialization U0(0) is not crucial to solve the optimization problem. The following estimates are made by
using the optimal input vector computed by the MPC at k − 1, U∗k−1. According to the definition of the
optimal input vector (4.25), the initial estimate of U0 for k > 0 is given by
U0(k) = [U∗k−1(2 : end)T 1]T . (4.27)
The model predictive controller as the diagram block as Figure 4.3 suggests.
MPCT (k), T ∗k+1 u(k)
Figure 4.3: Model Predictive Controller - block diagram.
Concluding, the model predictive controller will receive as an input the actual state of the system
which in this work is the tumor density and the vector with the reference signal (will receive also the
discretization step size h, the initial estimate of U∗, U0, among other variables that are omitted in the
block diagram for simplicity). The MPC will compute the system input variable u after running all the
three phases described in this chapter.
35
36
Chapter 5
Adaptive control
In this chapter, a method to estimate the tumor growth model parameters is studied. Although the well-
known Leasts Squares Method (LS) would probably be the first choice for parameters estimation, this
method has the huge disadvantage of being an off-line method. This means that all the data has to be
known before the estimation procedure, which is not admissible, in the context of this work. Given that,
a recursive method is defined, starting from the LS method and estimating the parameters as a function
of the past estimates and using the system input and output signals [35].
5.1 Recursive least squares method
The objective consists in adjusting the parameters of a linear in the parameters model to best fit a data
set of size n. Let the model be defined as
y(k + 1) = φT (k)θ + e(k + 1), (5.1)
where y is the dependent variable, φ is a vector with the independent variables, θ is a vector with length
m, equal to the number of parameters to be estimated and e is noise present in the measurement. The
LS method finds its optimum estimative θ when the sum S of the squared errors
S =1
2
n∑i=1
(y(i+ 1)− φT (i)θ)2 (5.2)
is minimum. The error is defined as the difference between the actual value of the dependent variable
and the value predicted by the model. Solving for minimizing S, the closed form solution is obtained as
follows
θ =
(n∑i=1
φ(i)φT (i)
)−1( n∑i=1
φ(i)y(i)
), (5.3)
or, rewriting in matrix notation
θ = (ΦTΦ)−1(ΦTY ), (5.4)
37
where
Y = [y(1) · · · y(n)]T ,
Φ = [φT (1) · · · φT (n)]T .(5.5)
This is the usual LS method for off-line parameter estimation. Note that to perform this estimation it is
necessary to have full knowledge of the data set.
By assuming now that one additional observation becomes available, the problem is then to find the
current parameters estimative θ(k+1) as a function of the previous ones θ(k), the system output y(k+1)
and the system input u(k). Defining Φ(k + 1) and Y (k + 1) as
Φ(k + 1) = [Φ(k) φT (k + 1)]T ,
Y (k + 1) = [Y (k) y(k + 1)]T ,(5.6)
and defining the covariance matrix
P (k) = (ΦT (k)Φ(k))−1, (5.7)
the following equations are true
P (k + 1) = [ΦT (k)Φ(k) + φ(k + 1)φT (k + 1)]−1,
θ(k + 1) = P (k + 1)[ΦT (k)Y (k) + φ(k + 1)y(k + 1)].(5.8)
Combining (5.8) and the Matrix Inversion Lemma (Appendix A) with
A = ΦT (k)Φ(k), B = φ(k + 1), C = φT (k + 1), (5.9)
and after some simple matrices manipulations, the Recursive Least Squares method (RLS) is then
θ(k + 1) = θ(k) +K(k + 1)[y(k + 1)− φT (k)θ(k)],
K(k + 1) =P (k)φ(k)
1 + φT (k)P (k)φ(k),
P (k + 1) = P (k)− P (k)φ(k)φT (k)P (k)
1 + φT (k)P (k)φ(k),
(5.10)
where K is the Kalman gain. Note that to run the RLS method, it is only necessary to know an initial
estimative of P and θ.
5.2 Gompertz model parameters estimation
Although knowing the model structure is extremely important, the model parameters also play a key role
in respect to adjusting the model to each patient. In fact, parameters may change for each patient which
could lead to different therapies. To overcome this issue, the RLS method is used to estimate the model
parameters. As seen in chapter 2.5, the discretization of the Gompertz model using the 4th order Runge
38
Kutta method is an accurate method but it is an hard-working process and the result is a model with
nonlinear dependence on the parameters to be estimated. For this purpose, the 1st order Euler method
is used. Recalling the Gompertz model with input control and discretized by the 1st order Euler method
T (k + 1) = T (k) + h
(aT (k)log
(η
T (k)
)− bu(k)T (k)
). (5.11)
Rewriting the model to be in the same form of (5.1)
T (k + 1)− T (k) = h
(aT (k)log
(η
T (k)
)− bu(k)T (k)
). (5.12)
This means that is the difference of tumor growth between consecutive time instants that is estimated.
By comparing (5.1) with (5.12) the RLS is defined as
y(k + 1) = T (k + 1)− T (k),
φ(k) =
[hT (k)log
(η
T (k)
)− hu(k)T (k)
]T,
θ = [a b]T .
(5.13)
It is assumed that the plateau level η, is well-known.
As MPC algorithm and RLS method use different discretization methods for obtain a discrete Gom-
pertz model, it is expected that the estimated parameters with the Euler method have a deviation to
the real ones, according to the results saw in chapter 2.5 Figure 2.5. It is acceptable to consider, for
instance, that this difference is a result of an error on the tumor density measurement process.
The Recursive Leasts Squares method has the block diagram illustrated in Figure 5.1. In future
illustrations, the branch containing θ(k) and P (k) as well as the output P (k + 1) will be omitted, for
simplicity.
RLSu(k), T (k), T (k + 1)
θ(k), P (k)
θ(k + 1), P (k + 1)
Recursive Least Squares method (RLS)
Figure 5.1: Recursive Least Squares (RLS) - block diagram. The RLS algorithm is composed by equa-tions (5.10) and (5.13).
39
40
Chapter 6
MPC Results Analysis and Features
In this chapter, the previous developed work is applied to control the growth of a tumor. Consider two
different simulations: in Simulation A, MPC is used to control the growth of a tumor by assuming that
it is possible to control directly the drug effect u; in Simulation B, the pharmacodynamical model PC is
also considered to obtain the therapy as closest as possible to clinical reality. The RLS method is used
to estimate the model parameters in both simulations A and B.
Then, some MPC features are tested. To study the quickness of MPC to make the system to ap-
proach the reference signal, the rise time is evaluated. Let 5% rise time be a measure that translates the
time that the tumor density T takes to reach 95% of its equilibrium value, in days. Figure 6.1 illustrates
how the rise time is measured.
Figure 6.1: Illustration of 5% rise time. Parameter Teq is the equilibrium point of tumor density (blue line)and T∞ is the equilibrium point of the reference signal (red line).
6.1 Simulation A
For the purpose of studying the MPC algorithm and its features, consider a block diagram composed
by the tumor growth model that represents the patient and the MPC controller, designed as Figure 6.2.
At one iteration of the algorithm, MPC block will compute the optimal drug effect u∗(k), that will make
tumor density T to approach the reference signal Tref .
Consider now, that Gompertz model b parameter was set to b = 1.5, meaning that tumor growth is
41
MPC T
z−1
u(k)
T (k)
T (k), T ∗k+1 T (k + 1)
Figure 6.2: Simulation A - block diagram. z−1 represents an unit delay.
now sensitive to drugs, and also the initial tumor density was set to T0 = 0.975. By applying the MPC
controller with a constant reference signal, Tref (k) = 0.1, the unconstrained and constrained results are
shown in Figure 6.3, using a prediction horizon N = 7 days (the reason for this choice of the prediction
horizon value is explained later).
0 0.5 1 1.5 2 2.5 3 3.5 40
0.2
0.4
0.6
0.8
1
ReferenceTumor Density
0 0.5 1 1.5 2 2.5 3 3.5 40
2
4
6
8
Dru
g ef
fect
, u
Time, k [week]
0 0.5 1 1.5 2 2.5 3 3.5 40
0.2
0.4
0.6
0.8
1
ReferenceTumor Density
0 0.5 1 1.5 2 2.5 3 3.5 40
0.2
0.4
0.6
0.8
1
Dru
g ef
fect
, u
Time, k [week]
Figure 6.3: Left: Unconstrained optimization problem — Right: Constrained optimization problem. Pro-cedure 1 parameters: a = 0.15, b = 1.5, η = 1, T0 = 0.975, h = 1/7. Reference signal: Tref (k) = 0.1.
The left column of Figure 6.3 shows the unconstrained situation and it is possible to conclude that this
controller causes the system output T to follow the reference signal. Drugs typically have a maximum
drug dose that may be administered to patients. As seen in chapter 3.2 dedicated to pharmacodynamics,
the effect of a drug was normalized to values between 0 (no drug effect) and 1 (maximum drug effect).
The unconstrained situation suggests that it should be administered to the patient such a drug dose that
the drug effect is almost 8 times the maximum drug effect, which for sure would eliminate the disease,
but also would take away patient’s life.
In many other industry applications of MPC, constrains are present and it is necessary to solve an
42
optimization problem taking them into account. The results of the constrained situation are in Figure 6.3
on the right column. The tumor density T , needs more time to approach the reference Tref but the drug
effect u is now between admissible values.
As can be seen, on Figure 6.3 right column, when considering a continuous reference signal, at the
beginning of the simulation, the difference between the tumor density T and the reference signal Tref is
huge and leads MPC to compute high values for drug effect u. To overcome this issue, it is possible to
consider an exponential reference, slow enough to give small errors between the tumor density T and
the reference signal Tref but fast enough to reach a small tumor density in the same therapeutic time.
Thus, consider the following reference signal Tref
Tref (k) = (T0 − T∞)e−λk + T∞, (6.1)
where T0 is the initial tumor density, T∞ is the equilibrium value for the tumor density ({T0, T∞} ∈]0,
η], T0 > T∞) and λ ∈ <+ is a quickness parameter. Repeating the experiment with the exponential
reference (6.1), the results are in Figure 6.4. By comparing Figure 6.3 right column with Figure 6.4, it
0 1 2 3 4 50
0.5
1
Ref
eren
ce
0 1 2 3 4 50
0.5
1
ReferenceTumor density
0 1 2 3 4 50
0.5
1
Time, k [week]
Dru
g ef
fect
, u
Figure 6.4: System input and output with exponential reference signal. Tref parameters: T0 = 0.975,T∞ = 0.1, λ = 0.2.
is proved that smallest differences between T and Tref requires less drug effect and consequently, less
drug doses (in Figure 6.3 it is needed an input of 1, maximum drug effect, approximately until week
1.5 while in Figure 6.4 the input was never set to 1). Also, the time variation of the drug effect is now
smoother.
6.1.1 Effect of umax
In figure 6.5 the 5% rise time variation is shown as a function of the maximum drug effect umax. As umax
decreases, the slower is the system response. However when umax is small, MPC cannot drive T to
Tref , even if the simulation time is increased. The bottom graph shows the ratio between the equilibrium
43
0 0.2 0.4 0.6 0.8 10
10
20
30
40
5% r
ise
time
[day
]
0 0.2 0.4 0.6 0.8 10
2
4
6
umax
Rat
io T
eq/T
∞
Figure 6.5: 5% rise time as a function of umax.
value of T , Teq, and the reference equilibrium value T∞. A red horizontal line is pointing where this ratio
is 1 (ideal situation). As seen, umax is set to 1 and by inspection of Figure 6.5, it is concluded that this
value allows MPC to drive T to T∞ in a admissible time and without offset.
6.1.2 Effect of ρ
In chapter 4, the optimization problem associated to MPC was defined, as well as the cost function to
minimize. Parameter ρ is a tuning parameter, that allows to give more or less weight to the dot product
of the virtual input vector. A study of the rise time as a function of ρ is done, to better choose the value
for this parameter. Figure 6.6 shows the variation of the rise time as a function of ρ. As ρ increases the
0 0.02 0.04 0.06 0.08 0.10
5
10
15
5% r
ise
time
[day
]
0 0.02 0.04 0.06 0.08 0.10
2
4
6
ρ
Rat
io T
eq /
T ∞
Figure 6.6: 5% rise time as a function of ρ.
smaller is the band width and the slower the system response will be. Furthermore, more robustness is
given to the system because the high frequency dynamics is attenuated. Another factor when choosing
44
big values for ρ is the presence of offset (Ratio Teq/T∞, Figure 6.6 on the bottom). For these reasons, ρ
was set to ρ = 10−4, which gives robustness enough to the system and creates low offset.
6.1.3 Effect of N
The choice of the prediction horizon is, probably, the most important factor for the success of MPC.
A small number of predictors will make the system to not observe the future, and a huge number of
predictors will make the system computationally heavier, among other problems that will be seen. For
the purpose of testing the MPC performance consider two tests: in Test 1, MPC knows a model that is
exactly equal to the system behaviour; in Test 2, MPC knows a model similar to the system (10-20%
errors on the parameters).
To evaluate the cost of having N predictors, the following cost function is used, which evaluates all
the experiment
J(N) =1
N
[D∑i=1
(YJ(i)− T ∗J (i))2 + ρU2J(i)
], (6.2)
where YJ is a vector that contains the experiment tumor densities, T ∗J is a vector that contains the
tumor density references and Uj is a vector that contains the input signals. These three vectors have
D samples, which is the size of the experiment. As the cost function evaluates the level of drug effect
and the error between the system output and the reference signal, it is expected that the cost J would
decrease when N increase, because MPC is looking longer in the future, for both testes. By analysing
0 5 10 15 200
0.5
1
1.5
2
2.5x 10
−3
Cos
t J
Prediction horizon, N
Test 1Test 2
0 5 10 15 200
5
10
15
20
25
Prediction horizon, N
Sim
ulat
ion
time
[s]
Test 1Test 2
Figure 6.7: Left: Cost of having a prediction horizon of size N — Right: Simulation time as a function ofthe prediction horizon N .The vertical black line is pointing at the chosen prediction horizon N = 7.
Figure 6.7, Test 1 has smaller costs J than in Test 2. This happens because the difference between T
and Tref does not vanish in Test 2, due to the differences between the model that MPC knows and the
real system behaviour. There is no specific rule to choose the best number of predictors N . Although
there is a rule of thumb that suggests that the chosen N should be after the cost curve knee, because
more predictors will cause a very small decrease of the cost. For the Test 1 case, the cost curve knee
is at N = 2, which means that any N > 2 is probably a good choice (for the Test 2 case, the cost curve
knee is approximately at N = 5). However, big values of N does not mean that the cost J is reduced
45
proportionally. It is clear that when N increase, the variation of J is almost null. Also, when N increase,
the simulation time increases exponentially (Figure 6.7). Thus, the prediction horizon N was set to 7.
Since the algorithm step size h has the unit of days, this prediction horizon value means that MPC is
observing one week in the future. Concluding, there is no significant difference in the simulation time
between the two tests, which is admissible, since MPC has to make exactly the same computations for
both tests.
6.1.4 MPC with RLS for parameters estimation
Unless it is possible to have an exact model that represents the real system with no errors, it is absolutely
necessary to have an accurate system model. In chapter 5, RLS method was studied in order to estimate
the Gompertz model parameters in real-time. Consider the addiction of the RLS block (Figure 5.1) to
the block diagram of Figure 6.2. The new block diagram is in Figure 6.8. As discussed previously,
MPC T
RLS z−1
u(k)
T (k)
T (k), T ∗k+1 T (k + 1)
θ(k + 1)
T (k)
Figure 6.8: MPC with RLS for parameters estimation. z−1 represents an unit delay.
there are two discretization methods operating in the algorithm. The 4th order Runge Kutta method is
more accurate, although the resulting model is nonlinear relative to its parameters. Thereafter RLS uses
the 1st order Euler method. For that reason, the existence of errors in the parameters estimations θ is
expected. The results of simulating the block diagram in Figure 6.8 are in Figure 6.9.
0 1 2 3 4 50
0.2
0.4
0.6
0.8
1
ReferenceTumor density
0 1 2 3 4 50.2
0.4
0.6
0.8
1
Time, k [week]
Dru
g ef
fect
, u
0 1 2 3 4 50
0.2
0.4
0.6
0.8
1
a
0 1 2 3 4 51
1.2
1.4
1.6
1.8
2
Time, k [week]
b
Figure 6.9: Left: System output and input — Right: Estimation of the model parameters. θ(0) =
[a(0) b(0)]T = [1 1]T and P (0) = 104 · I, which traduce the low confidence on the initial estimations.
Figure 6.9, on the right column, shows the estimation a and b (blue line) of parameters a = 0.15 and
46
b = 1.5 (red line). Although a is close to the real value, b shows an error when compared to the real
one, which is admissible as discussed. The left column represents the system output, that follows the
reference signal even with the error in the parameters estimation. This also shows the robustness of
MPC to deal with small errors between the output predictions and the real system output.
6.2 Simulation B
To give more realism to this work, the pharmacodynamical model were also included on the block dia-
gram. The resultant block diagram is presented in Figure 6.10. By applying the MPC controller, the PC
MPC PC T
RLS z−1
T (k)
T (k), T ∗k+1 u∗(k) u(k) T (k + 1)
T (k)
θ(k + 1)
Figure 6.10: MPC for therapy. z−1 represents an unit delay.
block and the RLS algorithm, the results obtained are in Figure 6.11. When all the models are joined,
0 1 2 3 4 50
0.2
0.4
0.6
0.8
1
ReferenceTumor density
0 1 2 3 4 50.2
0.4
0.6
0.8
1
Time, k [week]
Dru
g ef
fect
, u
0 1 2 3 4 50
5
10
Dru
g co
ncen
trat
ion,
c
0 1 2 3 4 50
5
10
15
Res
ista
nce,
r
Time, k [week]
0 1 2 3 4 50
2
4
c 50
Figure 6.11: Left: Tumor density, reference signal and input signal — Right: Drug concentration (thered line shows the clim parameter), c50 and drug resistance variation. PK and PD model: c50(0) = 1,∆ = 1/7. Drug resistance model: δ = h = 1/7, clim = 5 and Kr = 3.
it is interesting to analyse some results. In Figure 6.11, right column the drug concentration variation
is presented as well as the drug resistance variation. On the drug concentration graphic, the red hori-
zontal line represents the parameter clim above which no drug resistance is developed. It is clear that
until k ≈ 0.5 no drug resistance is developed and therefore the parameter c50 remains constant. Nev-
ertheless, when drug concentration becomes small and under the clim threshold, resistance to drug is
developed and the parameter c50 starts to increase. This means that to have the same drug effect u, the
47
drug concentration has to be higher. This is easy to check, for instance, between k = 3.5 and k = 4.7
since drug concentration is increasing but the drug effect u remains constant. It is also proved that even
with the pharmacodynamical model, MPC is able to control and to drive tumor density to the reference
signal. Figure 6.12 shows the drug dosage d variation, that is one of the outputs of the PC block. This
value reveals the dose that should be administered to the patient, to get the optimal drug effect u∗. Note
that the drug dose could be the number of pills or even the number of chemotherapy cycles. Since the
therapy is not specified in this work, the drug dose d is dimensionless.
0 1 2 3 4 50
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6x 10
−3
Dru
g do
se
Time, k [week]
Figure 6.12: Drug dose.
6.2.1 Effect of umax and ρ
Similar to the work done in Simulation A, it is also studied the influence of adding the pharmacodynamical
model in the response quickness of the system by evaluating the 5% rise time as a function of umax and
ρ.
Figure 6.13 shows that the 5% rise time as a function of umax has now an horizontal asymptote and
the variation of ρ has also increased in the number of days. This happens due to the presence of the
PC model, especially the PK system that makes the full system to have an higher order.
6.2.2 Effect of N
The variation of cost J as a function of the prediction horizon N is given again by the cost function (6.2).
For the same two tests considered before in chapter 6.1.3, the results obtained are shown in Figure
6.14.
The variation of cost J has the same curve as in Simulation A. However, the simulation time has
increased, due to the add of the PC block and the RLS method which introduce more computations on
the framework.
48
0 0.2 0.4 0.6 0.8 10
10
20
30
5% R
ise
time
[day
]
0 0.2 0.4 0.6 0.8 10
2
4
6
8
10
umax
Rat
io T
eq /
T ∞
0 0.02 0.04 0.06 0.08 0.10
5
10
15
20
5% r
ise
time
[day
]
0 0.02 0.04 0.06 0.08 0.10
0.5
1
1.5
2
2.5
ρ
Rat
io T
eq /
T ∞
Figure 6.13: 5% rise time as a function of umax and ρ. Prediction horizon: N = 7. Left figure: ρ = 10−4.Right figure: umin = 0, umax = 0.99.
0 5 10 15 200
0.005
0.01
0.015
Cos
t, J
Prediction horizon, N
Test 1Test 2
0 5 10 15 200
5
10
15
Sim
ulat
ion
time,
[s]
Prediction horizon, N
Test 1Test 2
Figure 6.14: Cost and simulation time of having a prediction horizon of size N . MPC parameters:ρ = 10−4, umin = 0, umax = 0.99. The vertical black line is pointing at the chosen prediction horizonN = 7.
6.2.3 Effect of the output prediction
When considering the real clinical situation, it is not admissible to assume that the tumor density mea-
sures are available every days. Thus, consider that measures are only available periodically, with a
period of two days and seven days. The unknown measurements are predicted using the estimated
model of the patient, which is updated by the RLS method. The results of the experiment are in Figure
6.15.
When the real measurements are only available in two in two days, it is clear that the tumor density
curve is not following the reference signal so well as it was when the measurements were right every days
(Figure 6.10). Although the system computes the maximum drug effect u for more time, the reference
signal is reached in the same therapeutic time (5 weeks).
When the measurements are only available weekly and it is necessary to predict more measurements
the results are even worst. Figure 6.16 shows the results when the measurements are available weekly.
49
0 1 2 3 4 50
0.2
0.4
0.6
0.8
1
ReferenceTumor density
0 1 2 3 4 50
0.2
0.4
0.6
0.8
1
Time [week]
Dru
g ef
fect
, u
0 1 2 3 4 50
5
10
Dru
g co
ncen
trat
ion,
c
0 1 2 3 4 50
5
10
Res
ista
nce,
r
Time [week]
0 1 2 3 4 50
2
4
c 50
Figure 6.15: Measurements available two in two days.
0 1 2 3 4 50
0.2
0.4
0.6
0.8
1
ReferenceTumor density
0 1 2 3 4 5−0.5
0
0.5
1
1.5
2
Time [week]
Dru
g ef
fect
, u
0 1 2 3 4 54
6
8
10D
rug
conc
entr
atio
n, c
0 1 2 3 4 5−1
0
1
Res
ista
nce,
r
Time [week]
0 1 2 3 4 5−1
0
1
2
c 50
Figure 6.16: Measurements available weekly.
This result happens because the errors between the real system and the model are now propagated
for six days until the real measurement is available again. In this case, to achieve the expected reference
signal the therapeutic time is not enough, because the drug effect u is always at the maximum value
possible. Consider now that the therapeutic time is increase from 5 to 7 weeks, and the reference signal
is an exponential which varies more slowly in time (Figure 6.17).
With those changes it is possible to control the tumor density, driving it to the reference signal, in
an admissible therapeutic time and with regular values for the drug effect u. This result is obtained by
accessing the real tumor density measurement only once in a week.
6.2.4 Effect of the optimization problem initialization
The initialization of the solution to the optimization problem (4.20), U0(0), was chosen to be a vector of
ones, representing the worst case, a non optimal therapy where the drug effect is always at its maximum
value.
50
0 1 2 3 4 5 6 70
0.2
0.4
0.6
0.8
1
ReferenceTumor density
0 1 2 3 4 5 6 70.4
0.6
0.8
1
Time [week]
Dru
g ef
fect
, u
0 1 2 3 4 5 6 70
50
100
Dru
g co
ncen
trat
ion,
c
0 1 2 3 4 5 6 70
5
10
15
Res
ista
nce,
r
Time [week]
0 1 2 3 4 5 6 70
2
4
c 50
Figure 6.17: Measurements available weekly, slower reference signal and higher therapeutic time thanin figures 6.15 and 6.16. λ = 0.1.
0 1 2 3 4 50
0.2
0.4
0.6
0.8
1
Tum
or d
ensi
ty, T
ReferenceTest1Test2Test3Test4
0 1 2 3 4 50.2
0.4
0.6
0.8
1
Time, k [week]
Dru
g ef
fect
, u
Test1Test2Test3Test4
Figure 6.18: System input and output for different optimization problem initializations. Test1: U0(0) =0.1 · 1. Test2: U0(0) = 0.5 · 1, Test3: U0(0) = 1, Test4: U0(0) is a random vector between 0 and 1, whereits elements may be different.
Besides the biological interpretation of this choice, any value for this vector will lead to the same
drug effect therapy and consequently the same system response. Figure 6.18 shows the system input
and output for four different U0(0). Although the system input and output are different in the first weeks,
all the system inputs converges to the same result, showing that the value of the initialization of the
optimization problem is not crucial.
6.3 Multi-drug therapy
Cell-kill drugs have the function of killing drug-sensitive cells. However, due to the develop of drug
resistance or other factors, an higher proportion of drug-resistant cells are leaved behind. As the tumor
begins to grow again, drug therapy may fail because the remaining tumor cells are now resistant [36].
It is also common to have a patient therapy that is composed by several different drugs, whose effects
complement each other [32]. Until here, only a single drug therapy was used and MPC formulation was
51
done to a single input signal. On this chapter, a multi-drug therapy is developed. Consider that the
Gompertz model has now σ input variables, and is given by
T = aT log( ηT
)− T
σ∑i=1
biui. (6.3)
Therefore, model (6.3) discrete version is given by 4th order Runge Kutta method (2.25), with additional
input variables. The MPC phase 1 is equal in both single and multiple drug therapy, only with additional
input variables (and additional virtual input vectors). However the MPC phase 2 has a modification on
the optimization problem. Starting with the quadratic cost function (4.16), but now with σ input vectors,
the new cost function is then
J(x(k), U1, · · · , Uσ) = (Y − Y ∗)T (Y − Y ∗) +
σ∑i=1
ρiUTi Ui, (6.4)
and the constrained optimization problem is
[U∗1 ; · · · ; U∗σ ]T = argminU1, ··· , Uσ
J (6.5)
s.t. Ac · [U1; · · · ; Uσ]T ≤ bc,
Considering σ = 2 and setting equal constraints to all drug effects u1 and u2, the results are in Figure
6.19.
0 1 2 3 4 50
0.2
0.4
0.6
0.8
1
ReferenceTumor density
0 1 2 3 4 50
0.2
0.4
0.6
0.8
1
Time, k [week]
Dru
g ef
fect
u
1
u2
u1+u
2
0 1 2 3 4 50
2
4
6
drug
con
cent
ratio
n
0 1 2 3 4 50
5
10
Res
ista
nce,
r
Time, k [week]
0 1 2 3 4 50
10
20
30
C50
drug 1drug 2
Figure 6.19: Multi-drug therapy simulation. Number of drugs: σ = 2. Procedure 1 parameters: a = 0.15,b1 = 1.5, b2 = 1.25, η = 1, h = 1/7. Optimization parameters: ρ1 = ρ2 = 10−4. It was assumed that thePC model is equal to both drugs.
Figure 6.19 shows the combination of two drugs effects to reduce the tumor density. Note that, the
drug effect is limited and umax = 1 for both drugs, meaning that the sum of the drug effects can be
greater than 1. The multi-drug therapy is thereafter more powerful.
52
Chapter 7
Bone Microenvironment
Normal bone metabolism depends on several factors: adequate minerals, vitamins, several hormones
and physical exercise. To better understand bone remodelling and the factors that lead to pathologies
with bone, microenvironment actions need to be studied. Osteoclasts and its precursor cells are regu-
lated by different signalling pathways and molecules. One key regulatory molecule is the protein Recep-
tor Activator of Nuclear Factor KB Ligand (RANKL) that is produced by a variety of cell types including
osteoblasts and its precursor cells. This protein has been shown to exercise its effect on osteoclasts
and their precursors due to interaction with the cell surface receptors Receptor Activator of Nuclear
Factor KB (RANK). The interaction between the RANKL and the receptor RANK triggers the activation
of signalling pathways which are involved in the activation, differentiation and survival of osteoclasts.
Such signalling pathways can trigger the fusion of precursor cells to form a mature osteoclast cell. On
these cells, RANKL can interact with its receptors, promoting osteoclasts activity and survival, leading
to an increase bone resorption. To balance the effect of RANKL, osteoprotegerin (OPG) is secreted
from osteoblasts. This protein acts like a decoy, biding to RANKL, which prevents this one to interact
with RANK on the surface of osteoclasts. Various hormones model the balance between RANKL and
OPG, for example, parathyroid hormone (PTH) stimulates the production of RANKL and at the same
time represses the formation of OPG. Estrogen can increase the formation of OPG and prevent RANKL
signalling [37–39].
7.1 Bone remodelling model
The interactions between osteoclasts, osteoblasts and consequently with the bone mass density were
modelled in [23], that explains the relations in the bone microenvironment
C = α1Cg11Bg21 − β1C,
B = α2Cg12Bg22 − β2B,
Z = −k1 ·max(0, C − C) + k2 ·max(0, B − B).
(7.1)
53
Osteoclasts and osteoblasts are responsible for bone remodelling and their quantities are represented
by C and B respectively. Both cells produce regulatory effectors which can affect their own production,
called autocrine factors (represented by g11 and g22), or regulators that can affect the other cell type
production, called paracrine factors (represented by g12 and g21). The production and removal rate of this
cells are given by parameters αi and βi, with i = 1, 2. The bone mass density is represented by Z and
this measure depends on the number of osteoclasts and osteoblasts in activity. The bone mass density
is normalized between 0 (no bone mass density) and 100 (normal bone mass density), and in a healthy
situation, the steady state value should be Z = 100. The parameters ki, with i = 1, 2, represent the bone
resorption and formation activity. Model (7.1) shows the interaction between osteoclasts, osteoblasts
and the bone mass density in continuous time. To run model (7.1) in a computer it is necessary to find
the equivalent model in discrete time. Thus, consider the 4th order Runge Kutta method for a system of
equations, studied in chapter 2.5.5. The result of the discretization is in Figure 7.1.
0 500 1000 15000
5
10
15
Ost
eocl
asts
0 500 1000 15000
200
400
600
800
Ost
eobl
asts
0 500 1000 150090
95
100
105
Bon
e m
ass
dens
ity
Time, k [day]
Figure 7.1: Osteoclasts, osteoblasts and bone mass density discretization. The blue curve representsthe discrete version of the continuous model (red curve). C = 1.16, B = 231.72, C(0) = 11.16, B(0) =231.72, k1 = 0.0748 % cell−1 day−1, k2 = 0.0006395 % cell−1 day−1 [7]. Discretization step size: h = 1.
Note that, for big time values (k > 750 days), there are errors in the discretization. This happens
because the 4th order Runge Kutta is a numerical method, therefore the solution is an approximation
of the continuous model. Figure 7.1 shows the osteoclasts and osteoblasts populations exhibiting their
regular periodic oscillations around the respective steady-states. This healthy balance between the bone
remodelling cells makes the bone mass density to oscillate around Z = 100.
7.2 Bone remodelling model with tumor interaction
It is still unknown how tumor appears in the bone marrow but after it is established it starts to deregulate
the balance between the osteoclasts and osteoblasts in order to be able to spread to other organs, i.e.
metastasize [5].
54
The concept of vicious cycle for supporting and maintaining metastatic tumor growth has successfully
explained how bone and cancer cells interact in a loop to allow the tumor to grow within the bone
microenvironment. The model describes how cancer cells comunicate with osteoblasts, which ultimately
leads to osteoclasts activation and accelerate bone resorption. This not only makes room for the cancer
to grow, but also triggers the release of growth factors who promote further tumor growth [5]. Figure
7.2 illustrate the vicious cycle. Summarizing, the presence of tumor deregulate the balance between
osteoclasts and osteoblasts and consequently deregulate the bone mass density.
Figure 7.2: Representation of the vicious cycle: Up-regulation of RANKL in bone cells and subsequentosteoclasts activation is driven primarily by tumor derived factors such as PTHrP. The action of osteo-clasts then trigger the release of growth factors which aid tumor to growth [5].
Ayati et al proposed in [7] the inclusion of the tumor density dynamics and the relations between the
tumor and the bone microenvironment. By changing the model of [23], the new model expresses the
interactions that the tumor has in the osteoclasts and osteoblasts dynamics
C = α1Cg11
1+r11T
η
Bg21
1+r21T
η
− β1C,
B = α2Cg12/
1+r12T
η
Bg22−r22
T
η − (β2 − u1)B,
T = (a− u2)T log( ηT
),
Z = −k1 ·max(0, C − C) + k2 ·max(0, B − B).
(7.2)
As discussed in chapter 2.4, the tumor density model was changed to a more realistic one. Hereafter,
the complete model is
55
C = α1Cg11
1+r11T
η
Bg21
1+r21T
η
− β1C,
B = α2Cg12/
1+r12T
η
Bg22−r22
T
η − (β2 − u1)B,
T = aT log( ηT
)− bu2T,
Z = −k1 ·max(0, C − C) + k2 ·max(0, B − B).
(7.3)
In the absence of control, u1 = u2 = 0, the system has the behaviour presented in Figure 7.3.
0 500 1000 15000
10
20
30
Ost
eocl
asts
0 500 1000 15000
500
1000
Ost
eobl
asts
0 500 1000 1500
−100
0
100
Bon
e m
ass
dens
ity
0 500 1000 15000
50
100
Tum
or d
ensi
ty
Time [day]
Figure 7.3: Osteoclasts and osteoblasts populations in the presence of tumor stimulated by an increasein the initial quantity of osteoclasts. C = 5, B = 316, α1 = 3 cell day−1, α2 = 4 cell day−1, β1 = 0.2day−1, β2 = 0.02 day−1, g11 = 1.1, g12 = 0, g21 = −0.5, g22 = 0, η = 100, a = 0.005, b = 0.7, r11 = 0.005,r12 = 0, r21 = 0, r22 = 0.2, C(0) = 15, B(0) = 316, T (0) = 1 [7].
The osteoclasts and osteoblasts populations in Figure 7.3 show the oscillations converging to a
steady-state, higher than the one of the healthy situation. The tumor grows to the plateau level and the
bone mass density converges with oscillations to zero. It is visible that the osteoclasts and osteoblasts
starts to increase in number, reflecting the attempt of the system to maintain the normal bone resorption
and formation even in the presence of the tumor. Although the amplitude of the osteoclasts decreases
over time, this reflects the decrease of the bone mass density. Note that the amplitude of the osteoclasts
oscillations in the presence of tumor is higher than in the healthy situation, reflecting the overall increase
of the osteoclasts associated to the myeloma bone disease.
7.2.1 PID discrete controller
To restore an healthy bone microenvironment, the osteoclasts and osteoblasts density oscillations must
return to a specific steady state. For this purpose, a discrete PID was used to compute the manipulated
variable u1 that traduce the osteoblasts regulator drug effect. Starting from the equation of a continuous
56
PID controller
u(t) = Kpe(t) +Ki
∫ t
0
e(τ)dτ +Kdde(t)
dt, (7.4)
where e(t) is the difference between the reference signal and the system output, and Kp, Ki and Kd are
respectively, proportional, integral and derivative parameters. Applying the Laplace transform with null
initial conditions on equation (7.4), the result is
U(s) = (Kp +Ki1
s+Kds)E(s). (7.5)
For digital implementation, the Z transform is applied with the forward Euler integrator
U(z)
E(z)= Kp +Ki
1
1− z−1+Kd(1− z−1). (7.6)
The canonical transfer function is given by
U(z)
E(z)=
b0 + b1z−1 + b2z
−2
a0 + a1z−1 + a2z−2, (7.7)
and matching (7.6) with (7.7)
b0 = Kp +Ki +Kd,
b1 = −(Kp + 2Kd),
b2 = Kd,
a0 = 1,
a1 = −1,
a2 = 0.
(7.8)
The controller in discrete time is given by the following difference equation [40]
u(k) = −a1
a0u(k − 1) +
b0a0e(k) +
b1a0e(k − 1) +
b2a0e(k − 2). (7.9)
�
7.3 Treatment - Killing tumor and recovering bone mass density
In this chapter, the framework represented by the block diagram in Figure 7.4 is applied to decrease the
tumor density and to recover the bone mass density. Two time instants were defined: k1 is the same as
in [7] and represents the begin of both of the treatments and k2 is where it is considered that the tumor
is eliminated.
The approach is the following: between k1 and k2 both osteoblasts recovery drug d1 and cell-kill drug
d2 are being administered to the patient and after k2, only d1 is still administered. This decision is based
on the bibliography [5, 41] which suggest that, to break the vicious cycle, the osteoclasts number must
return to a normal value, so that the excessive bone resorption stops which makes the tumor to stop
spreading to other organs.
The tumor is considered to be eliminated when its density is less or equal to T = 2%. The results of
57
MPC PC T
RLS z−1
T (k)
T (k), T ∗k+1 u∗2(k) u2(k) T (k + 1)
T (k)
θ(k + 1)
PIDBref (k + 1), B(k)
B
C
Z
B(k), C(k), T (k)
u1(k)
B(k), C(k)
B(k + 1)
C(k + 1)
Z(k + 1)
Figure 7.4: MPC for bone marrow cancer therapy. The block diagrams C, B and Z represents thedynamics of the osteoclasts, osteoblasts and bone mass density respectively. The dynamics of thesecells are given by the model (7.3).
the simulations, where both drugs are applied between k1 and k2 are in Figure 7.5 and 7.6.
0 500 1000 1500 2000 25000
50
100
Time, k [day]
Tumor densityReference
0 500 1000 1500 2000 250060
80
100
120
Bon
e m
ass
dens
ity
0 500 1000 1500 2000 25000
10
20
30
Ost
eocl
asts
0 500 1000 1500 2000 25000
500
1000
Ost
eobl
asts
Figure 7.5: Osteoclasts, Osteoblasts, bone mass density and tumor density behaviour when both drugsd1 and d2 are applied between k1 and k2 (vertical red lines). Discrete PID controller parameters: Kd =0, Ki = −10−8, Kp = −10−7. k1 = 600 and k2 = 1435. Reference signal quickness parameter:λ = 0.0044105. Initial tumor density at the beginning of the therapy: T (k1) = 79.51.
Between k = 0 and k1, where no drugs were applied, tumor density starts to increase, deregulating
the balance between osteoclasts and osteoblasts and consequently the bone mass density starts to
decrease from the steady state. The presence of the vicious cycle is also evident. The amplitude of
58
0 500 1000 1500 2000 2500−2
0246
x 10−4
u 1
0 500 1000 1500 2000 25000
0.05
0.1
0.15
u 2
0 500 1000 1500 2000 25000
0.01
0.02
0.03
Time, k [day]
Dru
g do
se, d
2
0 500 1000 1500 2000 25000
10
20
30
Con
cent
ratio
n, c
0 500 1000 1500 2000 25000
1000
2000
Res
ista
nce,
r
Time, k [day]
0 500 1000 1500 2000 25000
100
200
c 50
Figure 7.6: Drug effects u1 and u2, drug dose d2, drug concentration, c50 and drug resistance variation.
0 5 10 15 200
10
20
30
40
50
a
0 5 10 15 200
20
40
60
Time, k [day]
b
Figure 7.7: Tumor density model parameters estimation. The red horizontal lines show the patient modelparameters a = 0.005 and b = 0.7. RLS method initial conditions: θ(0) = [a(0) b(0)]T = [50 50]T andP (0) = 104 · I, which traduce the low confidence on the initial estimations. The estimation graph startswith the tumor cell kill therapy. The graph shows only the first 20 days of therapy.
osteoclasts and osteoblasts dynamics is changed, osteoclasts increase in number, destroying more
bone that the one that is created and also the period of oscillations is affected. Between k1 and k2
where both drugs d1 and d2 are applied, tumor density decrease until the tumor is consider to be extinct.
Osteoclasts and osteoblasts balance starts to recover as also as the bone mass density. After k2 where
only drug 1 is being administered, bone mass density continues to recover until it reach the steady
state Z = 100. After the elimination of the tumor, there is no meaning for the drug concentration c and
for the drug resistance level r, reason why the curves were set to 0 (Figure 7.6 right graph). Figure
7.7 shows the time evolution of the tumor density model parameters estimation, for the first 20 days of
therapy. The estimations, as explained in chapter 6, has the deviations to the real parameters due to
the difference on the discretization methods, although they converge approximately in the first 10 days
of therapy. Concluding, the tumor is eliminated in a period of two years and the bone mass density is
recovered in more three years approximately, assuming model validity.
59
60
Chapter 8
Conclusions
The aim of this dissertation was to develop a framework that could compute a drug dosing for optimal
cancer therapy with implications in bone metastasis. Accomplished this goal, here are made some
conclusion about the work and the results obtained as well as comments about the future work.
8.1 Work and results analysis
This thesis began by studying several mathematical models to represent the tumor growth as well as
the drug pharmacokinetics and pharmacodynamics models, to obtain a realistic representation of the
patient-drug relation. To compute the optimal drug dosage, an MPC algorithm was used, consisting on
a controller that solves an optimization problem with a receding horizon strategy in real time. Although
MPC is commonly known as an industry controller, nowadays it has been applied to a wide range of
applications. However, in those applications, it is common to want the system to follow a constant
reference signal. Usually, at the beginning of the simulation the system state is not near the reference,
which produces a high error and consequently the input signal will be high. In the context of this work,
the input signal is the drug dose, which should be minimized as much as possible due to its toxic effect.
Considering an exponential reference, which decays slowly in time, it allows the system to fetch the
reference and the controller to suggest lower input values.
Therapies main problem (and not only for cancer therapies) is the development of drug resistance.
This phenomenon is a natural process of the human body and is another factor that all therapies should
take into account. To represent this phenomenon, a drug resistance model was defined considering that
when the drug concentration is below a given threshold, drug resistance effect is developed proportion-
ally to that difference. Also, a multi-drug therapy was used with this framework, to combine multiple drug
effects that can be used to overcome the drug resistance phenomenon.
To do an online estimate of the patient model parameters, the RLS method was used, to better
traduce its behaviour. Thus, the MPC controller still works even if the model parameters are not well-
known. In fact, MPC is robust enough in the event of: errors between the system model and the real
system; errors on the measures of the system output; errors between the optimal drug effect computed
61
by the MPC and the actual drug effect, consequence of the PC model erros.
Finally, the developed framework was applied to a state-of-the-art research on bone marrow cancer.
This is a severe disease, because the tumor can interact with the bone microenvironment and spread
to other organs. Less than 20% of the patients with this disease survive for 5 years, thus, it is extremely
necessary to develop an effective therapy. Indeed, with the developed framework, and assuming model
validity, the tumor is eliminated in approximately two years and the bone mass is recovered in less than
five years.
8.2 Future work
When the project is simulated on a computer, the results are according to the expectations. Nevertheless
it would be interesting to deal with the dificulties of changing from the simulation environment to a real
environment. The pharmacodynamical models considered: pharmacokinetics, pharmacodynamics and
drug resistance, were strongly based on medical theories and it would be necessary to have real data, to
adjust the models parameters. For instance, the drug resistance model is based on the clim parameter,
which probably is not constant over time. Indeed, the drug resistance phenomenon in cancer diseases is
hard to understand due to the tumor dynamic adaptation to the environment where they are established.
As seen, if the drug concentration increases, the drug effect tends to saturate, meaning that even if
there are huge values of drug concentration in the bloodstream, the drug effect would be the same.For
this reason it would be also needed to study the importance of using an upper concentration limit. This
phenomenon is called drug toxicity, and it is difficult to understand how the drug concentration and tox-
icity relate. Indeed, there are no mathematical models in the toxicity bibliography representing those
interactions. However, the multi-drug therapy developed in this work can be used with the purpose of
adjusting drugs toxicity, by changing the tuning parameters ρ in the MPC optimization problem. The
therapy for the bone mass density recovery should also be optimized and the respective pharmacody-
namical model considered.
According to [5] and as seen in this work, the bone marrow cancer has the ability to deregulate
the healthy balance between the osteoclasts and osteoblasts in order to accelerate the bone resorption
process which, in turn, promote further tumor growth, known as the vicious cycle. Thus, the tumor
growth model presented in [7] should be review in order to model the interference that the osteoclasts
and osteoblasts activity have in the tumor growth dynamics.
Lastly, assuming that the framework would be ready to be used in clinical institutions, it would be
interesting to implement the developed framework on a low-cost platform, perhaps an Arduino or a
Raspberry PI, so every patient could have access to this type of personalized therapy.
62
Bibliography
[1] G. Tortora and B. Derrickson. Introduction to the Human Body: Essentials of Anatomy and Phisiol-
ogy. Wiley and Sons, 6:116–150, 2011.
[2] Amgen. www.youtube.com/watch?v=78rbpwsol08. Accessed on Jun 24, 15h37, 2015.
[3] F. Michor, Y. Iwasa, and M.A. Nowak. Dynamics of Cancer Progression. Nature Rev. Cancer, 4:
197–205, 2004.
[4] J. Domingues. Gompertz model: Resolution and Analysis for Tumors. Journal of Mathematical
Modelling and Application, I(7):70–77, 2012.
[5] Y. Zheng, H. Zhou, C. Dunstan, R. Sutherland, and M. Seibel. The role of the microenvironment in
skeletal metastasis. Jounal of Bone Oncology, 2:47–57, December 2013.
[6] D. Caiado and J. M. Lemos. Optimal Control for Cancer Therapy Design. Inesc-ID Thechical
Report, (6/2015), March 2015.
[7] B. Ayati, C. Edwards, G. Webb, and J. Wiskwo. A mathematical model of bone remodeling dynamics
for normal bone cell populations and myeloma bone disease. Biology Direct, 5, 2010.
[8] R. Martin and K. Teo. Optimal Control of Drug Administration in Cancer Chemotherapy. World
Scientific, 1993.
[9] A. Matveev and A. Savkin. Optimal Control Applied to Drug Administration in Cancer Chemother-
apy: the Case of Several Toxicity Constraints. 39a IEEE Conference on Decision and Control,
2000.
[10] T. Chen, N. Kirkby, and R. Jena. Optimal dosing of cancer chemotherapy using model predic-
tive control and moving horizon state/parameter estimation. Computer methods and programs in
biomedicine, Elsevier, 108:973–983, May 2012.
[11] P. Bumroongsri and S. Hheawhom. Optimal Dosing of Breast Cancer Chemotherapy using Robust
MPC Based on Linear Matrix Inequalities. Engineering Journal, 19(1), January 2015.
[12] J. Florian, J. Eiseman, and R. Parker. A Nonlinear model predictive control algorithm for breast
cancer treatment. Fakukltet for Naturvitenskap og teknologi, 2004.
63
[13] J.M. Lemos and D. Caiado. Receding Horizon Control of Tumor Growth Based on Optimal Control.
23rd Mediterranean Conference on Control and Automation, 2015.
[14] J.M. Lemos, J. Pinheiro, and S. Vinga. A Nonlinar MPC Approach to minimize toxicity in HIV-1
infection multi-drug therapy. Controlo, 2012.
[15] G. Xia and H. Wu. Neural Network Based Nonlinear Model Predictive Control for Ship Path Follow-
ing. 2013 Ninth International Conference on Natural Computation (ICNC), pages 210–215, 2013.
[16] M. Kadri M. Khalid. Liquid Level Control of Nonlinear Coupled Tanks System using Linear Model
Predictive Control. 2012 International Conference on Emerging Technologies (ICET), pages 1–5,
2012.
[17] J. Liu, Z. Huang, and C. Luo. Model Predictive Control of DC/DC Converter for Ultracapacitors
Energy Storage Union on T-S Model. 2014 IEEE Energy Conversion Congress and Exposition
(ECCE), pages 2316–2321, September 2014.
[18] M. Engelhart, D. Lebiedz, and S. Sager. Optimal control for selected cancer chemotherapy ODE
models: a view on the potential of optimal schedules and choice of objective function. Math Bio-
science, PubMed, (229(1):123-34), 2011.
[19] P. Dua, V. Dua, and E. Pistikopoulos. Optimal delivery of chemotherapy agents in cancer. Elsevier,
32(1,2):99–107, January 2008.
[20] S. Nanda, H. Moore, and S. Lenhart. Optimal control of treatment in a mathematical model of
chronic myelogenous leukemia. Elsevier, 210(1):143–156, November 2007.
[21] S. Jambhekar and P. Breenm. Basic Pharmacokinetics. Pharmacentral Press, 2009.
[22] Y. Lugmani. Mechanisms of drug resistance in cancer chemotherapy. Med Princ. Pract., PubMed,
(1:35-48):35–48, 2005.
[23] S. Komarova, R. Smith, S. Dixon, S. Sims, and L. Wahl. Mathematical model predicts a critical role
for osteoclasts autocrine regulation in the control of bone remodeling. Bone, Elsevier, 33:206–215,
April 2003.
[24] U. Ledzewicz and H. Schattler. A review of the optimal chemotherapy protocols: from MTD towards
metronomic chemotherapy. Mathematical Modelling of Natural Phenomena, 9(4), January 2014.
[25] U. Ledzewicz, H. Schattler, and H. Faraji. Optimal Protocols for a Mathematical Model of Tumor-
Immune Interactions chemotherapy with Immune Boost. Discrete and Continuos Dynamical Sys-
tems, 18(4)(Series B):1031–1051, 2012.
[26] S. Benzekry, C. Lamont, A. Beheshti, A. Tracz, J. Ebos, L. Hlatky, and P. Hahnfeldt. Classical Mathe-
matical Models for Description and Prediction of Experimental Tumor Growth. PLOS Computational
Biology, 10(8), August 2014.
64
[27] A. Laird. Dynamics of tumor growth. Unknown, pages 490–502, June 1964.
[28] K. Atkinson. An Introduction to Numerical Analysis. Wiley and Sons, 2, 1989.
[29] J. Wooten. Pharmacotherapy Considerations in Elderly Adults. Southern Medical Journal, 105(8):
437–445, 2012.
[30] K. Ogata. Modern Control Engineering. Prentice Hall, 5, 2009.
[31] M. Gottesman. Mechanisms of cancer drug resistance. Annu. Rev. Med., PubMed, (53:615-27),
2002.
[32] M. Liscovitch and Y. Lavie. Cancer multidrug resistance: A review of recent drug discovery re-
search. PharmaPress, 5(4):18–20, 2002.
[33] R. Haber, R. Bars, and U. Schmitz. Predictive Control in Process Engineering. Wiley and Sons,
2011.
[34] Wikipedia. https://en.wikipedia.org/wiki/model-predictive-control. Accessed on Dec 02, 10h52,
2014.
[35] A. Vahidi, A. Stefanoulou, and H. Peng. Recursive Least Squares with Forgetting for Online Es-
timation of Vehicle Mass and Road Grade: Theory and Experiments. Vehicle System Dynamics:
International Journal of Vehicle Mechanics and Mobility, 43(1):31–55, 2005.
[36] Cancer multidrug resistance. Nature Biotechnology, 18:18–20, 2000.
[37] P. Ducy, T. Schinke, and G. Karsenty. The Osteoblast: A Sophisticated Fibroblast under Central
Surveillance. Science Magazine, 289:1501–1504, September 2000.
[38] T. Phan and M. Zheng. Interaction between osteoblast and osteoclast: inpact in bone disease.
Histology and Histopathology, 19:1325–1344, 2004.
[39] Amgen. www.youtube.com/watch?v=gpmv197xzxc. Accessed on Jun 24, 16h34, 2015.
[40] S. Haykin. Adaptive Filter Theory. Prentice Hall, 3, 1996.
[41] J. Edwards and M. Weivoda. Osteoclasts: Malefactors of Disease and Targets for Treatment.
Discovery Medicine, March 2012.
65
66
Appendix A
Matrix algebra
A.1 Operations with matrices
Let A, B, C and D be matrices of the same size. The following rules are applied when summing or
multiplying matrices:
• A+B = B +A
• (A−B)(C −D) = AC −AD −BC +BD
Let A and B be matrices with the same size. The following rule is true when transposing matrices:
• (A+B)T = AT +BT
Let A and B be matrices and r ∈ < a scalar. The following rules are true when transposing matrices:
• (rA)T = rAT
• (AB)T = BTAT
67
A.2 Matrix inversion lemma
Lemma: Let A, D and [D−1 +CA−1B] be invertible squares matrices. Then A+BDC is invertible and
(A+BDC)−1 = A−1 −A−1B(D−1 + CA−1B)−1CA−1
Proof: If the equality is right, by multiplying both equation sides by A + BDC, the result has to be the
identity matrix I. The result of the first side multiplication is obvious, the second side multiplication result
is:
(A+BDC)(A−1 −A−1B(D−1 + CA−1B)−1CA−1) =
I +BDCA−1 −B(D−1 + CA−1B)−1CA−1 −BDCA−1B(D−1 + CA−1B)−1CA−1 =
I +BDCA−1 −BD(D−1 + CA−1B)(D−1 + CA−1B)−1CA−1 =
I
�
68
