ORIGINAL ARTICLE

Sensitivity to model structure: a comparison ofcompartmental models in epidemiology

Sheetal Prakash Silal1,Francesca Little1, Karen I.Barnes2 and Lisa Jane White3,4

1Department of Statistical Sciences, University ofCape Town, Cape Town, South Africa;2Department of Medicine, University of CapeTown, Cape Town, South Africa; 3Mahidol-OxfordTropical Medicine Research Unit, MahidolUniversity, Bangkok, Thailand; 4NuffieldDepartment of Clinical Medicine, Churchillhospital, University of Oxford, Oxford, UK

Correspondence: Sheetal Prakash Silal,Department of Statistical Sciences,University of Cape Town, Rondebosch,Rm. 5.52 P.D. Hahn Building, Cape Town,Western Cape 7700, South Africa.Tel: +27 21 6505321;Fax: +27 21 6504773;E-mail: Sheetal.Silal@uct.ac.za

Received: 21 August 2014Revised: 17 December 2014Accepted: 24 December 2014

AbstractCompartmental models have provided a framework for understanding diseasetransmission dynamics for over 100 years. The predictions from these models areoften policy relevant and need to be robust to model assumptions, parametervalues and model structure. A selection of compartmental models with the sameparameter values but different model structures (ranging from simple structuresto complex ones) were compared in the absence and presence of several policyinterventions to assess sensitivity to model structure. Models were fitted to datato assess if this might reduce this sensitivity. The compartmental modelsproduced wide-ranging estimates of outcome measures but when fitted to data,the estimates obtained were robust to model structure. This finding suggeststhat there may be an argument for selecting simple models over complex ones,but the complexity of the model should be determined by the purpose of themodel and the use to which it will be put.

Keywords: mathematical modelling; differential equations; sensitivity testing

The online version of this article is available Open Access

IntroductionAdvances in computational power have lead to mathematical modellingbeing used increasingly to solve real-world problems in all fields and levels ofdecision making. In epidemiology, mathematical models and in particularcompartmental models, have been used to explore (among other things) theemergence and spread of disease, and the impact and efficacy of interventionssuch as drug treatment, vaccine introduction and parasite control (Murray,2003). There is no single compartmental model structure that fits all diseasesand there are many different structures that may be used to model the samedisease. The results of these models are often policy relevant and in manycases used by policy makers to estimate populations at risk, design andimplement strategies to combat disease and monitor and evaluate on-goinginterventions. In this regard, models are and should be subjected to rigoroussensitivity testing (Chubb & Jacobsen, 2010). This testing process involvesidentifying parameters that strongly influence model outcomes and testingassumptions that when relaxed, strongly influence model results. Thesemodel assumptions can pertain to population size and initial conditionsamong other things. While sensitivity is assessed in the case of modelassumptions and parameter values, sensitivity to model structure is not oftenexplored, as the particular model structure is chosen in advance from a suiteof models. Studies exploring sensitivity to model structure includeRahmandad & Sterman (2008) who compared compartmental models withagent-based models to assess heterogeneity and network structure and Ferrer

Health Systems (2015), 1–14© 2015 Operational Research Society Ltd. All rights reserved 2047-6965/15

www.palgrave-journals.com/hs/

Health Systems (2016) 5, 178–191. doi:10.1057/hs.2015.2; published online 17 April 2015

(2016) 5, 178–191© 2016 Operational Research Society Ltd. All rights reserved 2047-6965/16

www.palgrave.com/journals

Sensitivity to compartmental model structure Sheetal Prakash Silal et al 3

Health Systems

et al (2012) who evaluated the impact of anti-malarialinterventions on compartment and agent-based models.Yet there may still be differences in model predictionsbetween compartmental models of the same disease withequivalent parameters but different model structure. Inthis regard, this paper explores the sensitivity of a selectionof epidemiologically relevant compartmental models thatdiffer only in model structure. While this may seemlimited, it serves to show that differences because of modelstructure may not only occur between classes of modelsbut within them as well. In this paper this sensitivityto model structure is assessed in a number of ways:differences in model predictions in the implementationof routine drug therapy, general vector control (VC), massdrug administration and whether these predictions differbetween models if the models have been fitted to dataor not. The next section provides an introduction tocompartmental models in epidemiology. The models aredeveloped in the section after that while the penultimatesection illustrates the results and the final section followsthe discussion.

Compartment models in epidemiologyThe first known contribution of mathematical modellingin epidemiology is Daniel Bernoulli’s work on the inocula-tion against smallpox in 1760 (Brauer et al, 2008). Ross,Halmer, Soper, Kermack and McKendrick all contributedto the application of compartmental models to epidemiol-ogy between 1900 and 1935 (Anderson & May, 1992). Themodel proposed by Kermack and McKendrick in 1927 hascome to be known as the Susceptible-Infective-Recovered(SIR) model with underlying equations:

dSdt

¼ - βSI (1)

dIdt

¼ βSI -1αI (2)

dRdt

¼ 1αI (3)

where t is time, S is the susceptible population (at risk ofinfection), I is the infectious population (capable of trans-mitting infection) and R is the recovered population(removed and playing no further role in the epidemic).β is the number of contacts per unit time and 1/α is the rateof recovery.This model results in a fixed population N (S+I+R)where

members of the population mix homogeneously (interactwith one another to the same degree). There is no entryinto or departure from the population as the dynamics ofthe disease are much faster than the time scale of birth anddeath processes; and hence the impact of these processeson the population can be ignored. Any inherent age,demographic and spatial structure is also ignored. There isno initial immunity as all ‘members’ of the susceptiblepopulation are equally likely to get infected. The modelinfers permanent immunity; once recovered, a secondinfection is impossible. The incubation period of the

infectious agent is instantaneous and the duration ofinfectivity is the same as the duration of the disease (oneis infectious as long as one has the disease). Discreteindividuals do not exist in the model and it is assumedthat individuals who reside in the compartments areidentical and as such variation among individuals isunimportant. Thus compartmental models are describedas population-level models. It is fractions of the popula-tion that flow between compartments and these move-ments are continuous. The rate of recovery 1/α is constantfor each ‘member’ of the population and hence theaverage duration of infectiousness (and in this case dis-ease) is α.There are several extensions of the SIR model, including

the Susceptible-Infectious (SI) model where immunity isignored (by excluding the Recovered compartment), theSusceptible-Exposed-Infectious-Recovered (SEIR) modelthat allows for a period of latency/exposure before becom-ing infectious, the Susceptible-Infectious-Recovered-Susceptible (SIRS) model that allows for temporary immu-nity and other similar models ( Jacquez, 1996). As diseaseshave different characteristics, these models may beextended to include biology (stages of immunity, vector/pathogen dynamics, super-infection), demography (birthand death processes, age, gender), interventions (drugtherapy, vaccines, VC) and geography (spatial structure,migration). Even the same disease may be modelled withvery different structures (Figure 1). In the case of malaria,Koella & Antia (2003) published SIRS models incorporat-ing resistance to drug therapy and super-infection, Yang(2000); Yang & Ferreira (2000) incorporated socio-economic and environmental factors into SEIR models forhosts and vectors (incorporating dynamics in an SEI modelfor mosquitoes), Torres-Sorando & Rodriguez (1997)included migration and visitation in an SIS model andAuger et al (2008) extended the Ross-Macdonald model toseveral patches in a meta-population compartmentalmodel (Ross, 1911; Mandal et al, 2011).In using several different structures to model the same

disease, it is of interest to assess if there are large differencesin model predictions because of model structure alone.This paper compares a selection of model structures andthe predicted impact of policy interventions. This paperalso explores whether fitting the models to data to deter-mine parameter values empirically, reduces sensitivity tomodel structure.

Model developmentDifferent model structures are compared for a disease thathas a latent period (L), a period where clinical symptomshave manifested but the host is not yet infectious (B), andan infectious period (I ) that does not grant immunity thatis, a person may be re-infected once susceptible (S) again.These models are compared in the absence and presence ofdrug therapy. Once infected and clinically ill, a patient hasthe potential to receive drug therapy and those who do notreceive drug therapy recover naturally that is, they do

179

Sensitivity to compartmental model structure Sheetal Prakash Silal et al2

Health Systems

not die from the disease but recover through the body’snatural defences at a period longer than the drug recoveryperiod. Patients may seek drug therapy when symptomshave manifested (they feel ill) as well as at the infectiousstage. The natural recovery period is assumed to be longerthan the drug recovery period and the time to infectious-ness, hence natural recovery is only possible once thedisease is at the infectious stage and not any earlier. Birthand death, super-infection and the development of immu-nity through repeated infections are ignored. Such adisease with a latent period and no immunity would needto be modelled using variants of an SEIS general modelstructure.Three models are compared in the absence of drug

therapy (SLI, SBI, SLBI) and five models are compared inthe presence of drug therapy (SLI, SBI, SLBI, stratified SBI,and stratified SLBI) (Figure 2). These models are comparedwith an alternate version of the SLBI model where theprobability of receiving treatment is applied at the time ofacquiring infection rather than during the infection, as analternative way of capturing the proportion of infectionsthat are treated (Model 6 in Figure 2). This alternate SLBImodel assumes a per-infection probability of treatmentwhereas the other models assume a per unit-time

probability of treatment. Differences between models areestimated by measuring incidence and prevalence of thedisease and the treatment coverage. This alternate SLBImodel is used to simulate data for data-fitting purposes soas to assess if validating models with data throughempirical estimation of parameter values reduces thesensitivity of the models to differences in model struc-ture. Finally policy interventions (VC and mass drugadministration) are also modelled to assess differences inmodel predictions because of model structure.The model parameters driving these models and their

assumed values are in Table 1. All models are from theSusceptible-Exposed-Infectious-Susceptible (SEIS) classof models where L is used to characterise the latent phaseof the disease. In Model 1 (SLI), the latent (L) andsymptomatic (B) stages of the disease have been com-bined; hence the rate of flow between Latent compart-ment and the Infectious compartment is 1/(σ1+σ2).When incorporating drug therapy into the models, onehas to consider at what stage of the disease the popula-tion has access to drug therapy. As the disease has alatent stage, it is only in the infectious compartment thatthe population may seek drug therapy and this occurs atthe rate p/(q+τ) (incorporating the treatment probability

Figure 1 Evolution and grouping of different types of SEIR malaria models. Subscripts ‘h’ and ‘m’ stands for human and mosquito.Double-folded boxes are for both human & mosquito population, and single fold boxes are only for human. First time addition of anew compartment is shown in grey script. The subscript ‘j’ (=1, 2, 3) indicates further subdivision of the corresponding compartment.Three models inside the big grey box are considered as the Basic malaria models. Dotted arrows show the incorporation of complexfactors in different models or specific compartment (encircled). Total population size is constant for all models, except the ones insidethe dashed box.Source: Mandal et al, (2011): published in Malaria Journal.

180

Sensitivity to compartmental model structure Sheetal Prakash Silal et al 3

Health Systems

(p), the time to seek treatment (τ) and the drug recoveryperiod). This rate naturally comprises two steps: treat-ment seeking and recovery through drug therapy. Asthere is no treated compartment in this model to

explicitly allow for this, the total time to move frombeing infectious to becoming susceptible again is q+τ andhence the population who receive drug therapy (withprobability p) do so at a rate of p×1/(q+τ). The populationthat is infectious but remains untreated recover naturallyat the natural recovery rate ((1−p)/δ).Models 2 (SBI) and 4 (Stratified SBI) do not have a

latent compartment, only compartments reflecting thesymptomatic (B) and infectious (I) stages of the disease,where the subscripts u and t in Model 4 representuntreated and treated infections respectively. Thesemodels account for the latent period by including atime delay in the force of infection (λ) of size σ1. Assymptoms have manifested (population feels ill) in thesymptomatic compartment (B), the population mayreceive drug therapy from both this and the infectiouscompartments. At the symptomatic stage, p% of thepopulation will seek and receive drug therapy (Model 2) orseek treatment (Model 4) while (1−p)% of the populationwill remain untreated and become infectious. In Model 2,the time to seek treatment is incorporated in thesame manner as for Model 1. In Model 4, owing to theinclusion of Symptomatic and Infectious compartmentsfor the treated population, the time to seek treatmentis incorporated explicitly. The latent and symptomatic stagesof the disease are captured separately in Models 3 (SLBI) and5 (Stratified SLBI) and treatment is incorporated in the sameway as Models 2 and 4 respectively. Model 6 stratifies thepopulation into the ‘never treated’ and those ‘destined tobe treated’ by multiplying the force of infection λ by thetreatment probability (p). Treatment can take place atboth the symptomatic and infectious compartments andoccurs at the rate 1/(q+τ) (incorporating the time to seektreatment and time to drug recovery) while theuntreated recover naturally at the natural recovery rate(1/δ). The equations underlying all these models aredisplayed in the Appendices.

a

b

c

d

e

f

Figure 2 Model flowcharts.; (a) Model 1: SLI (With treatment);(b) Model 2: SBI (With treatment); (c) Model 3: SLBI (With treat-ment); (d) Model 4: Stratified SBI (With treatment); (e) Model 5:Stratified SLBI (With treatment); (f) Model 6: Alternate StratifiedSLBI (With treatment).

Table 1 Model parameters: Description and value

Parameter Description Value Rate offlow

σ1 Period of latency 1 week 1/σ1σ2 Time to

infectiousness1 week 1/σ2

τ Time to seektreatment

0.5 week 1/τ

q Drug recovery period 1 week 1/qδ Natural recovery

period12 weeks 1/δ

p Probability oftreatment

0, 0.1, 0.5, 1

N Population Size 1000 peopleβ Contact Rate 10 per annum βλ Force of Infection β×I/N(Models 1, 2&3)

β×(Iu+It)/N(Models 4,5&6)

181

Sensitivity to compartmental model structure Sheetal Prakash Silal et al2

Health Systems

Models are compared using steady state measures ofincidence, prevalence and treatment coverage. Incidenceis measured as the number of new cases at each time step,prevalence is measured as the population infected with thedisease (L, B or I) at each time step and treatment coverageis estimated by cumulative treated cases as a proportion ofcumulative cases.

ResultsModels were fitted to data using least squares approach(Hansen et al, 2012) and differential equations weresolved using the linearised analytic method for ordinarydifferential equations. All models were programmed inR v3.02 (R Core Group, 2013). Model results are pre-sented under the conditions of no treatment, treatmentat different levels, and where external anti-disease inter-ventions are imposed on the models. These results arecontrasted between models that have been fitted to dataand models that have not.

No treatmentOf the six models, only models 1, 2 and 3 are tested underthe condition of no drug treatment. This is because whenthe probability of treatment (p) is 0, model 4 collapses tomodel 2 andmodels 5 and 6 collapse to model 3. Under thecondition of no treatment, prevalence in the three modelsis equivalent while incidence differs slightly among modelstructures (Figure 3). Incidence in model 2 is higher thanmodels 1 and 3 because the force of infection λ is a functionof the Infectious compartment and the rate of flow betweenthe Symptomatic and Infectious compartments is fasterthan that between the Latent and Infectious compartmentsin model 1. Similarly, incidence in model 3 is lower than inmodels 1 and 2 as the infectious reservoir is comparativelysmaller in model 3.

With treatmentFigure 4 shows that when imposing a 10% treatmentprobability on the infected population, models 1–5 predicta higher number of treated cases than model 6 and thetreatment coverage, as measured by cumulative treatedcases as a proportion of cumulative incidence, is estimatedcorrectly by model 6 (10%) but overestimated by the other5 models. A 10% probability of treatment implies a treat-ment coverage between 34 and 39% in models 1–5. Thetwo stratified models (4 and 5) behave similarly in thatthey have the same prevalence and treatment coverage.Likewise models 2 and 3 behave similarly. The treatmentcoverage is higher in all models compared with the model6 because these models predict a higher number of treatedcases and lower incidence. At a 50% treatment probabilitymodels 1–5 predict a lower number of treated cases thanmodel 6 and the treatment coverage is estimated correctlyby model 6 (50%) but underestimated by the other fivemodels (10.6–17.2%). This is because Models 1–5 alsopredict that incidence and prevalence will decrease to 0,albeit at different rates, while model 6 maintains stable

non-zero prevalence and incidence levels. At a 100%treatment probability, all models predict a decrease inincidence and prevalence to 0, again at varying rates, withmodel 6 decreasing to 0 at the slowest rate compared to theother models. These results are shown in the Appendices.

Treatment coverage and the probability of treatmentThe results in the section ‘With treatment’ show largedifferences in treatment coverage in all six models. It isonly in model 6 that the treatment coverage reflects thetreatment probability. Exploring why this may be sorequires understanding of how treatment coverage isestimated mathematically. Generally values of the para-meters that drive compartmental models may be sourcedfrom literature, take on assumed values or may be esti-mated from data. In modelling the impact of drug ther-apy, the proportion of infections that are treated(treatment coverage in this paper) is sometimes the bestsource of data available to mathematical modellers to useto estimate the treatment probability (the chance ofreceiving drug therapy) in their models. As shown in theresults, depending on the structure of the compartmentalmodel, these two values (treatment coverage and thetreatment probability) will not always be equal. Treat-ment probability p is used in the models to determine theproportion of the infected and infectious populationsthat get treated and treatment coverage is estimated fromthe models to determine cumulatively, the proportion ofcases that actually get treated.In order to assess the relationship between treatment

probability (p) and treatment coverage (denoted as π),consider a simple SIS model (S+I=1) for a cohort ofinfected individuals governed by the equation:

dIdt

¼ βIS -pq+1 - pδ

� �I (4)

0 200 400 600 800 1000

0

10

20

30

40

Weeks

Inci

denc

e

0 200 400 600 800 1000

0

100

300

500

Weeks

Pre

vale

nce

a

b

Figure 3 No treatment: Incidence and prevalence for Model 1(solid), Model 2 (dashed) and Model 3 (dotted).

182

Sensitivity to compartmental model structure Sheetal Prakash Silal et al 3

Health Systems

where β is the number of contacts per unit time, p is theprobability of treatment, 1/q is the drug recovery rate and1/δ is natural recovery rate. For ease of computation leta= p/q+(1−p)/δ and assume initial conditions of S(0)=0and I(0)=1.Using the result that in an SIS model, S=1−I,

dIdt

¼ βI 1 - Ið Þ - aI: (5)

Solving this differential equation amounts to solving,

ZI tð Þ

I 0ð Þ

dIβI 1 - Ið Þ - aI ¼

Zt0

ds: (6)

Using the initial condition I(0)=1, this integrates to

-ln Ið Þ - ln a + βI - βð Þ

a - β-ln að Þa - β

� �¼ t (7)

and hence

I ¼ a - βð Þe β - að Þt

a - βe β - að Þt : (8)

The treatment coverage (π) is a function of the cumula-tive treated cases (pI/q) and the cumulative cases (βIS).

In particular,

π ¼ limt!1

R t0pq IdtR t

0 βI 1- Ið Þ dt(9)

¼ limt!1

R t0pq

a - βð Þeðβ - aÞta - βeðβ - aÞt

� �dtR t

0 βa - βð Þe β - að Þta - βe β - að Þt

� �1 - a - βð Þe β - að Þt

a - βe β - að Þt

� �dt

(10)

¼ limt!1

pqβ ln βeβt - aeat

� �- at - ln β - að Þ� �

- a a - βð Þβ

βeβta βeβt - aeatð Þ +

ata - β +

ln βeβt - aeatð Þβ - a - β

a β - að Þ -ln β - að Þβ - a

� � :(11)

Using L’Hospital’s rule to simplify the limit equation(L’Hospital, 1696):

π ¼ limt!1

pqβ

β a - βð Þeβtaeat - βeβt

� �

- a a - βð Þβ

β2eβt eβt - eat� �

aeat - βeβtð Þ2 ! (12)

¼ limt!1

p- qa a - βð Þ ´

a - ββ

aeat - βeβt

eβt - eat

� �: (13)

0 200 400 600 800 1000

0

10

20

30

40

50

Weeks

Inci

denc

e

0 200 400 600 800 1000

0

100

300

500

Weeks

Pre

vale

nce

0 200 400 600 800 1000

0

10

20

30

40

Weeks

Tre

ated

cas

es

0 200 400 600 800 1000

0.0

0.1

0.2

0.3

0.4

0.5

Weeks

Tre

atm

ent C

over

age

a b

c d

Figure 4 10% Treatment Probability: (a) Incidence; (b) Prevalence; (c) Treated Cases and (d) Treatment Coverage for Model 1 (solid),Model 2 (dashed), Model 3 (dotted), Model 4 (dot-dash), Model 5 (long dash) and Model 6 (two dash).

183

Sensitivity to compartmental model structure Sheetal Prakash Silal et al2

Health Systems

Eq. (13) will tend to a finite limit depending on therelation between a (the average recovery rate) and β (thetransmission coefficient). To establish this relation, onecan look at the dynamics of the disease when the SISmodel has reached a steady state. At equilibrium, the sumof the inflows to a compartment is equal to the sum of itsoutflows. Thus for the infectious compartment (I) in an SISmodel,

βIS ¼ pq+1 - pδ

� �I (14)

Substituting a= p/q+(1−p)/δ and S=1−I into Eq. (14)leads to

βI 1 - Ið Þ ¼ aI (15)

This simplifies to

I ¼ β - aβ

: (16)

This system has two equilibria:

1. The trivial (infection-free) equilibrium2. The non-trivial equilibrium where infection is present

Both equilibria exist depending on the relationshipbetween β and a. If β<a, the trivial equilibrium is stablebut if β> a then the non-trivial equilibrium, the equili-brium where infection is present is non-negative andstable.Including β> a and multiplying through by e−βt/e−βt

leads to

π ¼ limt!1

p- qa a - βð Þ ´

a - ββ

ae a - βð Þt - β1 - e a - βð Þt

� �(17)

¼ p- qa a - βð Þ ´

a - ββ

´ - βð Þ (18)

¼ pqa

(19)

¼pq

pq +

1 - pδ

� � : (20)

Thus for an SIS model, treatment coverage (π) is notequal to the treatment probability (p), but rather a func-tion of the p, δ and q.This result holds for Model 1 but will change for

models with different structures (Models 2–5). The resultsin the section ‘With treatment’ showed that treatmentcoverage in Model 6 correctly represented the 10%treatment probability. This is because at equilibrium in

Model 6,

pλS ¼ 1σ1

L (21)

1σ1

L ¼ 1σ2

+1

q + τ

� �B (22)

1σ2

B ¼ 1q + τ

I: (23)

Therefore,

pλS ¼ 1q + τ

B + Ið Þ (24)

and,

p ¼1

q + τ B + Ið ÞλS

: (25)

Treatment coverage (π) at equilibrium is treated cases((B+I)/(q+τ)) as a proportion of incidence (λS) and hence forModel 6, π= p. Thus while treatment coverage data is oftenused by modellers as the best estimate of treatment prob-ability, depending on the model structure, the treatmentcoverage predicted by the model may be very differentfrom the treatment probability.

Data fittingShowing that for different model structures, treatmentcoverage is not always equal to treatment probability,demonstrates that even for routine anti-disease interven-tions like drug therapy, model structure has a large impacton model results. This section explores if fitting models todata reduces this sensitivity to model structure that is, ifmodels are fitted to data, does model structure matter lessor at all? Model 6 is used to simulate the number of treatedcases for data fitting purposes as this was the only one ofthe six models where p= π. Many national health systemscollect routine data on cases that are treated so this wouldbe typical of data available to mathematical modellers. It isnot usually the case that true incidence data would beavailable as the untreated infected population is generallyhidden from the health system. The treated cases from allfive models are fitted to the model 6 data estimating thevalue of p (0.1 in the model 6). All other parameter valuesare held constant at their assumed values in Table 1.Parameter estimation is achieved using the Least Squaresalgorithm for data fitting. All five models fit the data wellwith the estimated value of p, the probability of beingtreated, being in the range of 0.0063–0.0097. At thesevalues of p, the six models make identical predictions forprevalence, treated cases and treatment coverage with onlysmall differences observed in the prediction of incidence,thereby showing the decrease in sensitivity to modelstructure that can be achieved by fitting models to data.

Impact of other interventionsCompartment and other models of disease transmissionmay be used to assess the impact of policy interventionson transmission. Two interventions are applied to the

184

Sensitivity to compartmental model structure Sheetal Prakash Silal et al 3

Health Systems

six models to assess sensitivity because of model struc-ture; mass drug administration and VC. These inter-ventions are then scaled up, aimed at eliminating thedisease.

Mass drug administration Mass drug administrationinvolves administering drug therapy to the populationat risk regardless of disease status (Greenwood, 2010). Anexample of this intervention is reflected in the modelsby an increase in the treatment probability p. Mass drugadministration is implemented over 8 weeks at 75%probability in all models. The graphs on the left in Figure5 show the impact of this intervention on models thathave not been fitted to data while the graphs on the rightshow the impact this intervention has on the modelsonce they were fitted to the data from model 6. Theresults show that when mass drug administration isapplied to the fitted models, all models reach the sameequilibrium before the intervention and show anincrease in the number treated at the time of the

intervention but revert to the previous equilibriumeventually, whereas models that were not fitted to datareached different points of equilibrium. Even thoughthe fitted models achieve the same pre- and post-inter-vention equilibrium, the immediate impact (decrease inincidence and prevalence) of the intervention variesbetween model structures.

Vector control In some diseases like malaria and dengue,vectors are an active part of disease transmission. VC as anintervention, requires acting on the vector population in away that interrupts transmission. This may for example beachieved through larviciding, the use of insecticidetreated bednets and household spraying with pesticides(World Health Organization, 1997). In models 1–6, thisintervention is captured as having a decreasing effect on β;the number of contacts with vectors. The coverage of thecontrol as well as the efficacy of process are takeninto account in decreasing β where βVC= (1−cover-age*efficacy)*β. In these models an example of VC is

0 200 400 600 800 1000

0

20

40

Weeks

Inci

denc

e

0 200 400 600 800 1000

0

20

40

60

Weeks

Inci

denc

e

0 200 400 600 800 1000

0

200

500

Weeks

Pre

vale

nce

0 200 400 600 800 1000

0

200

500

Weeks

Pre

vale

nce

0 200 400 600 800 1000

0

20

40

60

80

Weeks

Tre

ated

cas

es

0 200 400 600 800 1000

0

50

150

Weeks

Tre

ated

Cas

es

0 200 400 600 800 1000

0.0

0.2

0.4

Weeks

Tre

atm

ent C

over

age

0 200 400 600 800 1000

0.0

0.2

0.4

Weeks

Tre

atm

ent C

over

age

a b

c d

e f

g h

Figure 5 Mass Drug Administration (8 weeks, 75% probability of treatment). (a & b) Incidence; (c & d) Prevalence; (e & f) TreatedCases and (g & h) Treatment Coverage for Model 1(solid), Model 2 (dashed), Model 3 (dotted), Model 4 (dot-dash), Model 5 (longdash) and Model 6 (two dash).

185

Sensitivity to compartmental model structure Sheetal Prakash Silal et al2

Health Systems

introduced into the models with 50% coverage and 50%efficacy. Figure 6 (left) shows that the different modelstructures (not fitted to data) measure the impact of VCvery differently; some showing a much greater impact onprevalence and incidence than others. Once the modelshave been fitted to data however, these differences areminimal (Figure 6 (right)).

Modelling to disease elimination When interventions areused to eliminate a disease rather than attempt to controlit, mathematical models will aim to produce estimates ofthe length of time to eliminate the disease, the intensityof each of intervention as well as the combinations ofinterventions that may be used to eliminate a disease.Elimination of infection is defined as the reduction to 0 oflocally acquired incidence in a geographical area as theresult of deliberate efforts (Dowdle, 1998). Two veryresource-intensive interventions aimed at eliminating adisease are tested on the six models: VC with a 90%coverage and at a 90% efficacy and a scaling up of drug

therapy from a 10% treatment probability to a 90% treat-ment probability. As the compartments and hence theflows between compartments will never actually reach 0to achieve elimination as outlined above, the disease isassumed to be eliminated when prevalence decreasesbelow 10−6. The impact of these interventions is esti-mated as the time to elimination (in weeks) from the startof the interventions. These interventions are tested onthe six models that have not been fitted to data and onthe models once they have been fitted to data.Table 2 shows that by scaling up VC, the models that are

not fitted to data predict widely varying time to elimina-tion values ranging from 193 weeks (<4 years) to 458weeks (<9 years). When fitted to data the predictions fallin a narrower range between 416 and 458 weeks. Theresults are quite different for a drug scale up intervention.When fitting the models to data, p, the probability oftreatment, is the estimated parameter from the data-fittingprocess. Thus when uniformly increasing the p to 0.9 atthe start of the intervention, the previously fitted valuefalls away and hence the time to elimination estimates are

0 200 400 600 800 1000

0

20

40

60

Weeks

Inci

denc

e

0 200 400 600 800 1000

0

20

40

60

Weeks

Inci

denc

e

0 200 400 600 800 1000

0

200

500

Weeks

Pre

vale

nce

0 200 400 600 800 1000

0

200

500

Weeks

Pre

vale

nce

0 200 400 600 800 1000

0

10

30

Weeks

Tre

ated

cas

es

0 200 400 600 800 1000

0

2

4

6

8

Weeks

Tre

ated

Cas

es

0 200 400 600 800 1000

0.0

0.2

0.4

Weeks

Tre

atm

ent C

over

age

0 200 400 600 800 1000

0.0

0.2

0.4

Weeks

Tre

atm

ent C

over

age

a b

c d

e f

g h

Figure 6 Vector control (50% Coverage, 50% Efficacy): (a & b) Incidence; (c & d) Prevalence; (e & f) Treated Cases and (g & h) Treat-ment Coverage for Model 1 (solid), Model 2 (dashed), Model 3 (dotted), Model 4 (dot-dash), Model 5 (long dash) and Model 6 (twodash).

186

Sensitivity to compartmental model structure Sheetal Prakash Silal et al 3

Health Systems

the same regardless of whether the models are fitted or not.These estimates range widely from 50 weeks (<1 year) to399 weeks (<8 years) depending on the model structure.These estimates are not robust and clearly sensitive tomodel structure.

DiscussionAs mathematical modelling is increasingly used to aiddecision making in the public health sector models needto be rigorously tested for sensitivity to parameter valuesand model assumptions so that model predictions arerobust (Chubb & Jacobsen, 2010). Incorporating thestandard intervention of drug therapy produced verydifferent results for all models tested in terms of preva-lence, incidence and the treatment coverage. With onlya 10% treatment probability, the decrease in prevalenceof the disease ranges between 3 and 49%. At highertreatment rates (> 50%), the disease was even eliminatedin some models. These differences occurred for modelswith the same parameter values and model assumptionsbut different model structures. The practical significanceof these differences is great in that reducing a diseasemarginally (3%) or reducing a disease by half have verydifferent impacts on a public health sector with minimalscarce resources and high opportunity cost on theseresources. This is reiterated by groups such as the Mal-ERA Consultative Group on Modelling, who recognisedthe contribution modelling can make to the eliminationof malaria globally and developed a framework of prior-ity areas for modelling to inform such as optimalresource allocation and expected timelines to achievegoals (The malERA Consultative Group on Modeling,2011). Model results need to be robust to model structureif they are to play a role in informing strategy and policydesign, where scarce resources will be committed on thebasis of the models’ predictions.Fitting the five models to data simulated from model 6

(to empirically estimate the treatment probability),shows that the sensitivity because of model structurecan be reduced. As countries become better at collectingroutine data and performing clinical trials, opportunitiesexist for modellers to validate their models with realdata. Data quality issues aside, modellers may still be

able to use their models to reproduce the data, as wellas validate model predictions with a testing dataset. Asmodels are extended to incorporate specific diseasedynamics such as immunity (Aron, 1988; Yang &Ferreira, 2000; Smith et al, 2006), vector populations(Dietz, 1980; Luz et al, 2010) and geography (Brauer,2001; Juan, 2006; Tumwiine et al, 2010), different modelstructures may well produce very different results andpredictions. Fitting models to data where possible canreduce the sensitivity of the model results to modelstructure.In attempting to eliminate the disease using VC, it was

found that fitting the models to data decreased thesensitivity to model structure, but this was not the casefor drug scale-up. The varying predictions on time toeliminate the disease can be partly explained by thetreatment seeking behaviour defined in each model. Inmodel 1, the population only has one chance at receiv-ing drug therapy (Infectious compartment), whereas inmodels 2 and 3, the population has two chances toreceive drug therapy (Symptomatic and Infectious com-partments) and hence it is harder to eliminate the diseasein model 1 than in models 2 and 3. The longest time toelimination estimate is from model 6 (the alternatestratified SLBI). In this model, a segment of the popula-tion has no chance to receive drug therapy in the courseof the infection and will have to wait until the nextround of infections to stand a chance of receiving drugtherapy. In this case, it is much harder to eliminate thedisease. Modellers should be aware of the treatment-seeking behaviour of the populations they model, butwhere this information is not available, there is anargument for modelling different treatment-seekingbehaviours so that this heterogeneity may be betterunderstood and prudent and conservative estimatesmay be produced. O’Connell et al (2012) found in theirstudy of malaria in Cambodia that treatment-seekingbehaviour is complex, often driven by cultural normsand other practicalities.Some of the models in this paper have a simple

structure while others have more complex structures.The results have shown that models seeking to measurethe same phenomenon (e.g. estimating the impact ofdrug therapy) that differ in structure only, can producevery similar results if they are fitted to the same set of data.Does the argument exist for choosing simple modelsover complex ones? The answer is yes, but the converseis also true. On one extreme one could have a model thatis too simple to be biologically plausible and on theother extreme, the most biologically plausible modelmay be too complex to be of any practical use. Thereneeds to be a balance between model simplicity andbiological plausibility for models to be of practicalrelevance. White et al (2009) compared the results of asimple deterministic compartmental model with otheragent-based stochastic complex models in a malaria-elimination context. They concluded that in situationswhere data is sparse, yet urgency exists to provide input

Table 2 Time to elimination (weeks) for vector controland scale-up of drug therapy

Time to Elimination (weeks) Vector Control Drug Scale-up

Not Fitted Fitted Not Fitted Fitted

Model 1 206 416 120 120Model 2 193 424 50 50Model 3 197 426 54 54Model 4 260 432 121 121Model 5 266 434 241 241Model 6 458 458 399 399

187

Sensitivity to compartmental model structure Sheetal Prakash Silal et al2

Health Systems

into strategy design, simple model structures are suitablebut complex models can provide more information (onthe context being modelled), especially in the long term.This paper sought to compare results between determi-

nistic compartmental models only and did not extend toinclude other model structures such as agent-based modelsand time series models. While this is a limitation of thepaper, it also serves to show that differences in resultsbecause of model structure arise not only among differentclasses of models but among models in the same class also.Several aspects have been ignored such as immunity, superinfection, chemoprophylaxis, drug resistance, heterogene-ity of the population and vector population dynamics.Stochasticity was also ignored. It was not in the scope ofthe paper to include all aspects of disease, but rather toshow that even in the most basic disease context, differ-ences still exist because of model structure. These are allareas where further work can be undertaken. Ultimately,model structure (and its complexity) needs to be chosencarefully depending on the focus of the model and the useto which it will be put.Compartmental model structures are based on the

underlying epidemiological and demographic interac-tions of a particular disease. Given that there are manychoices for these interactions, the number of possiblecombinations are large (Hethcote, 1994). This manu-script is one of few to compare a selection of thesemodels to assess their similarities and differences. Theprimary contribution of this paper is to show that not allmodels are equal. Differences exist even with the smal-lest changes in model structure and increases in com-plexity of models may result in different conclusionsbeing drawn from the model predictions. Gaining a clearunderstanding of these models enables one to choose a

structure that is suitable to the epidemiological processbeing modelled. Additionally the paper shows that fittingmodels to data can reduce the differences because ofmodel structure, but even so offers limited assistanceonce models are used for predictions. One purpose of amathematical model is to simulate a current epidemiolo-gical situation for the purposes of testing potential policyinterventions to assess their impact on the disease. Thisoften requires expanding on the existing model structureto accommodate these interventions. As interventionsmay be modelled in many ways, differences in modelpredictions may still result because of differences instructure only. This is often not considered. Fittingmodels to data will assist with simulating the currentepidemiological situation, but not with the predictionson the proposed policy interventions. These model pre-dictions can be made robust by introducing stochasticityinto the model to produce a range of plausible predic-tions. Second, the interventions themselves should bemodelled in a set of different structures to assess the rangein predictions and gain an understanding of the impactthat the chosen model structure has on the output.Ultimately, mathematical models will not all producethe same results and thus an in depth understanding ofthe model structure is necessary to maximise the useful-ness of these models.

AcknowledgementsThis material is based upon work supported financially bythe National Research Foundation in South Africa. Anyopinion, findings and conclusions or recommendationsexpressed in this material are those of the authors andtherefore the NRF does not accept any liability in regardthereto.

ReferencesANDERSON RM andMAY RM (1992) Infectious Diseases of Humans: Dynamics

and Control. Oxford University Press, Oxford, ISBN: 019854040X.ARON JL (1988) Acquired immunity dependent upon exposure in an SIRS

epidemic model. Mathematical Biosciences 88(1), 37–47.AUGER P, KOUOKAM E, SALLET G, TCHUENTE M and TSANOU B (2008) The

Ross-Macdonald model in a patchy environment. Mathematical Bios-ciences, 216(2), 123–131.

BRAUER F (2001) Models for transmission of disease with immigration ofinfectives. Mathematical Biosciences, 171(2), 143–154.

BRAUER F, VAN DEN DRIESSCHE P and WU J, (Eds.) (2008) Mathematical Epide-miology. Number 1945. 1st edn, Springer, Berlin. ISBN 3540789103.[WWW document] http://books.google.com/books?id=gcP5l1a22rQC&pgis=1.

CHUBB MC and JACOBSEN KH (2010) Mathematical modeling andthe epidemiological research process. European Journal of Epidemiology,25(1), 13–19.

DIETZ K (1980) Models for vector-borne parasitic diseases. In Vito VolterraSymposium on Mathematical Models in Biology (CALUDIO BARIGOZZI, Ed),pp 264–277, Springer, Berlin.

DOWDLE WR (1998) The principles of disease elimination and eradication.Bulletin of the World Health Organization 76(Suppl 2), 22–25.

FERRER J, ALBUQUERQUE J, PRATS C, LOPEZ D and VALLS J (2012) Agent-basedmodels in malaria elimination strategy design. In EMCSR 2012, February.Creative Commons License. [WWW document] http://is.upc.edu/

seminaris-i-jornades/documents/AgentbasedModelsinmalariaeliminationstrategydesign.pdf (accessed 19 August 2013).

GREENWOOD B (2010) Anti-malarial drugs and the prevention ofmalaria in the population of malaria endemic areas. Malaria Journal9(Suppl 3), S2.

HANSEN PC, PEREYRA V and SCHERER G (2012) Least Squares Data Fitting withApplications. JHU Press, Baltimore, ISBN 1421408589.

HETHCOTE HW (1994) A thousand and one epidemic models. In Frontiers inMathematical Biology (SIMON A LEVIN, Ed), pp 504–515, Springer, Berlin.

JACQUEZ JA (1996) Compartmental analysis in biology and medicine.BioMedware. [WWW document] http://books.google.com/books?id=U81qAAAAMAAJ&pgis=1 (accessed 19 August 2013).

JUAN Z (2006) Global dynamics of an SEIR epidemic model with immigra-tion of different compartments. Acta Mathematica Scientia 26(3),551–567.

KOELLA JC and ANTIA R (2003) Epidemiological models for the spread ofanti-malarial resistance. Malaria Journal 2(3).

L’HOSPITAL GF (1696) Analyse des infiniment petits pour l’intellligence deslignes courbes. L’imprimerie Royale, Paris.

LUZ PM, STRUCHINER CJ and GALVANI AP (2010) Modeling transmissiondynamics and control of vector-borne neglected tropical diseases. PLoSNeglected Tropical Diseases, 4(10), e761.

MANDAL S, SARKAR RR and SINHA S (2011) Mathematical models of malaria-a review. Malaria Journal, 10(1), 202.

188

Sensitivity to compartmental model structure Sheetal Prakash Silal et al 3

Health Systems

MURRAY JD (2003) Mathematical Biology II, Volume 18. Springer, Berlin.ISBN 0387952284. [WWW document] http://www.amazon.com/Mathematical-Biology-II-J-D-Murray/dp/0387952284.

O’CONNELL KA et al (2012) Souls of the ancestor that knock usout and other tales. A qualitative study to identify demand-sidefactors influencing malaria case management in Cambodia. Malariajournal, 11(1), 335.

RAHMANDAD H and STERMAN J (2008) Heterogeneity and network Structurein the dynamics of diffusion: Comparing agent-based and differentialequation models. Management Science, 54(5), 998–1014.

ROSS R (1911) The prevention of Malaria. John Murray, Albemarle Street,London.

R Core Group (2013) [WWW document] www.r-project.org (accessed19 August 2013).

SMITH T et al (2006) Mathe-matical modeling of the impact of malariavaccines on the clinical epidemiology and natural history of Plasmodiumfalciparum malaria: Overview. The American Journal of Tropical Medicineand Hygiene 75(suppl 2), 1–10.

The malERA Consultative Group on Modeling (2011) A research agendafor malaria eradication: modeling. PLoS medicine, 8(1), January. ISSN

1549-1676. doi: 10.1371/journal.pmed.1000403. [WWW document]http://journals.plos.org/plosmedicine/article?id=10.1371/journal.pmed.1000403.

TORRES-SORANDO L and RODRIGUEZ DJ (1997) Models of spatio-temporaldynamics in malaria. Ecological Modelling, 104(2–3), 231–240.

TUMWIINE J, MUGISHA JYT and LUBOOBI LS (2010) A host-vector model formalaria with infective immigrants. Journal of Mathematical Analysis andApplications, 361 (1), 139–149.

World Health Organization (1997) Vector control: methods for use byindividuals and communities. Technical report. [WWW document]http://www.who.int/malaria/publications/atoz/9241544945/en/index.html (accessed 19 August 2013).

WHITE LJ et al (2009) The role of simple mathematical models in malariaelimination strategy design.Malaria Journal, 8(8), 212.

YANG HM (2000) Malaria transmission model for different levelsof acquired immunity and temperature-dependent parameters(vector). Revista de saude publica, 34(3), 223–231.

YANG HM and FERREIRA MU (2000) Assessing the effects of globalwarming and local social and economic conditions on the malariatransmission. Revista de saude publica, 34(3), 214–222.

Appendix A

Model equationsModel 1 (SLI)

dSdt

¼ - λt S +1 - pδ

I +p

q + τI

dLdt

¼λtS -1

σ1 + σ2L

dIdt

¼ 1σ1 + σ2

L -1 - pδ

I -p

q + τI

Model 2 (SBI)

dSdt

¼ - λ t - σ1ð ÞS +1 - pδ

I +p

q + τB + Ið Þ

dBdt

¼λ t - σ1ð ÞS -1 - pσ2

B -p

q + τB

dIdt

¼ 1- pσ2

B-1 - pδ

I -p

q + τI

Model 3 (SLBI)

dSdt

¼ - λtS +1 - pδ

I +p

q + τB + Ið Þ

dLdt

¼λt S -1σ1

L

dBdt

¼ 1σ1

L -1 - pσ2

B -p

q + τB

dIdt

¼1 - pσ2

B -1 - pδ

I -p

q + τI

Model 4 (Stratified SBI)

dSdt

¼ - λ t - σ1ð ÞS +1 - pδ

IU +1q

BT + ITð Þ

dBU

dt¼λ t - σ1ð ÞS -

1 - pσ2

BU -pτBU

dIUdt

¼1 - pσ2

BU -1 - pδ

IU -pτIU

dBT

dt¼ pτBU -

1σ2

BT -1qBT

dITdt

¼ pτIU +

1σ2

BT -1qIT

Model 5 (Stratified SLBI)

dSdt

¼ - λtS +1 - pδ

IU +1q

BT + ITð Þ

dLdt

¼λt S -1σ1

L

dBU

dt¼ 1σ1

L -1 - pσ2

BU -pτBU

dIUdt

¼1 - pσ2

BU -1 - pδ

IU -pτIU

dBT

dt¼ pτBU -

1σ2

BT -1qBT

dITdt

¼ pτIU +

1σ2

BT -1qIT

189

Sensitivity to compartmental model structure Sheetal Prakash Silal et al2

Health Systems

Model 6 (Alternate Stratified SLBI)

dSdt

¼ - λt S +1δIU +

1q + τ

BT + ITð Þ

dLU

dt¼ 1 - pð ÞλtS - 1

σ1LU

dBU

dt¼ 1σ1

LU -1σ2

BU

dIUdt

¼ 1σ2

BU -1δIU

dLT

dt¼pλt S -

1σ1

LT

dBT

dt¼ 1σ1

LT -1σ2

BT -1

q + τBT

dITdt

¼ 1σ2

BT -1

q + τIT

0 200 400 600 800 1000

0

10

20

30

40

50

60

Weeks

Inci

denc

e

0 200 400 600 800 1000

0

100

300

500

Weeks

Pre

vale

nce

0 200 400 600 800 1000

0

50

100

150

Weeks

Tre

ated

cas

es

0 200 400 600 800 1000

0.0

0.1

0.2

0.3

0.4

0.5

Weeks

Tre

atm

ent C

over

age

a b

c d

Figure B1 50% Treatment Probability. (a) Incidence; (b) Prevalence; (c) Treated Cases and (d) Treatment Coverage for Model 1(solid), Model 2 (dashed), Model 3 (dotted), Model 4 (dot-dash), Model 5 (long dash) and Model 6 (two dash).

Appendix B

Additional output

190

Sensitivity to compartmental model structure Sheetal Prakash Silal et al 3

Health Systems

This work is licensed under a Creative Com-mons Attribution 3.0 Unported License. The

images or other third party material in this article areincluded in the article’s Creative Commons license, unlessindicated otherwise in the credit line; if the material is not

included under the Creative Commons license, users willneed to obtain permission from the license holder toreproduce the material. To view a copy of this license, visithttp://creativecommons.org/licenses/by/3.0/

0 200 400 600 800 1000

0

20

40

60

80

Weeks

Inci

denc

e

0 200 400 600 800 1000

0

200

400

600

800

Weeks

Pre

vale

nce

0 200 400 600 800 1000

0

50

150

250

350

Weeks

Tre

ated

cas

es

0 200 400 600 800 1000

0.00

0.10

0.20

0.30

Weeks

Tre

atm

ent C

over

age

a b

dc

Figure B2 100% Treatment Probability. (a) Incidence; (b) Prevalence; (c) Treated Cases and (d) Treatment Coverage for Model 1(solid), Model 2 (dashed), Model 3 (dotted), Model 4 (dot-dash), Model 5 (long dash) and Model 6 (two dash).

191