TBM ORIGINAL RESEARCH

A control systems engineering approach for adaptivebehavioral interventions: illustration with afibromyalgia intervention

Sunil Deshpande, Ph.D.,1 Daniel E. Rivera, Ph.D.,1 Jarred W. Younger, Ph.D.,2 Naresh N. Nandola, Ph.D.1,3

AbstractThe term adaptive intervention has been used in behavioralmedicine to describe operationalized and individuallytailored strategies for prevention and treatment ofchronic, relapsing disorders. Control systems engineeringoffers an attractive means for designing andimplementing adaptive behavioral interventions thatfeature intensive measurement and frequent decision-making over time. This is illustrated in this paper for thecase of a low-dose naltrexone treatment intervention forfibromyalgia. System identification methods fromengineering are used to estimate dynamical models fromdaily diary reports completed by participants. Thesedynamical models then form part of a model predictivecontrol algorithm which systematically decides ontreatment dosages based on measurements obtainedunder real-life conditions involving noise, disturbances,and uncertainty. The effectiveness and implications ofthis approach for behavioral interventions (in general)and pain treatment (in particular) are demonstrated usinginformative simulations.

Keywords

Adaptive behavioral interventions, Pain treatment,Fibromyalgia , Dynamical systems, Systemidentification, Control systems engineering, Modelpredictive control

INTRODUCTIONAdvances in understanding disease and developingefficacious treatments, coupled with the challenge ofrising health-care costs, have resulted in an increasedinterest within the field of behavioral medicine fordeveloping more effective strategies to treat chronic,relapsing disorders [1, 2]. Conventional clinical prac-tice is traditionally based on treatment plans designedfor a standard response that may not recognize indi-vidual participant characteristics or apply optimiza-tion procedures. During a typical treatment phase,clinicians assess the response of a participant to treat-ment by noting changes in symptoms or expectedoutcomes and suggesting changes in treatment dos-ages (if required). Many of these treatment strategiesare inspired from the acute care model and, inspite of efficacious treatments, are not necessarily

well suited for chronic disorders. Hence, even withclinical histories available, the process of assigningtreatment dosages by clinicians is based more orless on intuition and experience and may be notindividualized.One promising approach lies in adaptive behavioral

interventions; these adjust treatment dosages over thecourse of the intervention based on the values oftailoring variables, that is, measures of participantresponse or adherence [3]. In this paper, we describehow control systems engineering [4] offers a novel andpotentially powerful framework for optimizing theeffectiveness of broad classes of adaptive behavioralinterventions. Specifically, the development of deci-sion policies from control engineering coupled withtechnological enhancements in information and com-puter technology can result in adaptive interventionsthat minimize waste, increase compliance, and en-hance overall intervention potency [5–9]. By relyingon repeated assessments of participant response andmodel-based operationalized treatment decisions, it ispossible to ascertain the optimal treatment regimenfor an individual participant with less active

1Control Systems EngineeringLaboratory, School for Engineering ofMatter, Transport, and Energy,Arizona State University, Tempe, AZ85287, USA2Neuroinflammation, Pain andFatigue Laboratory, Department ofPsychology,University of Alabama atBirmingham, Birmingham, AL 35294,USA3ABB Corporate Research Center,Bangalore, IndiaCorrespondence to: D Riveradaniel.rivera@asu.edu

Cite this as: TBM 2014;4:275–289doi: 10.1007/s13142-014-0282-z

ImplicationsPractice: Adaptive interventions based on controlsystems engineering principles represent a valu-able practical approach for personalizing and opti-mizing treatment in behavioral interventions thatfeature intensive data collection and frequent deci-sion-making.

Research: Dynamical systems and control engi-neering provide a powerful, broad-based method-ological framework for modeling and decision-making in behavioral settings that can serve tobenefit modern time-varying, adaptive interven-tions.

Policy: Adaptive, time-varying interventionsbased on control systems engineering can substan-tially improve individual treatment outcomeswhile lowering costs and reducing negative effects.

TBM page 275 of 289

involvement of a clinician, ultimately improving inter-vention outcomes while lowering costs.Consider an illustration from a hypothetical

smoking cessation intervention. The participant is giv-en a smartphone to track her daily craving andsmoking, while taking a certain recommended fixedtreatment, such as a nominal dose of bupropion orbehavioral therapy scheduled at regular intervals. Thismode of operation can be termed as “open-loop” be-cause the treatment dosages are not adjusted based onparticipant response and may be suboptimal. Subse-quently, the intervention can be switched to a “closed-loop” system where a “controller” (or decision algo-rithm) alters dosages over time based on the informa-tion provided by the participant. The smartphone re-cords symptom severity and automatically suggestsproper dosage changes based on a model of partici-pant’s response to treatment, other assessment infor-mation, and clinical constraints. Data can still be sentperiodically to clinicians for review, and flags can beset to trigger alerts to the clinician in case of an unex-pected response. In this way, the best dosage levels canbe determined with minimal time required on the partof clinicians and participants. The computer (orsmartphone) can then continue to monitor symptoms(perhaps changing sampling frequency to once a weekor once month) and make changes periodically as theindividual’s response to treatment changes over time.It is important to note that in the closed-loop, thedecisions are made by incorporating dynamics of thebehavioral process, optimization on the basis of anobjective function criterion, and clinical constraintsas discussed in the rest of the paper.We illustrate how control engineering can accom-

plish this vision for behavioral interventions with thetreatment of fibromyalgia (FM), a complex, multi-symptom illness [10–12]. The approach is based on asecondary analysis of intensive longitudinal data col-lected in a previously conducted clinical trial that re-lied on low-dose naltrexone for the treatment of FM.We apply a data-based modeling technique from en-gineering known as system identification [13] to devel-op dynamical system models which model symptomchanges over time from daily diary reports completedby intervention participants. We apply an idiographicapproach [14, 15] and use these diary reports whichinclude self-assessments of outcomes of interest (e.g.,general pain symptoms, sleep quality) and additionalexternal variables that affect these outcomes (e.g.,stress, anxiety, and mood). The various psychosocialvariables measured in the intervention, along with thepharmaceutical arm, are used to explain changes inreported pain and other symptoms. The estimateddynamical systems models then serve as the basis forapplying a control technology known as model pre-dictive control (MPC) as a decision algorithm for au-tomatic dosage selection of naltrexone. A multipledegree-of-freedom formulation for MPC is presentedthat enables a clinician to adjust the speed at which adesired target pain level should be reached, with the

capability to adjust to both anticipated and unantici-pated changes in symptoms (as well as possible sideeffects) independently in the closed-loop system. Sim-ulation results are presented to illustrate the perfor-mance of the proposed decision scheme that incorpo-rates individual participant response, clinical con-straints, modeling errors, and variability typically pres-ent in a real-life application.While the treatment of FMwith naltrexone serves as

the primary example in this paper, similar procedurescan be applied to many other disease-treatment com-binations associated with behavioral medicine. Pub-lished work that examines aspects of the control sys-tems engineering approach described in this paper inother behavioral health settings includes work on pre-vention of conduct disorder [8, 16–18], promotion ofmoderate-to-vigorous physical activity [19], generalweight change and body composition [20], gestationalweight gain [21–24], and smoking cessation [25–28].The paper is organized as follows: the second sec-

tion describes an approach for dynamical systemsmodeling with an example from the secondary analy-sis performed on the naltrexone intervention data. Thethird section demonstrates the application of a closed-loop control for assigning treatment adaptively usingsimulations based on the models estimated in the sec-ond section. The paper ends with a summary andconclusions in the fourth section that includes a briefdiscussion on the design of experimental protocols forbehavioral interventions from a system identificationperspective, motivated by the results of this study.

A DYNAMICAL SYSTEMS APPROACH FOR MODELINGBEHAVIORAL INTERVENTIONSDynamical systems modeling considers how to char-acterize the transient response resulting from changesin manipulated inputs (e.g., intervention components,denoted by u) and disturbance inputs (e.g., externalinfluences which are not manipulated by the user,denoted by d) on outputs (e.g., proximal or distaloutcomes, mediators, denoted by y) measured in anintensive longitudinal setting. In a typical pain inter-vention, the input (u) can represent the dosage of aprimary intervention component like medication orcounseling, while a disturbance (d) can correspond tobehavioral constructs associated with the disorder thatinfluence the outcomes but are independent of treat-ment, for example, a reported level of anxiety or stress.The output (y) can be an outcome of interest which theintervention aims to modify such as reported pain orsleep quality. In Fig. 1, we show that these three vari-ables can be related to each other by the model Pwhich is a symbolic representation of a dynamicalsystem. P can be mathematically expressed using asystem of ordinary differential equations (in continu-ous time t) or a system of difference equations (insampled or discrete time k) among various other rep-resentations. A dynamical systems approach allows foran efficientmapping of the causal relationship between

ORIGINAL RESEARCH

TBMpage 276 of 289

variables by capturing the concepts of change andeffect in interventions. A representative dynamicalsystem that incorporates many of the responses seenin this study is

τ2d 2y tð Þd t2

þ 2ζτdy tð Þdt

þ y tð Þ ¼ K p τadu tð Þdt

þ u tð Þ� �

: ð1Þ

Since treatment dosages can change over time, theyare represented as function of time or u(t). Equation (1)relates changes in the output y(t) and its derivativeswith the changes in the input u(t) and its derivatives;Kp

represents the process gain of the system which corre-sponds to the overall steady-state effect following aunit “step” change in the input. The parameters τ(natural time constant) and ζ (damping ratio) are relat-ed to the speed and shape of the time-domain re-sponse. For example, if ζ is less than 1, this wouldresult in a decaying oscillatory response. The parame-ter τa (called the system zero) determines the initialdirection of the symptom report in response to a dos-age change.In Fig. 2, we show a hypothetical case where a unit

step change in the input u(t) is prescribed. The changein an outcome of interest y(t) (for example, the pain

Fig 1 | An input–output “block” diagram representation of the effect of input dosage (u) and disturbance (d) on output (y) using adynamical system (P) in an open-loop configuration. The input u (e.g., dosages) induces, based on the system dynamics P, aresulting output y (e.g., pain report) in the presence of external disturbances d (e.g., symptoms such as anxiety and stress)

0 10 20 2520

30

40

50

60

Out

put (

sym

ptom

rep

ort)

ζ < 1, τ

a < 0

ζ = 1, τa = 0

ζ > 1, τa > 0

ζ > 1, τa > ζ

0 10 20 25

0

0.5

1

Time (t)

Inpu

t (do

sage

s (m

g))

Fig 2 | Representative time-domain responses of an output following a “step” change in the input for a second-order systemaccording to Eq. (1). In this case, a unit change in treatment dosage at time t=0 results in a 20magnitude decrease in a symptomreport over time. Based on the systemdynamics, the responses range from a decaying oscillation with inverse response (solid blueline) to increasing sluggish, nonoscillatory response (red (dash), green (dot), and maroon (dash-dot))

ORIGINAL RESEARCH

TBM page 277 of 289

report provided by a participant) resulting from thisinput change can be observed. Different values of thecoefficients in the differential equation (as shown in thelegend of Fig. 2) result in responses of varying speedand shape; these range from a decaying-oscillatoryresponse over time to a more sluggish, nonoscillatoryresponse. Figure 2 also depicts an “inverse response”where τa is a negative number, which causes the re-sponse to initially increase prior to decreasing to a finalsteady-state value (solid blue line in Fig. 2). This tran-sient response can be further quantified using dynam-ical system properties such as the rise time Tr (which isthe time at which the response reaches 90 % of thesteady state for the first time [29]) and the settling timeTs (which is the time when the response reaches 98 %of its steady-state value [29]). Brevity prevents a moredetailed discussion, but we note that the second-ordersystem structure shown in Eq. (1) can represent ahighly diverse range of dynamical system responsesobserved in real life, among these mechanical, electri-cal, and process systems applications [29, 30].Themodel structure shown in Eq. (1) can be extend-

ed to behavioral settings and, in the experience of theauthors, representative of a large class of dynamicalsystem responses in behavioral medicine. This will beshown in the paper with respect to fibromyalgia, butmodels within the general class of second-order sys-tems have been observed in the modeling of smokingcessation [25], human weight and body compositionchange [20], and physical activity [19]. Statisticallysound, data-centric procedures will be presented toestimate and validate these models; the procedureswill similarly indicate when it is necessary to usehigher order structures.

The dataFM is a disorder characterized primarily by chronicwidespread pain. The characteristic symptoms of FMare diffuse musculoskeletal pain and sensitivity to me-chanical stimulation at soft tissue tender points [31,32]. Other important symptoms of FM include fatigue,sleep irregularities, bowel abnormalities, and cognitivedysfunction. There is no accepted diagnostic laborato-ry test for FM, and its etiology is largely unknown andwithout any scientific consensus [33], although thecondition is suspected to involve central sensitizationof pain processing [34]. As the causes for FM areuncertain, unknown, or disputed, and due to its chron-ic nature, it has been difficult to single out a specifictype of treatment for this disease. There is good evi-dence to suggest that naltrexone, an opioid antagonist,has a neuroprotective role and may be a potentiallyeffective treatment for diseases like FM [11, 35]. Thedata for this paper has been taken from clinical trials ofa low-dose naltrexone (LDN) intervention [11, 12].The study was conducted in two phases: a single

blind pilot study on 10 participants and a double blindfull study on 30 participants; the full study involved alonger protocol. A crossover design was employedwhere participants received both treatments and hence

act as their own control (i.e., each participant takesboth drug and placebo). A fixed naltrexone dose of4.5 mg concentration was administered. In the pilotstudy, the participants received placebo followed bydrug (P-D protocol), whereas in the full study, partici-pants were randomized to receive either drug first (D-Pprotocol) or placebo first (P-D protocol). The timeseries is split into a baseline (during which participantsdo not receive any kind of medication), followed byplacebo and drug (or vice versa), and finally a washoutphase during which all study medications are stopped.The number of data points range from 98 to 154sampled daily (T=1). Participants entered their re-sponses in a handheld computer to 20 questions suchas “Overall, how well did you sleep last night?” on ascale of 0–100. The continuous scale used for measur-ing variables in the FM study facilitates its suitabilityfor dynamical systemsmethods, as will be explained inmore detail later. The daily diary data consists of oneprimary endpoint “Overall, how severe have your FMsymptoms been today?” [FM sym] and 13 secondaryendpoints: fatigue, sadness, stress, mood, anxiety, sat-isfaction with life, overall sleep quality, trouble withsleep, ability to think, headaches, average daily pain,highest pain, and gastric symptoms [11, 12].

General description of variablesFrom an input–output dynamical systems viewpoint,one can broadly classify these variables from the FMclinical study as follows:

– Outputs (y): Since we are primarily interested inunderstanding the magnitude and speed at whichthe treatment component affects various FM symp-toms during the intervention, typical symptomslike pain, fatigue, and sleep problems correspondto dependent variables in the system and proximaloutcomes which we classify as outputs.

– Inputs (u,d): Drug and placebo are classified as theprimary inputs in this analysis, as they are externalto the system and their magnitude and duration canbe manipulated by the clinician; these are referredto by the symbol u. In addition to these primaryinputs, there are other exogenous disturbance var-iables d affecting the outputs. Certain variables suchas anxiety, stress, and mood are treated as mea-sured disturbance inputs that, when coupled withthe drug and placebo inputs, can help to betterexplain the variance in the output.

In Fig. 3, we show selected variables associated withthe naltrexone intervention for a participant from thepilot study; illustrating our proposed methodology byanalyzing data from this participant is the focus of thispaper; however, Deshpande [36] documents this anal-ysis for all 40 participants associated with the com-bined pilot and full study. The plot shows the primaryinputs (i.e., drug and placebo) and its resulting effecton outcomes such as FM symptoms and sleep. Vari-ables such as anxiety, stress, and mood are also shown.

ORIGINAL RESEARCH

TBMpage 278 of 289

It can be observed that with administration of naltrex-one (drug phase), the participant reports a significantdecrease in pain. Our aim is to find a parsimoniousdynamic representation for this relationship. Beforedescribing the modeling procedure, we briefly men-tion some of the challenges associated with this partic-ular dataset:

1. The limited amount of data collected during thestudy and the nature of the protocol create barriersto cross validation; care must be taken to avoidmodel overparameterization.

2. A lack of a priori knowledge about the system andthe possible presence of feedback between signalspresent challenges when classifying variables asinputs or outputs.

System identification procedure to model FM interventiondynamicsIn light of the unknown dynamics of FM, we apply anempirical modeling approach where input–output da-ta for each individual participant is used to build amodel describing the effect of drug and external fac-tors on FM symptoms. It should be stressed that thegoal of the modeling task is not to capture the detailedinternal mechanisms of FM, but rather to build aninformative empirical model describing how changesin drug dosage and external factors affect a number ofFM symptoms over time. The estimated model canserve a myriad of purposes, among them providing

predictions that can be used by a controller to assigndosages based onmeasured participant responses. Themodeling procedure undertaken in this study is sum-marized in three subparts as follows:

1. Data preprocessing. Initially, the data is preprocessedfor missing entries, and then to reduce the high-frequency (i.e., rapid day-to-day) variations in thedata, a 3-day moving average filter is applied.

2. Discrete-time modeling using multi-input ARX models.The filtered data is fitted to an autoregressive withexogenous input (ARX) parametric model (ARX[na nb nk]) defined through the linear differenceequation:

y kð Þ þ a1y k−1ð Þ þ…þ ana y k−nað Þ ¼ b11u1 k−nkð Þ þ…

þ bnb1u1 k−nk−nb þ 1ð Þ:::þ b1i ui k−nkð Þ þ…

þ bnb i ui k−nk−nb þ 1ð Þ:::þ b1nuunu k−nkð Þ þ…

þ bnbnu unu k−nk−nb þ 1ð Þ þ e kð Þ

ð2Þ

where nu represents the number of inputs; na, nb, and nkare model orders; e(k) is the prediction error; k is adiscrete time index (e.g., day); and

a1;…; ana ; b11;…; bnb1; b1i ;…; bnb i ; b1nu ;…; bnbnu� � ð3Þ

represents the model parameter vector estimated bythe regression. ARX model estimation constitutes alinear least-squares regression problem and has

0 10 20 30 40 50 60 70 800

50100

0 10 20 30 40 50 60 70 800

50100

0 10 20 30 40 50 60 70 800

50100

0 10 20 30 40 50 60 70 800

2040

0 10 20 30 40 50 60 70 8060

80100

FM Symptoms

Sleep

Anxiety

Stress

Mood

0 10 20 30 40 50 60 70 800

5

Dru

g (m

g)

Time (days)

0 10 20 30 40 50 60 70 800

1

Pla

cebo

DrugPlacebo

Fig 3 | Primary variables associated with naltrexone intervention of fibromyalgia as shown for a representative participant withplacebo-drug (P-D) protocol from the pilot study. With the introduction of naltrexone at day 27 for the participant, one notices asignificant decrease in FM symptoms and a substantial increase in sleep quality over time, suggesting a lagged dynamicalresponse to treatment

ORIGINAL RESEARCH

TBM page 279 of 289

favorable statistical properties when estimating thedynamics of a system such as consistency [13]. In thedetailed analysis of all participants done in [36], ARX[441] models were the highest order required; in manyinstances, models of lower complexity (such as ARX[221]) were suitable.The procedure for selecting which input signals

should be included in the model begins with thechoice of drug and placebo, which are expected tocontribute significantly to FM symptoms for all par-ticipants. Given the lack of a well-understood etiolo-gy for FM discussed previously, we take an approachdriven by system identification theory where addi-tional input variables are introduced sequentially suchthat they are minimally cross-correlated [37]. Whileincreasing the number of inputs improves the overallfit, in the absence of a cross-validation dataset, anexceptionally high fit may not necessarily imply ahighly predictive model. As the protocol applied inthis study did not allow for a cross-validation dataset,proper judgment on the choice of input variables thatadequately describes the data across all participantsmust be made. The “model fit” terminology used inthis paper points to the amount (percentage) of outputvariance explained by the model as:

model fit %ð Þ ¼ 100� 1−y kð Þ−by kð Þ

��� ���2

y kð Þ−y�� ��2

0@

1A ð4Þ

where y(k) is the measured output, by kð Þ is the simu-lated output, y is the mean of all measured y(k)values, and ‖⋅‖2 indicates a vector 2-norm. Finally,as a part of model validation, the residuals frommodel fitting are evaluated for whiteness, to confirmthat all the important dynamics have been capturedby the model structure [13].3. Simplification to a continuous time model. Step re-

sponses from the ARX model are individually fitto a parsimonious continuous second-order modelstructure of the form:

τ2d 2y tð Þd t2

þ 2ζτdy tð Þdt

þ y tð Þ ¼ K p τadu tð Þdt

þ u tð Þ� �

: ð5Þ

As noted previously, Eq. (5) provides importantdynamical system information such as gain, timeconstant, overshoot, rise, and settling times for eachinput which can be used to better understand partic-ipant response. These measures provide insights re-garding the strength and speed of response to an inter-vention component for an individual. The estimationprocedure applied in this step relies on predictionerror minimization to a continuous model structure,as implemented in the Process Models routine inMATLAB’s System Identification toolbox [38]. Acomparison of step responses between the ARXmod-el and the estimated continuous model can readily

determine if a higher order model than Eq. (5) isnecessary.

The estimation procedure requires that the data beuniformly sampled; this assumption is reasonable forconstructsmeasured on a daily basis and with standardapproaches applied to address any missing data (as isthe case in our fibromyalgia study). For cases involvingtruly irregularly sampled data, recent advances in con-tinuous time system identification could be incorpo-rated in the methodology described here [39].

Case study: representative participant from the pilot studyIn this subsection, we focus on the application of thesystem identification modeling procedure to a repre-sentative participant from the pilot study, with data asseen in Fig. 3. The following multi-input ARX [221]models (with FM symptoms treated as the primaryoutput and corresponding inputs noted below) areconsidered:

1. Model 1 (Drug)2. Model 2 (Drug, Placebo)3. Model 3 (Drug, Placebo, Anxiety)4. Model 4 (Drug, Placebo, Anxiety, Stress)5. Model 5 (Drug, Placebo, Anxiety, Stress, Mood)6. Model 6 (Drug, Placebo, Anxiety, Stress, Mood,

Gastric)7. Model 7 (Drug, Placebo, Anxiety, Stress, Mood,

Gastric, Headache)8. Model 8 (Drug, Placebo, Anxiety, Stress, Mood,

Gastric, Headache, Life)9. Model 9 (Drug, Placebo, Anxiety, Stress, Mood,

Gastric, Headache, Life, Sadness)

Figure 4 shows the corresponding evolution of fit formodels 1–9 which explain 46.57–79.69 % of the vari-ance in the output (daily symptom severity). As seen inFig. 4, beyond the five inputs that define Model 5,adding more input variables does not improve the fitsignificantly and results in overparameterization.Hence, we rely on the input variables from Model 5as our base for multi-input ARX models for this par-ticipant which captures the effect of drug, placebo,anxiety, stress, and mood variables on outcomes ofinterest. It has a gain parameter Kp=−2.47, indicatinga nearly 2.5 point decrease in the pain report permilligram dose of naltrexone (see Table 1). The nega-tive gain associated with administration of naltrexoneallows us to classify this participant as a responder totreatment. A rise time (Tr) of slightly over 5 days wasseen. The 98 % settling time (Ts) of nearly 11.5 dayscharacterizes the naltrexone speed of response for thisparticipant. The overall response resulting from themodel is an example of an overdamped responseshowing no oscillations [29]. Table 1 also shows howadditional inputs improve the goodness-of-fit and con-trasts the Akaike information criterion (AIC) with thegoodness-of-fit metric defined in Eq. (4).Table 2 summarizes the model parameters for all

inputs (manipulated and disturbance) for the Model 5

ORIGINAL RESEARCH

TBMpage 280 of 289

structure. The positive gain for the placebo input indi-cates that for this participant, pain symptoms wors-ened when placebo was administered. The large mag-nitude of the placebo gain is partly a consequence ofhow the input signal is coded (1 when present and 0when not). Examining the gains for the measureddisturbance models (anxiety, stress, and mood), thesecorrespond to 0.86, 2.29, and −0.091, respectively.The positive values for the anxiety and stress gains

and negative value for mood agree with the clinicalobservations that increases in anxiety and stress anddecrease in mood should worsen FM symptoms. Ta-ble 2 also includes the model relating the effect ofnaltrexone on sleep. The positive gain in this modeldemonstrates improved sleep quality in this partici-pant as a result of treatment. The rise and settling timesassociated with the sleep outcome are approximately37 % longer than those for general FM symptoms,

Fig 4 | Evolution of model fit with input addition to the ARX [221] model. The percent fits for eachmodel are shown in parenthesis.No significant improvement in model fit is obtained beyond Model 5

Table 1 | Model parameter tabulation corresponding to the drug-FM input–output pair for different multi-input models for aparticipant from the pilot study

Model %fit AIC Kp, τ, ζ, τa Tr (days) Ts (days)

1 46.5 3.64 −12.03, 5.67, 4.14, 21.3 75.5 139.692 59.2 3.58 −0.91, 3.5, 2.67, 44.4 0.43 75.063 64.7 3.54 −1.02, 2.09, 1.5, 15.3 0.43 25.64 71.8 3.42 −3.11, 1.62, 1.24, 0.22 7.53 14.385 73.9 3.44 −2.47, 1.57, 1.26, 1.96 5.12 11.49AIC Akaike information criterion, Tr the rise time (the time at which the response reaches 90% of the steady state for the first time), Ts the 98% settling time (thetime at which the response reaches 98 % of its steady-state value)

ORIGINAL RESEARCH

TBM page 281 of 289

which implies that sleep benefits lag general FM symp-tom improvements.

Dynamic modeling for behavioral interventions: beyond blackbox approachesIn earlier sections, we described an empirical “blackbox” modeling approach for determining the processdynamics associated with the fibromyalgia interven-tion where the choice of the model structure has beenprimarily driven by aspects of goodness-of-fit andoverparameterization of clinical data. However, it ispossible and in many instances desirable to incorpo-rate theories from behavioral science in a dynamicalsystems framework relevant to interventions [40, 41].The interested reader is directed to work using thetheory of planned behavior (TPB) to develop a dy-namic model for a behavioral intervention for weightloss [20]; meanwhile, the combination of TPB and self-regulation to model the dynamics of an adaptive inter-vention for gestational weight gain is explored in [21–24]. The use of self-regulation for modeling smokingcessation dynamics is explored in [25–28], while adynamical model for social cognitive theory (SCT) inthe context of improving physical activity interven-tions is described in [41].

“CLOSED-LOOP” BEHAVIORAL INTERVENTIONS USINGCONTROL SYSTEMS ENGINEERINGWe noted in the “INTRODUCTION” that the ulti-mate goal of this research is to accomplish closed-looptreatment in which the control system assigns appro-priate dosage magnitudes automatically over time.Control systems are used widely in industrial practiceto achieve desired behavior of a system by systemati-cally adjusting manipulated variables based on mea-sured system information [4, 29, 42]. Prior work [8] hasestablished that time-varying adaptive interventionsfeaturing repeated assessments, intensive data, andfrequent decision-making can be conceptualized asengineering control systems. In a control engineeringapproach to adaptive interventions, the controller as-signs dosages to each participant as dictated by thesolution of a formal optimization problem which fullyincorporates the parameters or predictions from a dy-namical model. The optimization problem is solvedtaking into account clinical constraints and relies on

measurements (of both outcomes of interest and symp-toms reports, collectively known as tailoring variables)provided by or assessed from the participant.To conceptually illustrate this adaptive treatment

strategy, the open-loop dynamical system shown inFig. 1 is extended by “closing” the loop using a con-troller. In the block diagram shown in Fig. 5, P repre-sents the dynamics expected of the treatment interven-tion on outcomes of interest; this is the same block asin Fig. 1. The controller, represented by the symbol C(which is part of the decision algorithm and does notrepresent the clinician) is supplied with clinical con-straints and a cost function providing a performancemetric for the optimization problem. Based on thediscrepancy between the measured outcomes y (e.g.,pain reports) and their desired reference values and thevalues of current and future symptom reports (as pre-dicted by the model), the controller assigns dosages uto achieve a closed-loop response for each participant[8, 30]. The closed-loop control system aims atperforming the following three functional tasks:

1. Reaching a desired goal (setpoint tracking). Treatmentdosages are assigned to move an outcome of inter-est (such as pain report or sleep quality) to a desiredgoal. For example, the clinician may decide on agoal of 45 % reduction in the general pain symp-toms report within 2 weeks of the start of theintervention.

2. Known symptoms handling (measured disturbance rejec-tion). The controller manipulates treatment dosagesto mitigate the effect from reported or assessedexternal influences (e.g., anxiety in case of FMintervention) using estimated disturbance models.For instance, if some external event that leads tostress or anxiety is known a priori, then dosagescan be adjusted to compensate for this distur-bance. It should be noted that the control engi-neering approach does not preclude providingthe participant with a separate intervention thatdirectly targets anxiety. However, as no treat-ment is perfect, if changes in anxiety levels leadto changes in FM symptoms, then it makessense to include this information as a feedforwardadjustment in the control system. The controlalgorithm possesses the functionality to allowthis feature to be eliminated or “detuned” (asdiscussed later in the paper).

Table 2 | Model parameter tabulation for various input-FM models for a participant from the pilot study as per Model 5. Theestimated gain for the drug-FM model is negative, implying that a reduction in pain symptoms with the introduction of drug, whilethe gain for placebo-FM model is positive, implying worsening outcomes as a result of placebo. The table includes the case of thedrug-sleep model, where the use of drug improves sleep quality

Model Kp, τ, ζ, τa Tr (days) Ts (days)

Drug-FM −2.47, 1.57, 1.26, 1.96 5.12 11.49Placebo-FM 45.81, 1.57, 1.26, 1.15 6.59 13.06Anxiety-FM 0.86, 1.57, 1.26, 0.24 7.45 14.24Stress-FM 2.29,1.57, 1.26, 0.49 7.31 13.94Mood-FM −0.091, 1.57, 1.26, 4.67 0.8 11.93Drug-OSleep 4.98, 2.13, 1.04, −3.35 7.06 15.83

ORIGINAL RESEARCH

TBMpage 282 of 289

3. Unknown symptoms handling (unmeasured disturbancerejection). The controller can adjust treatment dos-ages to mitigate the effect of unknown orunmodeled external influences. For example, therecould be an increase in a pain report that may notbe directly associated with a change in a measuredor reported condition (such as stress or anxiety). Insuch cases, the controller through the process offeedback compensation is able to adjust dosagesto mitigate the effects of this unmeasureddisturbance.

The three functional modes of the control systemhave to be achieved under a number of practical clin-ical requirements, and hence, this functionality has tobe integrated into the controller design. In clinicalpractice, treatment dosage limits are often set to min-imize adverse effects, such as drug toxicity. In addition,treatment dosages are generally categorical in nature.For example, counseling sessions can either be weekly,bi-weekly, or monthly. Similarly, drug dosages arecompounded in standard dosage concentration, andsubsequent increase in the dosage can be prescribed asan integer multiple of that basic dose (for example, acommercially produced drugmay be available only in100, 150, and 300 mg dosages). These dosage limits

are factored into the control algorithm. Furthermore,dosage changes should not be very abrupt due topotential negative consequences that the participantmay experience. Hence, the controller should betuned in such a way that dosing can be varied from amore aggressive case, where treatment dosages changerapidly over a relatively short period of time, to amoreconservative case where treatment dosages changerelatively slowly over time.

Model predictive controlIn this work, we describe MPC as the algorithmicframework for making systematic dosage assign-ments. MPC has widespread application in indus-try, ranging from chemical process control toaerospace engineering [43]. This control technol-ogy has also been useful in designing treatmentregimens for diverse medical applications, fromdiabetes mellitus control to HIV/AIDS treatment[16, 44–47].Figure 6 depicts the “receding horizon” strategy

that is the basis for the model predictive controlalgorithm. The predicted change in FM symptomseverity over time (calculated using the estimated

Fig 5 | Conceptual “block diagram” representation of a control system showcasing a closed-loop treatment strategy using adesired reference for a designated cost function and clinical constraints. This figure extends the concept introduced in Fig. 1 by“closing the loop.” The dosages u are assigned by the controller C based on cost function and clinical constraints to takeoutcomes to a desired reference setpoint or goal

ORIGINAL RESEARCH

TBM page 283 of 289

models from system identification) with predic-tion of measured variables like anxiety (if avail-able) determines an error projection which showsactual and expected deviations in symptoms fromthe goal. On the basis of this error projection,the optimizer chooses the sequence of future con-trol actions which minimize the error with re-spect to a desired goal; this process is repeatedfor each sample instant. The control actions arecalculated by an on-line optimization algorithmas follows: the constrained optimization problemshown in Eq. (6) is numerically solved at eachtime instant (e.g., daily) based on model-basedprediction over a period of time (defined by theprediction horizon p) and optimal treatment dos-ages over a period of time (defined by the movehorizon m). Instead of using all of the recom-mended treatment dosages, only the first of thecalculated dosage is applied, with the processrepeated at the next assessment period (e.g., dai-ly) until the end of the intervention. Hence, thealgorithm is able to respond to unexpected symp-tom changes as it systematically relies on up-to-date participant response information. In Eq. (6),the optimization objective is related to the errorbetween the predicted values (y(k+1),…,y(k+p))and the reference setpoint yr, with Qy as a userdefined weight. It is solved under constraints onthe allowable minimum and maximum outcomevalues, input treatment dosages, and their rate ofchange (represented by Δu(k)=u(k)−u(k−1)).

Variables p and m correspond to the predictionhorizon and the move horizon, respectively.

u k þ ið Þ½ �m−1i¼0

n omin J△ ¼

Xi¼1

p

y k þ ið Þ−yr� �T Q y y k þ ið Þ−yr

� �ymin≤y k þ ið Þ≤ymax; 1≤ i≤p

umin≤u k þ ið Þ≤umax; 0≤ i≤m−1

Δumin≤Δu k þ ið Þ≤Δumax; 0≤ i≤m−1 ð6Þ

The solution to the optimization problem denotedin Eq. (6) is accomplished through established numer-ical procedures from operations research. For lineardynamical models with categorical inputs, the optimi-zation problem in Eq. (6) is solved using a mixedinteger quadratic program (MIQP). Details of the al-gorithm and its solution are provided in [16].The control algorithm relies on a three degree-of-

freedom (3 DoF) tuning approach to flexibly achievedesired levels of performance [48, 49]. The 3 DoFtuning methodology enables the three performancerequirements associated with reaching a desired goaland known and unknown symptoms handling to beadjusted independently by varying three “knobs” rep-resented by the parameters αr, αd, and fa, respectively[16, 36, 44]. This tuning approach provides the user aflexible and intuitivemethod to adjust the controller sothat the outputs achieve a desired speed and shape ofresponse. In the following section, we demonstratehow the 3 DoF formulation gives the flexibility toobtain desirable participant treatment profiles overtime, while accommodating clinical requirements.

Fig 6 | Receding horizon strategy used by themodel predictive control algorithm illustrated for the case of naltrexone interventionwith one controlled variable (FM symptoms), one manipulated variable (naltrexone dosage), and one measured disturbance(anxiety report). Control moves (i.e., dosage change decisions) are calculated by the algorithm over a horizon and only the firstcontrol move calculated using the optimization procedure is implemented. The entire procedure is repeated at the nextassessment period and continues until the end of the intervention

ORIGINAL RESEARCH

TBMpage 284 of 289

Closed-loop control simulationIn this section, we demonstrate an adaptive treatmentscenario on the representative participant modeledfrom the pilot study. For simulation purposes, we con-sider a version of Model 5 described previously withone manipulated variable (drug), one disturbance var-iable (anxiety), and one output (FM symptoms). Nal-trexone dosages are assumed to be available at eightlevels equally spaced between 0 and 13.5 mg. The FMsymptoms variable serves as the primary outcome inthe analysis, while anxiety (assumed to be reporteddaily by the participant) serves as the measured (orknown) disturbance signal.Figure 7 illustrates the operation of the control system

for adaptive drug dosage assignment under deterministicconditions. In this simulation, we show the three func-tional tasks that this controller can perform: reaching asetpoint goal with dosage change and dosage changes forknown and unknown symptoms. Setpoint tracking(reaching the goal) starts at day 0, while the measureddisturbance (anxiety report) acts between days 20 and 40,and the unmeasured disturbance (an unknown causeresulting in an increase in reported pain) acts at day 55.The baseline value for the pain report is 50 %, and asetpoint change of magnitude −9.5 is applied at day 0 asshown by the reference line (dashed red line). The painreport reaches 95% of the setpoint in 8 days and the finalgoal in 11 days. Themeasured anxiety report acts in sucha manner that it has an effect for a limited period of time(e.g., the increase in anxiety can be attributed to somestressful event). The controller increases the drug dosages

to mitigate the effect of increased anxiety, with the re-sponse reaching the setpoint in around 7 days. There isan increase in the pain report (initially), but this returns tothe desired symptom baseline. When the anxiety reportdiminishes, the controller can correspondingly reducedosage amounts while staying within the pain thresholdlevel acceptable to the participant. In this way, the con-troller responds favorably to participant feedback byefficiently varying dosages over time. At day 55, anabrupt change in the pain report occurs due to an un-measured disturbance (e.g., a psychosocial stressor that isunrelated to the measured symptoms). In this case, weobserve that since the stressor is unknown (and hence nota part of participantmodel), a sudden increase in the painreport is seen to which the controller responds by in-creasing the drug dosage to bring back the pain level tothe desired goal. The response reaches the setpoint inapproximately 14 days.In the 3 DoF formulation, the user/clinician can vary

themagnitude of the three knobs (αr for setpoint tracking,αd for measured disturbance rejection, and fa for unmea-sured disturbance rejection) between 0 and 1 to reach at adesired system response. The value of the knobs deter-mines the speed of the closed-loop response. However, afaster response may result in drug dosing being tooaggressive for participant comfort; hence, a propertrade-off between the speed of response and correspond-ing dosage profile has to be determined based on theintervention goals. The effect of tuning changes isdiscussed in [16] and illustrated for this simulated inter-vention in more detail in [36] and [50].

0 10 20 30 40 50 60 70

40

45

50

Res

pons

e

FM sym

0 10 20 30 40 50 60 700

1.923.855.767.68

9.611.58

13.5

Dru

g S

tren

gth

(mg)

Input drug

0 10 20 30 40 50 60 700

10

20

Time (day)

d(k)

Measured Disturbance (Anxiety)

Fig 7 | Deterministic simulation of adaptive closed-loop naltrexone dosage assignment by the model predictive controller usingeight drug dosage levels. The controller assigns dosages to reach a target pain level while addressing the effects of known andunknown symptoms on this pilot study participant. The three tuning knobs (αr, αd, and fa) are all equal to 0.5; p=25,m=15,Qy=1.To reach the desired goal (a pain report of 40.5 %), the controller assigns corresponding dosages which have to be adjustedbased on the values of known and unknown symptom reports

ORIGINAL RESEARCH

TBM page 285 of 289

The simulation in Fig. 7 considered a scenario with adeterministic disturbance. A scenario in which thedisturbance is of stochastic nature (generated by anARMA (2,1) model driven by a Gaussian noise) isshown in Fig. 8. The subplot on the top shows the timeseries for an unmeasured disturbance, and the corre-sponding figure on the bottom represents the simula-tion under two different tuning settings and a constantdosage. The performance of these interventions is mea-

sured by the tracking error J e ¼ ∑N −1

k¼0

y kð Þ−yr� �2

;

total change in drug dosage JΔu ¼ ∑N −1

k¼0

Δu kð Þ2;and total amount of drug dosage consumed in the

intervention JΔu ¼ ∑N −1

k¼0

u kð Þ: Setting f a equal to 1

results in a more aggressive control action in which thevariance around the pain target is minimized, but thisis accomplished at the expense of a large variation indrug dosage changes. A detuned controller with f a

equal to 0.1 reduces the variance in drug dosagechanges at the expense of increased variance in theparticipant’s pain report as noted in Table 3. In prac-tice, f a must be adjusted to accomplish a desired trade-off between dosage changes and reported symptoms,as deemed acceptable by the clinician and the partic-ipant. Finally, the dosage profile under tuning f a=0.1is compared with a fixed dose equal to 1.92 mg tohighlight the benefits of adaptation in the presence ofunmeasured disturbances. The adaptive interventionoffers lower tracking error while also consumingless treatment compared to the fixed dosage case,as noted in Table 3.

0 10 20 30 40 50 60 70 80 90

−30

−20

−10

0

10

20

30

Time (days)U

nmea

sure

d D

istu

rban

ce (

on a

sca

le o

f 100

)

0 10 20 30 40 50 60 70 800

20

40

60

Res

pons

e

FM sym

0 10 20 30 40 50 60 70 80

0

1.92

3.85

5.76

7.68

9.6

11.58

13.5

Time (day)

Dru

g S

tren

gth

(mg)

Input drug

fa=1

fa=0.1

ReferenceConstant

Fig 8 | Simulation for an unmeasured stochastic disturbance. A simulation showing change in the FM symptoms report for anunmeasured stochastic disturbance (a) is shown for the representative pilot study participant under closed-loop control (b). The drugdosages are assigned by themodel predictive controller on eight levels where the controller adjusts drug dosages to compensate forchanges in the general FM symptoms reports. Based on the tuning value (fa), the resulting drug dosage changes can vary fromaggressive to conservative. The fixed dosage is set at 1.92 mg. Other controller and simulation parameters remain as in Fig. 7

ORIGINAL RESEARCH

TBMpage 286 of 289

In industrial practice, the robustness of a controlsystem to modeling errors and other forms of uncer-tainty is an important consideration [4, 42]. In thecontext of this application, robustness issues arise fromthe statistical errors resulting from parameter estima-tion during system identification, the variability thatmay exist between participants, and changes in themodel that may occur within the participant over time.Space limitations prevent us from describing theseconcepts in more detail, but robustness of the controlsystem is evaluated in a series of simulated scenariosrelevant to the adaptive intervention that are describedin [36] and [50].

SUMMARY AND CONCLUDING THOUGHTSIn this paper, we have described how techniques fromdynamical systems and control engineering can be used toachieve more efficacious treatments with increased poten-cy by enabling adaptive behavioral interventions. In thisapproach, a parsimonious dynamical systems model thatexplains the relationship between treatment and symp-toms is estimated from intensive participant data. Fromthese estimated models, an adaptive “closed-loop” dosageassignment system can be developed. The closed-looptreatment algorithm solves a receding horizon optimiza-tion problem that effectively integrates predictions fromthe dynamic model with repeated assessment of partici-pant outcomes and symptoms to accomplish a personal-ized, optimal dosage profile over time.To illustrate this concept, the paper focused on an

example of an adaptive intervention designed to deter-mine appropriate dosage levels of naltrexone as a treat-ment for fibromyalgia. Because of a lack of understand-ing of fibromyalgia etiology and the absence of first-principles models, we performed a secondary data anal-ysis to estimate parsimoniousmodels from data availablethrough clinical trials. Amulti-input ARXmodel, furtherapproximated with continuous second-order differentialequation models, was used to explain the effect of drug,placebo, and other variables on symptom reports ofinterest. Subsequently, models from a representative par-ticipant were used by a model predictive control algo-rithm to systematically assign dosages in the presence ofdisturbances and clinical constraints. We illustrated howthe controller can assign treatment dosages to reach adesired set point target and thenmaintain this goal underconditions that involve disturbance changes in symptoms(known and unknown). This capability was demonstrat-ed for both deterministic and stochastic disturbances.

Although the data and illustrations in this paperrelied primarily on data collected from self-reports,the methodology presented can be applied withoutmodification to data obtained from mHealth-relatedtechnologies. Increasing advances in the computingand information technologies associatedwithmHealthimply that the dynamical systems and optimizationmethods described in this paper will become moreimportant (and commonplace) in behavioral settingsin the future. Ultimately, these improvements will al-low, wherever possible, for more direct and accuratemeasurement of human behavior and its environment,with corresponding impact in the development ofmore relevant and predictive models of behaviorchange [6, 51].One important consideration in extending the use-

fulness of these methods is the role of experimentaldesign. Traditional population-level clinical data isgenerally not amenable for constructing dynamicalmodels as these are typically designed for “static”systems and are geared toward hypothesis testing andfinding treatment efficacy. There has been an increas-ing interest in using “n-of-1” or single subject experi-mental protocols; these designs are highly individual-ized and, hence, offer certain advantages over popula-tion-level design [52–54]. Typically, treatment dosagesremain constant throughout the duration of a conven-tional trial, which limits the effectiveness of systemidentification techniques. In a system identificationapproach to single subject experimental design, thetreatment dosage is varied over time to multiple signallevels, the changes designed to provide sufficient exci-tation and consequently resulting in richer informationregarding the dynamics of the system. Optimizationapproaches come into play as the experiments mustincrease the information content in the data whilerespecting clinical constraints such as limits on treat-ment dosage values and the rate of change of dosagesover time. The work in [50, 55] discusses such ap-proaches in detail.In conclusion, we envision that the concepts pre-

sented in this paper will lead to novel individualizedtreatments where treatment dosages can be assignedby a mobile device at the disposal of the participant.The closed-loop dosage assignment can adjust treat-ment dosages based on daily participant reports ofoutcomes of interest and by incorporating predictivebehavioral theories. In this way, an optimal dosageprofile for that individual could be rapidly determinedwithout requiring office visits or substantial clinicianinvolvement. Dynamical systems modeling and

Table 3 | Comparison of the performance of the intervention from the control system (fa=1,0.1) with a fixed dosage of naltrexone(1.92 mg) under stochastic disturbances. The control system offers lower tracking error Je for the cost of higher variability in drugdosage JΔu. In comparison with a constant dosage, the case fa=0.1 also offers lower total drug consumption Ju

Scenario Je JΔu Ju

MPC (fa=1) 6,204 1,205.1 275.79MPC (fa=0.1) 9,211.3 55.791 144.64Constant drug dosage 12,328 3.6864 170.88

ORIGINAL RESEARCH

TBM page 287 of 289

control engineering offer a rich set of theoretical toolsto address various demands of practice. Ultimately, wesee adaptive interventions using control engineering asa cost -effect ive and eff icient approach foraccomplishing personalized behavioral interventions.

Acknowledgments: Support for this work has been provided by the Officeof Behavioral and Social Sciences Research (OBSSR) of the National Institutesof Health (NIH) and the National Institute on Drug Abuse (NIDA) through grantsR21 DA024266 and K25 DA021173. The content is solely the responsibility ofthe authors and does not necessarily represent the official views of OBSSR,NIDA, or the NIH. J. W. Younger received support from the American Fibromy-algia Syndrome Association (AFSA). Insights provided by L. M. Collins and J.Trail of the Methodology Center, Penn State University during the conduct ofthis research are greatly appreciated.Conflict of interest and adherence to ethical standards statement:The authors have no conflicts of interest to disclose. This paper presented a de-identified secondary data and simulation analysis of two previously executedclinical studies performed in accordance to ethical standards and protection forhuman subjects.

1. Hamburg MA, Collins FS. The path to personalized medicine. N EnglJ Med. 2010;363(4):301-304.

2. Wellstead P, Bullinger E, Kalamatianos D, Mason O, Verwoerd M.The role of control and system theory in systems biology. Annu RevControl. 2008;32(1):33-47.

3. Collins LM, Murphy SA, Bierman KL. A conceptual framework foradaptive preventive interventions. Prev Sci. 2004;5:185-196.

4. ÅströmK,Murray R. Feedback Systems: An Introduction for Scientistsand Engineers. Princeton: Princeton University Press; 2009.

5. Chakraborty B, Murphy SA. Dynamic treatment regimes. Annu RevStat Appl. 2014;1(1):447-464.

6. Riley WT, Rivera DE, Atienza AA, Nilsen W, Allison SM, MermelsteinR. Health behavior models in the age of mobile interventions: areour theories up to the task? Transl Behav Med. 2011;1:53-71.

7. Rivera DE. Optimized behavioral interventions: what does systemidentification and control engineering have to offer? In: Proceedings of16th IFAC Symposium on System Identification; 2012: 882–893.

8. Rivera DE, Pew MD, Collins LM. Using engineering control principlesto inform the design of adaptive interventions: a conceptual intro-duction. Drug Alcohol Depend. 2007;88(Supplement 2):S31-S40.Customizing Treatment to the Patient: Adaptive TreatmentStrategies.

9. Zafra-Cabeza A, Rivera DE, Collins LM, Ridao MA, Camacho EF. Arisk-based model predictive control approach to adaptive interven-tions in behavioral health. IEEE Trans Control Syst Technol.2011;19(4):891-901.

10. Boissevain MD, McCain GA. Toward an integrated understanding offibromyalgia syndrome. I. Medical and pathophysiological aspects.Pain. 1991;45(3):227-238.

11. Younger J, Mackey S. Fibromyalgia symptoms are reduced by low-dose naltrexone: a pilot study. Pain Med. 2009;10(4):663-672.

12. Younger J, Noor N, McCue R, Mackey S. Low-dose naltrexone for thetreatment of fibromyalgia: findings of a small, randomized, double-blind, placebo-controlled, counterbalanced, crossover trialassessing daily pain levels. Arthritis Rheum. 2013;65(2):529-538.

13. Ljung L. System Identification: Theory for the User. 2nd ed. UpperSaddle River: Prentice Hall; 1999.

14. Molenaar P, Campbell C. The new person-specific paradigm inpsychology. Curr Dir Psychol Sci. 2009;18:112-117.

15. Velicer W. Applying idiographic research methods: two examples.In: Proceedings of the 8th International Conference on TeachingStatistics; 2010.

16. Nandola NN, Rivera DE. An improved formulation of hybrid modelpredictive control with application to production-inventory systems.IEEE Trans Control Syst Technol. 2013;21(1):121-135.

17. Pina AA, Holly LE, Zerr AA, Rivera DE. A personalized and controlsystems engineering conceptual approach to target childhood anx-iety in the contexts of cultural diversity. J Clin Child Adolesc Psychol.2014;43(3):442-453.

18. Davison DE, Vanderwater R, Zhou K. A control-theory reward-basedapproach to behavior modification in the presence of social-normpressure and conformity pressure. In: Proceedings of the 2012American Control Conference; 2012: 4076–4052.

19. Hekler EB, Buman MP, Poothakandiyil N, et al. Exploring behavioralmarkers of long-term physical activity maintenance: a case study ofsystem identification modeling within a behavioral intervention.Health Educ Behav. 2013;40(1 suppl):51S-62S.

20. Navarro-Barrientos JE, Rivera DE, Collins LM. A dynamical model fordescribing behavioural interventions for weight loss and body com-position change.Math Comput Model Dyn Syst. 2011;17(2):183-203.

21. Dong Y, Rivera DE, Thomas DM, Navarro-Barrientos JE, Downs DS,Savage JS, Collins LM. A dynamical systems model for improvinggestational weight gain behavioral interventions. In: Proceedings ofthe 2012 American Control Conference; 2012: 4059–4064.

22. Dong Y, Rivera DE, Downs DS, Savage JS, Thomas DM, Collins LM.Hybrid model predictive control for optimizing gestational weightgain behavioral interventions. In: Proceedings of the 2013 AmericanControl Conference; 2013: 1973–1978.

23. Savage JS, Downs DS, Dong Y, Rivera DE. Control systems engineer-ing for optimizing a prenatal weight gain intervention to regulateinfant birth weight. Am J Public Health. 2014;104(7):1247-1254.

24. Dong Y, Deshpande S, Rivera DE, Downs DS, Savage JS. Hybridmodel predictive control for sequential decision policies in adaptivebehavioral interventions. In: Proceedings of the 2014 AmericanControl Conference; 2014: 4198–4203.

25. Timms KP, Rivera DE, Collins LM, Piper ME. A dynamical systemsapproach to understanding self-regulation in smoking cessationbehavior change. Nicotine Tob Res. 2014;16(Suppl 2):S159-S168.

26. Timms KP, Rivera DE, Collins LM, Piper ME. Continuous-time systemidentification of a smoking cessation intervention. Int J Control.2014;87(7):1423-1437.

27. Timms KP, Rivera DE, Piper ME, Collins LM. A hybrid model predic-tive control strategy for optimizing a smoking cessation interven-tion. In: Proceedings of the 2014 American Control Conference;2014: 2389–2394.

28. Trail JB, Collins LM, Rivera DE, Li R, Piper ME, Baker TB. Functionaldata analysis for dynamical system identification of behavioralprocesses. Psychol Methods. 2014;19(2):175-187.

29. Ogata K.Modern Control Engineering. Upper Saddle River: PrenticeHall; 2001.

30. Ogunnaike BA, Ray WH. Process Dynamics, Modeling, and Control.Oxford: Oxford University Press; 1994.

31. Wolfe F, Clauw D, Fitzcharles M, et al. The American College ofRheumatology preliminary diagnostic criteria for fibromyalgia andmeasurement of symptom severity. Arthritis Care Res. 2010;62:600-610.

32. Wolfe F, Smythe HA, Yunus M, et al. The American College ofRheumatology 1990 criteria for the classification of fibromyalgia:report of the multicenter criteria committee. Arthritis Rheum.1990;33:160-172.

33. Perrot S. Fibromyalgia syndrome: a relevant recent construction ofan ancient condition? Curr Opin Support Palliat Care.2008;2(2):122-127.

34. Lee YC, NassikasNJ, ClauwDJ. The role of the central nervous systemin the generation and maintenance of chronic pain in rheumatoidarthritis, osteoarthritis and fibromyalgia. Arthritis Res Ther.2011;13(2):211.

35. Mattiloi TM, Milne B, Cahill C. Ultra-low dose naltrexone attenuateschronic morphine-induced gliosis in rats. Mol Pain. 2010;6(22):1-11.

36. Deshpande S. A control engineering approach for designing anoptimized treatment plan for fibromyalgia. Master’s thesis, ElectricalEngineering, Arizona State University, USA; 2011.

37. Gevers M, Miskovic L, Bonvin D, Karimi A. Identification of multi-input systems: variance analysis and input design issues.Automatica. 2006;42(4):559-572.

38. The Mathworks. System Identification Toolbox, MATLAB User Man-ual for version R2009b; 2009.

39. Garnier H, Young PC. The advantages of directly identifying contin-uous-time transfer function models in practical applications. Int JControl. 2014;87(7):1319-1338.

40. Hekler EB, Klasnja P, Froehlich JE, Buman MP. Mind the theoreticalgap: Interpreting, using, and developing behavioral theory in HCIresearch. In: Proceedings of the SIGCHI Conference on Human Fac-tors in Computing Systems, CHI ’13; 2013: 3307–3316.

41. Martin CA, Rivera DE, Riley WT, Hekler EB, Buman MP, Adams MA, KingAC. A dynamical systems model of social cognitive theory. In: Proceed-ings of the 2014 American Control Conference; 2014: 2407–2412.

42. Skogestad S, Postlethwaite I. Multivariable Feedback Control: Anal-ysis and Design. Hoboken: Wiley; 1996.

43. Qin SJ, Badgwell TA. A survey of industrial model predictive controltechnology. Control Eng Pract. 2003;11(7):733-764.

44. Deshpande S, Nandola NN, Rivera DE, Younger J. A control engineer-ing approach for designing an optimized treatment plan for fibro-myalgia. In: Proceedings of the 2011 American Control Conference;2011: 4798–4803.

45. Lee H, Buckingham B, Wilson D, Bequette B. A closed-loop artificialpancreas using model predictive control and a sliding meal sizeestimator. J Diabetes Sci Tech. 2009;3(5):1082-1090.

46. Wang Y, Dassau E, Doyle F. Closed-loop control of artificial pancre-atic β-cell in type 1 diabetes mellitus using model predictive itera-tive learning control. IEEE Trans Biomed Eng. 2010;57(2):211-219.

47. Zurakowski R, Teel AR. A model predictive control based schedulingmethod for HIV therapy. J Theor Biol. 2006;238(2):368-382.

ORIGINAL RESEARCH

TBMpage 288 of 289

48. Lee JH, Yu ZH. Tuning of model predictive controllers for robustperformance. Comput Chem Eng. 1994;18(1):15-37.

49. Wang W, Rivera DE. Model predictive control for tactical decision-making in semiconductor manufacturing supply chain manage-ment. IEEE Trans Control Syst Technol. 2008;16(5):841-855.

50. Deshpande S. Optimal input signal design for data-centric identifi-cation and control with applications to behavioral health and med-icine. Ph.D. thesis, Electrical Engineering, Arizona State University,USA; 2014.

51. NilsenW, Pavel M.Moving behavioral theories into the 21st century:technological advancements for improving quality of life. IEEE Pulse.2013;4(5):25-28.

52. Hersen M, Barlow DH. Single-Case Experimental Designs: Strategiesfor Studying Behavior Change. Oxford: Pergamon; 1976.

53. Lillie EO, Patay B, Diamant J, Issell B, Topol EJ, Schork NJ. The n-of-1clinical trial: the ultimate strategy for individualizing medicine?Personal Med. 2011;8(2):161-173.

54. Dallery J, Cassidy RN, Raiff BR. Single-case experimental designs toevaluate novel technology-based health interventions. J Med Inter-net Res. 2013;15(2):1-17.

55. Deshpande S, Rivera DE, Younger J. Towards patient-friendly inputsignal design for optimized pain treatment interventions. In: Pro-ceedings of the 16th IFAC Symposium on System Identification;2012: 1311–1316.

ORIGINAL RESEARCH

TBM page 289 of 289