Paper III

1

Multinomial Markov-chain model of sleep architecture in Phase Advanced Subjects

C. Steven Ernest II1,2

, Roberto Bizzotto3, David J DeBrota

2, Lan Ni

2, Cynthia J. Harris

2, Mats

O. Karlsson1, Andrew C. Hooker

1

(1) Department of Pharmaceutical Biosciences, Uppsala University, Sweden;

(2) Eli Lilly and Company, Indianapolis, IN, USA;

(3) Institute of Biomedical Engineering, National Research Council, Padova, Italy.

To whom correspondence should be addressed:

C. Steven Ernest II

Lilly Research Laboratories, Eli Lilly and Company, Lilly Corporate

Center, Indianapolis, IN 46285

Telephone: (317) 276-2350

Email: cse@lilly.com

Abstract The phase advanced sleep model is used to induce transient insomnia, where subjects go to sleep

several hours before their usual bedtime disrupting their normal sleep architecture. The ability of a

drug to allow a subject to sleep during this otherwise normal wake time may predict efficacy in

insomnia patients. The aims of this work were to: (1) model sleep stage transition probabilities from

polysomnography data (PSG) in phase advanced subjects (PAS) over 13 hours after placebo

administration, and (2) compare the transition probabilities in PAS to insomniac patients to identify

differences in sleep architecture between these two populations for the first 8-hours. Transition

probabilities for PAS from two placebo-controlled, parallel studies at two different sites were modeled

using a recently reported mixed-effect Markov-chain model based on transition probabilities as

multinomial logistic functions in insomniac patients examined after placebo dosing. The multinomial

Markov-chain model robustly described phase advanced sleep over the 13-hour observation period

after placebo dosing. Compared to insomniac patients, PAS generally displayed lower transition

frequencies, fell asleep quicker, spent less total time in REM and ST2, displayed a higher tendency to

awaken during early portions of the night and generally, displayed different sleep architectures. The

multinomial mixed-effect Markov-chain model provides a useful tool for analyzing sleep data in PAS

and may therefore prove useful in the analysis of PSG data from clinical sleep studies investigating

sleep promoting drugs using the phase advanced model as a surrogate for efficacy in insomniac

patients. Keywords: Markov; multinomial; sleep; NONMEM.

Introduction

Insomnia is a Central Nervous System

(CNS) disorder that affects a large number of

individuals. Approximately 10% to 15% of

adults suffer from chronic insomnia, and an

additional 25% to 35% suffer from transient or

occasional insomnia (1-3). Sleep maintenance

insomnia, or the inability to stay asleep

throughout the night, is significantly more

prevalent than sleep onset insomnia, and better

medications are needed to address this issue.

Inducing phase advanced sleep involves

moving bedtime 2-5 hours earlier than the

subject’s usual bedtime. This time period is

also known as the “forbidden zone” for sleep,

when the circadian arousal system promotes

alertness and reduces the likelihood of sleep.

These experiments produce reliable sleep

disruption mimicking transient insomnia in

subjects with normal sleep patterns (4-5). A

key advantage of phase advanced sleep is the

ability to produce the type of transient

insomnia which prevents people from

obtaining adequate sleep. Thus, inducing phase

advanced sleep is one way to test a drug’s

efficacy in altering sleep architecture in

subjects under a sleep laboratory-based

protocol. The ability of a drug to allow a

subject to sleep during this otherwise normal

Paper III

2

awake time demonstrates efficacy, and several

drugs with sedating properties have been

evaluated using this methodology (5-6).

In a typical study, monitoring equipment

is placed on the subject to obtain

polysomnography (PSG) data consisting of

simultaneous recordings of four

electrophysiological signals: (1) cerebral

activity recorded via the Electroencephalogram

(EEG); (2) ocular movement recorded via the

Electro-oculogram (EOG); (3) muscular tone

recorded via the submental Electromyogram

(EMG); and (4) cardiac activity recorded via

the Electrocardiogram (ECG). The PSG

provides objective measures of each sleep

stage based on a standard definition (7),

determined at 30-sec intervals (epochs) during

the course of the observation period. Based on

the data obtained from these measurements,

quantitative aggregated clinical endpoint

surrogates describing sleep efficacy such as

total sleep time (TST), latency to persistent

sleep (LPS), wake after sleep onset (WASO),

number of awakenings and time spent in each

stage of sleep, can be calculated to assess each

night’s sleep quality. These aggregated

clinical endpoint surrogates provide an overall

assessment of sleep behavior over the night but

provide few insights on the internal structure

of sleep.

Examination of the transitions to and from

each sleep stage based on the 30-sec epochs

provides a more granular approach to describe

the time course of sleep stage architecture.

Mathematical models can be created to

describe sleep structure as a stochastic process

assuming values in a finite discrete set. Such

approaches have previously been applied using

Markov-chain models to describe the

probability of transitioning from each stage of

sleep (8-12). These different models

characterized probability of transitioning to the

different sleep stages over the course of the

night with piecewise linear functions of

nighttime. These models have been refined to

include factors that improve the overall

performance for estimation of the different

sleep stage transition probabilities, including

the introduction of the duration of continuous

time spent in each sleep stage (stage time

effect), transitioning from initial sleeplessness,

multinomial logistic functions, and estimating

internal inflection points between epochs. The

first Markov chain models for describing sleep

data were parameterized as binary logistic

functions over the course of the night and

included stage time effects resulting in twenty

possible sub-models. These models were

further refined by including ‘initial

sleeplessness’ to describe the period of time a

patient goes to bed to falling asleep. Lastly,

the introduction of multinomial logistic

functions allowed a significant reduction in the

number of sub models created by

simultaneously estimating the transitions from

the current sleep stage to the other sleep stages.

However, all of these models were developed

in insomnia patients.

The purpose of this analysis was to: (1)

characterize the sleep stage transition

probabilities in phase advanced subjects over a

13-hr PSG measurement period after placebo

dosing expanding on a previously reported

multinomial Markov-chain model (11-12); and

(2) compare the first 8-hour PSG measurement

period in phase advanced subjects to that in

insomniac patients to identify the differences

in sleep architecture between these two

populations. The use of this model to describe

the PSG sleep architecture in phase advanced

subjects could provide valuable insight into the

efficacy of sleep-promoting agents in patients

with insomnia.

Methods and Materials

Study Design

A total of 51 subjects were enrolled to

participate in two phase advanced clinical trials

with an investigational drug, of which 27

subjects received a single dose of placebo.

Data collected from these two studies were

conducted at two different sites (France and

Japan) utilizing a randomized, placebo-

controlled, 4-period crossover, incomplete

block design with a washout duration of at

least 7 days between periods to minimize

carry-over effects related to drug concentration

and sleep pattern in previous periods. Subjects

were phase advanced on the first day of each

period of the study for the purpose of assessing

drug effects during what would normally be an

awake state. For each dosing day, study

medication was administered at approximately

05:30 pm, on a single occasion, 30 minutes

prior to the artificially early bedtime. Subjects

remained awake until lights were turned off at

approximately 06:00 pm (30 minutes after

dosing), and underwent 13 hours of PSG

Paper III

3

measurements while in bed. The first 3

periods of the study were devoted to PSG

measurements, whereas the fourth period was

used to obtain plasma concentrations of the

investigational drug.

Data

The data presented in the current analysis

were from 27 phase advanced subjects that

received placebo only. Few epochs of sleep

stages 3 and 4 were reported and were

therefore merged into a single sleep stage

termed ‘slow wave sleep’. Therefore the sleep

stages under evaluation were the awake stage

(AW), stage 1 sleep (ST1), stage 2 sleep (ST2),

slow wave sleep (SWS) and REM sleep

(REM).

Data Analysis

Multinomial mixed-effect Markov-chain model

Estimation of the transition probabilities

from each particular sleep stage to another was

obtained through the implementation of a non-

homogeneous Markov-chain model, similar to

that previously reported (12). To avoid

estimates being constrained between 0 and 1,

the parameters were estimated as logistic

functions. The logit of a probability p

describing a random variable is defined as the

logarithm of its odds, which is the natural

logarithm of the probability of achieving a

favorable outcome divided by the probability

of failing:

The logit function allows transformation

of a probability into an unconstrained variable.

As there are multiple potential transitions from

each sleep stage, these transitions must come

from a multinomial distribution, i.e. the

transition probabilities are transformed into

multinomial logit functions. For each subject i,

each starting state k, and each epoch t, the

logits for the sleep model are defined as:

where m takes all the values representing the

different sleep states.

In such a way, it is possible to define for

each triple (i, k, t) 5 different logit functions,

one of which is equal to zero (when m=k).

Note that as no correlation is assumed between

logits with different values for k, the model is

divided into five different smaller models.

Each sub-model describes the transitions from

a specific sleep stage, and its parameters can

be identified separately from the others. Each

of these sub-models was estimated using a

nonlinear, mixed-effect approach, taking the

variability of the population into consideration.

In the model considered here, each individual

logit gikm(t) is assumed to be distributed around

its typical value according to a multivariate

normal distribution.

The model parameters were assumed to

be time-varying considering the Markov-chain

model as non-homogeneous (9-12). Temporal

dependence was implemented assuming that

parameters are piecewise linear functions at

certain nighttime epochs (break points). The

parameters of the model are the transition

probabilities at these break points and the

associated inter-individual variability. The

previously described model in insomniac

patients included three break points with the

first (BPA) and last (BPC) fixed at the

beginning and at 8 hours of nighttime,

respectively, while the second break point

(BPB) was estimated according to the

maximum likelihood principle (12). These

break points were assumed to be common to

the whole population (no inter-individual

variability). For times in between break

points, the transition probability in the model is

given by linear interpolation of the logits at the

two adjacent break points. This previous model

formed the basis of the structural model for the

current data in phase advanced subjects. For

the current data, the first three break points

were similar to those described above (BPA,

BPB, and BPC) while a fourth break point

(BPD) was added and fixed at epoch 1560 (13-

hours). An additional, estimated break point

(BPE) was tested for significance between 960

and 1560 epochs using Akaike information

criterion and log-likelihood ratio test.

One additional feature of the ‘From AW’

sub-model was to differentiate the sleep

physiology time-course between initial

sleeplessness and the rest of the night.

Consequently, the 13-hour nighttime was

divided into 2 parts: (1) from t = 2 epochs to t

= IS, where IS ('Initial Sleeplessness') is the

first epoch in which a non-awake state is

observed in a specific subject; and (2) the

remaining part of the night after IS. In the

second time interval, the logits were modeled

Paper III

4

as previously described, changing BPA from

the beginning of the epoch measurements to

IS. The logits in the first part of the night were

modeled again as piecewise linear functions,

but no inter-individual variability or stage time

effects (see below) were included. In

particular, three additional break points were

defined at t = 2 epochs (BP1), the maximum IS

observed (BP3) and one estimated (BP2)

between BP1 and BP3.

The probability of moving from or staying

in a specific stage sleep k was influenced by

the duration of continuous time spent in stage k

designated as ‘stage time’. The stage time was

introduced in the model as a predictor for the

parameter values and modifies the logits at the

nighttime break points as the independent

variable of new piecewise linear functions with

three break points. The first break point

(BPsa) was fixed at the beginning of the stage

time (1 epoch) and the last break point (BPsc)

was fixed at the maximum stage time observed

in the data with respect to the particular stage

of departure. The second break point (BPsb)

was estimated as a parameter of the model

between BPsa and BPsc. No inter-individual

variability was assumed, so the stage time

influences the whole population in the same

manner. The identification of the sub-models

and parameter estimation was performed using

NONMEM VI (Globomax Corp.) (13).

Finally, stepwise covariate modeling was

used for assessing study effect on transition

probability estimates (14). The selective p

values for covariate effect inclusion (forward

search) and exclusion (backward search) were

0.05 and 0.01, respectively.

Simulations and Predictive Checks

A joint simulation model was constructed

from all transition models developed during

the analysis and 100 datasets were simulated.

Predictive checks were performed to assess

model performance. A simplified posterior

predictive check (sPPC) was performed for

each aggregated clinical endpoint surrogate

(e.g., LSO, WASO, TST) by comparing the

median of the individual values for each

simulated dataset to the median value (Median)

from the observed data to determine the

relative deviation (RD) as follows:

For each clinical endpoint surrogate, the

distribution of relative deviations was

computed and plotted in a box-whisker plot

using the R software package (15). For an

adequately described model, relative

deviations would be distributed around the

zero line.

A visual predictive check (VPC) was

performed to test the models’ ability to

properly describe the frequencies of sleep

stages and transitions throughout the night.

The median value and corresponding 95%

confidence interval derived from the simulated

datasets were plotted against the observed

median across the night to evaluate the

models’ predictive performance. The first 8

hours were divided into 10 intervals of equal

duration (48 minutes), whereas the final 5

hours were divided into 5 intervals of equal

duration (60 minutes). Transition and stage

frequencies were computed for both the

simulated and observed data.

Comparison of phase advanced subjects and

insomnia patients

The primary comparisons between phase

advanced subjects and insomniac patients were

PSG transition probabilities and aggregated

clinical endpoint surrogates. Data for

insomnia patients were provided by Bizzotto et

al. and consisted of transition probabilities

from modeling fits and clinical endpoint

surrogates of PSG measurements in the first of

the double-blind treatment nights from 116

patients treated with placebo (12).

Aggregated clinical endpoint surrogates were

chosen because of their frequent use as the

primary objective measures in placebo-

controlled trials of insomnia therapies (16-20).

Most aggregated clinical endpoint surrogates

were found to be non-normally distributed. As

a result, statistical testing was performed using

the Wilcoxon Rank Sum Test. The p value

chosen for statistical significance was 0.05.

Results

Data

The analysis was based upon all data for

subjects from whom adequate placebo dosing

and sampling time information were available.

The data included in the analyses consisted of

Paper III

5

42050 sleep stage observations collected from

27 phase advanced subjects (Table 1). Table 1. Subject Demographics.

Mean

(min-max)

AGE WT Ethnicity

Total

(N=27)

50 (20-76) 60.0 (45-82) 19 Female

8 Male

Study 1

(N=11)

43 64 8 Female

3 Male

NA

Study 2

(N=16)

54 57 11 Female

5 Male

Japanese

Multinomial mixed-effect Markov-chain

model for Phase Advanced Subjects

Expansion of a previous non-

homogeneous Markov-chain model using

multinomial logistic functions for describing

transition probabilities between sleep states

was explored for the current data in phase

advanced subjects. It was observed that some

of the 25 possible transitions between sleep

stages varied considerably between studies 1

and 2 and some transitions occurred

infrequently. Therefore, in order to simplify

the model, a frequency threshold of 0.1% was

used to impose transition frequencies equal to

zero (Table 2). Additionally, logits that were

close to zero in terms of probability ratios were

fixed to ‘−10’.

Table 2. Transition Frequencies Computed from

Data.

FROM/TO AW ST1 ST2 SWS REM

Study 1

AW 0.928 0.072 0.000a 0.000a 0.000a

ST1 0.124 0.471 0.377 0.000a 0.028

ST2 0.032 0.001 0.938 0.016 0.013

SWS 0.019 0.000a 0.031 0.949 0.000a

REM 0.026 0.002 0.018 0.000a 0.954

Study 2

AW 0.896 0.000a 0.003 0.000a 0.101

ST1 0.023 0.950 0.023 0.000a 0.005

ST2 0.035 0.018 0.920 0.027 0.001

SWS 0.022 0.001a 0.037 0.940 0.000a

REM 0.117 0.031 0.564 0.000a 0.288 a transition frequencies assumed equal to zero within the model.

For the “From AW” sub-model, an

additional estimated break point (BPE)

between 960 and 1560 epochs was statistical

significant and provided a better fit of the data.

Moreover, subjects in study 2 fell asleep much

more quickly than those in study 1. The timing

for the initial sleeplessness break points

included study effect to account for this

difference.

Figure 1 (left panel) illustrates the

estimated transition probability profiles for the

different sleep stages for both studies with no

stage time effect. The second break point

during the first 8 hr (BPB) occurred ~1.5 hrs

after the start of PSG measurements for all but

the “From SWS” sub-model. This inflection

indicates a change point in the overall

transition state within the sub-models to

approximate a change in the slope for the

transition probabilities. For the “From AW”

sub-model, study 1 demonstrated a shift

towards increasing the probability of

transitioning to ST1 after the inflection until

the end of the first 8 hours; whereas for study

2, the probability of transitioning to REM or

remaining in AW were approximately

equivalent. Additionally, the estimated break

point (BPE) between 8 and 13 hours indicates

an increase in the probability of transitioning

back to AW. For sub-model “From ST1” for

study 1, there is a shift toward increasing the

probability of transitioning to ST2 until BPB

and then back to ST1 thereafter. A similar

observation was observed for the “From REM”

sub-model for study 2, where there was an

increased probability in transitioning to ST2

back to REM. For the other sub-models, there

was a tendency to remain in the current

existing sleep stage and there were minor

changes in slope. The estimated logit values

and between-subject variability on the logits

are listed in Table 3.

The probability of moving from or

remaining in a specific stage was affected by

the duration of contiguous time spent in that

sleep stage. This duration, defined as the time

elapsed since the last change in sleep stage

time, was modeled to modify each logit as

previously described (Methods section).

Statistically significant differences on the

estimated stage time effect parameter values

between studies 1 and 2 were included for both

the “From ST1” and “From ST2” sub-models.

The estimated parameter values are shown in

Table 4 and the stage time effect (STE) over

stage time in the different sub-models is

illustrated in Figure 2. Because the additive

effect of stage time on the logits is equivalent

to a multiplicative effect on the probability

ratios, the logits were transformed to the

exponential value to better visualize STE over

the stage time. It follows that a decrease in the

exponential stage time effect increases the

probability of remaining in the current sleep

Paper III

6

Figure 1: Typical transition

probabilities for each sub-

model. The profiles are

computed with no stage time

effect (No STE) and with the

median stage time observed in

the data (Median STE).

Figure 2: Stage time effects

estimated in the different sub-

models. Median stage times

over the nighttime and the

whole patient population are

also reported in each plot.

stage. Conversely, an increase in the

exponential stage time effect increases the

probability of transitioning to the reference

sleep stage. Additionally, Figure 1 (right panel)

illustrates the estimated transition probability

profiles with the stage time effect produced by

the median stage time (median STE) for both

studies.

VPC and sPPC

Based on the final model, visual

predictive checks (VPC) and simplified

posterior predictive checks (sPPC) were

utilized to cumulatively examine the fidelity of

the model for both studies. The principle of a

VPC is to graphically assess a model’s ability

to reproduce both the central trend and

variability in the observed data through

simulations, whereas the sPPC assesses the

model’s capability of describing aggregated

characteristics of PSG data. Figure 3

illustrates the VPC results for the frequency of

transitions and stages. The VPC plots show a

good agreement between the observed and the

simulated statistics as the observed profiles fall

within the confidence intervals obtained from

the simulations. In addition, the profiles

demonstrate larger confidence intervals when

data for particular transitions are relatively

sparse and, conversely, less uncertainty when

data for particular transitions are more

prevalent.

Figure 4 illustrates the sPPC results for

aggregated clinical endpoint surrogates. The

sPPC indicate good agreement between

simulated and observed clinical endpoint

surrogates as none of the median aggregated

parameters fall outside the range of median

values computed from the simulated studies.

Overall, the VPC and sPPC provide evidence

that the final model effectively describes the

progression of sleep transitions along the night

0 2 4 6 8 10 12

0.00.20.40.60.81.0

Fro

m A

W

0 2 4 6 8 10 12

0.00.20.40.60.81.0

0 2 4 6 8 10 12

0.00.20.40.60.81.0

0 2 4 6 8 10 12

0.00.20.40.60.81.0

0 2 4 6 8 10 12

0.00.20.40.60.81.0

Fro

m S

T1

0 2 4 6 8 10 12

0.00.20.40.60.81.0

0 2 4 6 8 10 12

0.00.20.40.60.81.0

0 2 4 6 8 10 12

0.00.20.40.60.81.0

0 2 4 6 8 10 12

0.00.20.40.60.81.0

Fro

m S

T2

0 2 4 6 8 10 12

0.00.20.40.60.81.0

0 2 4 6 8 10 12

0.00.20.40.60.81.0

0 2 4 6 8 10 12

0.00.20.40.60.81.0

0 2 4 6 8 10 12

0.00.20.40.60.81.0

Fro

m S

WS

0 2 4 6 8 10 12

0.00.20.40.60.81.0

0 2 4 6 8 10 12

0.00.20.40.60.81.0

0 2 4 6 8 10 12

0.00.20.40.60.81.0

0 2 4 6 8 10 12

0.00.20.40.60.81.0

Fro

m R

EM

0 2 4 6 8 10 12

0.00.20.40.60.81.0

0 2 4 6 8 10 12

0.00.20.40.60.81.0

0 2 4 6 8 10 12

0.00.20.40.60.81.0

To AWTo ST1To ST2

To SWSTo REM

Time (hours)

Pro

bablit

y

No STE Median STEStudy 1 Study 2 Study 1 Study 2

0 40 80 120 160 200 240

0.0

0.2

0.4

0.6

0.8

1.0From AW

ST1/AWST2/AWREM/AW

Media

n 0 10 20 30 40 50

0.0

0.5

1.0

1.5

2.0

2.5

3.0

3.5

4.0From ST1

AW/ST1ST2/ST1 (Study 1)ST2/ST1 (Study 2)REM/ST1

Media

n 0 10 20 30 40 50

0

2

4

6

8

10

12From ST2

AW/ST2ST1/ST2SWS/ST2REM/ST2 (Study 1)REM/ST2 (Study 2)

Media

n

0 10 20 30 40 50 60 70

0.0

0.2

0.4

0.6

0.8

1.0From SWS

AW/SWSST2/SWS

Media

n 0 10 20 30 40 50

0.0

0.5

1.0

1.5

2.0

2.5

3.0From REM

AW/REMST1/REMST2/REM

Media

n

Exp

(st

age tim

e e

ffect)

Stage Time (minutes)

Paper III

7

and suitable to predict aggregated clinical

endpoint surrogate characteristics of PSG data

for phase advanced subjects.

Fig. 3: Visual predictive check on frequency of transitions (first 5 rows) and stage frequencies (last row).

Comparison between phase advanced subjects and insomnia patients

The typical transition probabilities (No

STE) over nighttime for both current data

(phase advanced subjects) and insomniac

patients for the first 8 hours is shown in Fig 5.

Many transition states represented by the sub-

models indicate that the probability of

transitioning between phase advanced subjects

and insomniac patients are approximately

equivalent. However, notable differences are

observed in the “From AW”, “From ST1” and

“From REM” sub-models. In the “From AW”

sub-model, phase advanced subjects exhibited

a higher probability of remaining AW for most

of the night compared to insomniac patients

whereas the latter have increased probability of

transitioning to ST1. In the “From ST1” and

“From REM” sub-models, phase advanced

subjects displayed different probability of

transitioning during the early portion of the

night compared to insomniac patients but the

most prominent difference was between

studies 1 and 2.

Additionally, several clinical endpoint

surrogates during the first 8 hours of

observation were examined (Fig 6). Out of the

24 clinical endpoint surrogates examined, 9

showed statistically significant differences.

Latency to sleep onset and persistent sleep

values in phase advanced subjects were about

25 minutes less than corresponding values in

insomnia patients. Although the quartiles of

the WASO (WASO1, WASO2, WASO2 and

WASO4) did not display any certain period

during which awakening was different, total

wake time after sleep onset over the course of

the first 8-hour observation period was about

50 minutes greater for phase advanced subjects

as compared to insomnia patients. In addition,

phase advanced subjects spent ~40 minutes

less time in REM and ST2 sleep states and ~47

minutes more time in SWS sleep state

compared to insomnia patients. Finally, phase

advanced subjects generally displayed a lower

average number of transitions from the current

sleep stage and a lower transition frequency

than that in insomnia patients.

0 2 4 6 8 10 12

0.0

0.2

0.4

0.6

0.8

1.0

To AWF

rom

AW

0 2 4 6 8 10 12

0.0

0.05

0.10

0.15

0.20

To ST1

0 2 4 6 8 10 12

0.0

0.01

0.02

0.03

0.04

0.05

To ST2

0 2 4 6 8 10 12

0.0

0.01

0.02

0.03

0.04

0.05

To SWS

0 2 4 6 8 10 12

0.0

0.05

0.10

0.15

0.20

To REM

0 2 4 6 8 10 12

0.0

0.1

0.2

0.3

0.4

Fro

m S

T1

0 2 4 6 8 10 12

0.0

0.2

0.4

0.6

0.8

1.0

0 2 4 6 8 10 12

0.0

0.1

0.2

0.3

0.4

0 2 4 6 8 10 12

0.0

0.01

0.02

0.03

0.04

0.05

0 2 4 6 8 10 12

0.0

0.01

0.02

0.03

0.04

0.05

0 2 4 6 8 10 12

0.0

0.02

0.04

0.06

0.08

0.10

Fro

m S

T2

0 2 4 6 8 10 12

0.0

0.02

0.04

0.06

0.08

0.10

0 2 4 6 8 10 12

0.0

0.2

0.4

0.6

0.8

1.0

0 2 4 6 8 10 12

0.0

0.02

0.04

0.06

0.08

0.10

0 2 4 6 8 10 12

0.0

0.01

0.02

0.03

0.04

0.05

0 2 4 6 8 10 12

0.0

0.05

0.10

0.15

0.20

Fro

m S

WS

0 2 4 6 8 10 12

0.0

0.01

0.02

0.03

0.04

0.05

0 2 4 6 8 10 12

0.0

0.05

0.10

0.15

0.20

0 2 4 6 8 10 12

0.0

0.2

0.4

0.6

0.8

1.0

0 2 4 6 8 10 12

0.0

0.01

0.02

0.03

0.04

0.05

0 2 4 6 8 10 12

0.0

0.2

0.4

0.6

0.8

1.0

Fro

m R

EM

0 2 4 6 8 10 12

0.0

0.02

0.04

0.06

0.08

0.10

0 2 4 6 8 10 12

0.0

0.2

0.4

0.6

0.8

1.0

0 2 4 6 8 10 12

0.0

0.01

0.02

0.03

0.04

0.05

0 2 4 6 8 10 12

0.0

0.2

0.4

0.6

0.8

1.0

0 2 4 6 8 10 12

0.0

0.2

0.4

0.6

0.8

1.0

AW

0 2 4 6 8 10 12

0.0

0.2

0.4

0.6

0.8

1.0

ST1

0 2 4 6 8 10 12

0.0

0.2

0.4

0.6

0.8

1.0

ST2

0 2 4 6 8 10 12

0.0

0.2

0.4

0.6

0.8

1.0

SWS

0 2 4 6 8 10 12

0.0

0.2

0.4

0.6

0.8

1.0

REM

Observed frequencyMedian PredictionSimulation based 95% CI

Time (hours)

Fre

quency

Paper III

8

F

rom

su

bm

od

el

Para

mete

rs

Para

mete

r la

bels

a a

nd

valu

es t

o d

iffe

ren

t sle

ep

sta

ges

AW

L

ogit

s a

t

ST

1

(BP

A)

ST

2

(BP

A)

RE

M

(BP

A)

ST

1

(BP

B)

ST

2

(BP

B)

RE

M

(BP

B)

ST

1

(BP

C)

ST

2

(BP

C)

RE

M

(BP

C)

ST

1

(BP

D)

ST

2

(BP

D)

RE

M

(BP

D)

ST

1

(BP

E)

ST

2

(BP

E)

RE

M

(BP

E)

N

ightt

ime B

P

-0.1

4

-4.8

1

0.1

6

-1.2

4

-2.7

3

0.1

1

-0.3

3

-4.9

5

-0.0

9

-0.5

8

-7.0

2

-0.6

4

0.0

52

-1

.95

0.0

84

L

ogit

s a

t IS

BP

ST

1

(BP

1)

ST

2b

(BP

1)

RE

M

(BP

1)

ST

1

(BP

2)

ST

2b

(BP

2)

RE

M

(BP

2)

ST

1

(BP

3)

ST

2b

(BP

3)

RE

M

(BP

3)

-3.8

9

-10

-2.3

1

-3.0

2

-10

-2.4

1

-3.4

3

-10

-3.3

5

B

P

BP

Ab

BP

Bc

BP

Cb

BP

Db

BP

E

BP

1b

BP

2d

BP

2e

BP

3d

BP

3d

IS

0.1

16

960

1560

1120

2

34.3

22.0

104

65.0

var–

co

v f

or

IIV

S

T1

S

T2

b

RE

M

0.1

6

0.0

0

0.1

9

ST

1

Lo

git

s a

t

AW

(BP

A)

S

T2 (

BP

A)

RE

M

(BP

A)

AW

b

(BP

B)

AW

e

(BP

B)

ST

2d

(BP

B)

ST

2e

(BP

B)

RE

M

(BP

B)

AW

d

(BP

C)

AW

e

(BP

C)

ST

2d

(BP

C)

ST

2e

(BP

C)

RE

M

(BP

C)

AW

d

(BP

D)

AW

e

(BP

D)

ST

2d

(BP

D)

ST

2e

(BP

D)

RE

Md

(BP

D)

RE

Me

(BP

D)

N

ightt

ime B

P

-0.3

1

-1.7

0

-4.0

6

-1.2

1

-3.9

8

0.1

2

-2.6

1

-5.3

8

-1.2

2

-3.2

5

-0.2

8

-3.7

2

-2.9

6

-1.3

9

-3.1

9

-0.8

7

-3.4

0

-1.9

3

-4.5

2

B

P

BP

Ab

BP

B

BP

Cb

BP

Db

2

189

960

1560

var–

co

v f

or

IIV

A

W

AW

-ST

2

ST

2

AW

-

RE

M

ST

2-

RE

M

RE

M

0.0

70

0.0

85

1.0

7

0.0

22

0.2

9

0.1

6

ST

2

Lo

git

s a

t

AW

(BP

A)

ST

1b

(BP

A)

SW

S

(BP

A)

RE

M

(BP

A)

AW

(BP

B)

ST

1d

(BP

B)

ST

1e

(BP

B)

SW

S

(BP

B)

RE

M

(BP

B)

AW

(BP

C)

ST

1d

(BP

C)

ST

1e

(BP

C)

SW

S

(BP

C)

RE

Md

(BP

C)

RE

Me

(BP

C)

AW

(BP

D)

ST

1d

(BP

D)

ST

1e

(BP

D)

SW

S

(BP

D)

N

ightt

ime B

P

-2.9

0

-10

-3.2

8

-5.0

4

-2.9

1

-5.2

1

-2.9

0

-3.2

6

-4.1

6

-3.3

6

-5.5

5

-3.3

9

-4.7

5

-3.7

9

-9.3

8

-2.6

1

-7.1

0

-3.0

1

-5.9

1

B

P

BP

Ab

BP

B

BP

Cb

BP

Db

2

149

960

1560

var–

co

v f

or

IIV

A

W

ST

1

SW

S

RE

Ma

0.1

4

0.0

84

0.6

2

0.0

0

SW

S

Lo

git

s a

t

AW

(BP

A)

S

T2 (

BP

A)

AW

(BP

B)

ST

2

(BP

B)

AW

(BP

C)

ST

2

(BP

C)

AW

(BP

D)

ST

2

(BP

D)

N

ightt

ime B

P

-3.7

2

-2.6

9

-3.5

5

-2.1

5

-3.6

4

-1.7

9

-2.9

7

-3.9

9

B

P

BP

Ab

BP

B

BP

Cb

BP

Db

-3.7

2

-2.6

9

-3.5

5

-2.1

5

-3.6

4

-1.7

9

-2.9

7

-3.9

9

var–

co

v f

or

IIV

A

Wb

ST

2

0.0

0

0.1

1

RE

M

Lo

git

s a

t

AW

(BP

A)

S

T1

b(B

PA

)

ST

2

(BP

A)

AW

d

(BP

B)

AW

e

(BP

B)

ST

1

(BP

B)

ST

2d

(BP

B)

ST

2e

(BP

B)

AW

d

(BP

C)

AW

e

(BP

C)

ST

1b ,

d

(BP

C)

ST

1e

(BP

C)

ST

2d

(BP

C)

ST

2e

(BP

C)

AW

d

(BP

D)

AW

e

(BP

D)

ST

1

(BP

D)

ST

2d

(BP

D)

ST

2e

(BP

D)

N

ightt

ime B

P

-1.7

8

-10

-0.4

6

-3.7

2

0.1

65

-3

.24

-2

.51

2.5

3

-3.1

9

-0.4

3

-10

-1.2

5

-3.0

2

1.1

1

-2.6

7

-0.5

8

-2.1

9

-3.1

6

1.0

1

B

P

BP

Ab

BP

B

BP

Cb

BP

Db

2

143

960

1560

var–

co

v f

or

IIV

A

W

ST

1

ST

2

0.1

3

0.5

9

0.2

6

Ab

bre

via

tion

s:

BP

– b

reak

poin

ts, II

V –

inte

r-in

div

idu

al

vari

abil

ity;

IS -

In

itia

l S

leep

lessn

ess;

var–

cov –

vari

an

ce-c

ovari

ance e

sti

mate

. a T

he v

alu

e o

f m

of

logit

gik

m(t

) is

tak

en

as

lab

el.

b f

ixed

c B

PB

=(I

S*(9

60

-IS

)*0

.116

d S

tud

y1

e S

tud

y 2

Ta

ble

3.

Nig

htt

ime

an

d I

nit

ial

Sle

eple

ssn

ess

Lo

git

s a

nd

Bre

ak

Po

ints

Mo

del

Pa

ram

eter E

stim

ate

d V

alu

es

Paper III

9

Fig. 4: Simplified posterior predictive check: relative deviation of median aggregated clinical endpoint

surrogates. Aggregated clinical endpoint surrogates are represented as: latency to sleep onset (LSO); wake after

sleep onset (WASO); quarter assessed WASO (WASO1, WASO2, WASO3 and WASO4); total sleep time

(TST); sleep efficiency index (SEI); quarter assessed SEI (SEI1, SEI2, SEI3 and SEI4); time spent in sleep

stages (tAW, tST1, tST2, tSWS, tREM and tNREM); number of transitions to sleep stage (nAW, nST1, nST2,

nSWS, nREM).

Figure 5: Transition probability versus time for phase advanced subjects (PAS) and insomniac patients (IP).

Discussion

Sleep disruption produced by phase

advanced sleep to induce transient insomnia

can allow assessment of drug efficacy in

altering sleep architecture (5-6).

Understanding the underlying

mechanisms related to the transitions between

sleep stages facilitates correlative analysis

between induced transient insomnia and

chronic sleep disorders. Examination of the

clinical endpoint surrogates provides a

highlevel view of the inherent similarities and

differences between groups or treatments.

-1.0

-0.5

0.0

0.5

1.0

1.5

2.0

LSO

WASO

WASO

1

WASO

2

WASO

3

WASO

4

TSTSE

ISE

I1SE

I2SE

I3SE

I4tA

WtN

REM

tREM

tST1

tST2

tSW

S

nAW

nREM

nST1

nST2

nSW

S

Rela

tive D

evi

atio

ns

of M

edia

n C

linic

al E

ndpoin

t S

urr

ogate

s

0 1 2 3 4 5 6 7 8

0.00.20.40.60.81.0

Fro

m A

W

TO AW

0 1 2 3 4 5 6 7 8

0.00.20.40.60.81.0

TO ST1

0 1 2 3 4 5 6 7 8

0.00.20.40.60.81.0

TO ST2

0 1 2 3 4 5 6 7 8

0.00.20.40.60.81.0

TO SWS

0 1 2 3 4 5 6 7 8

0.00.20.40.60.81.0

TO REM

0 1 2 3 4 5 6 7 8

0.00.20.40.60.81.0

Fro

m S

T1

0 1 2 3 4 5 6 7 8

0.00.20.40.60.81.0

0 1 2 3 4 5 6 7 8

0.00.20.40.60.81.0

0 1 2 3 4 5 6 7 8

0.00.20.40.60.81.0

0 1 2 3 4 5 6 7 8

0.00.20.40.60.81.0

0 1 2 3 4 5 6 7 8

0.00.20.40.60.81.0

Fro

m S

T2

0 1 2 3 4 5 6 7 8

0.00.20.40.60.81.0

0 1 2 3 4 5 6 7 8

0.00.20.40.60.81.0

0 1 2 3 4 5 6 7 8

0.00.20.40.60.81.0

0 1 2 3 4 5 6 7 8

0.00.20.40.60.81.0

0 1 2 3 4 5 6 7 8

0.00.20.40.60.81.0

Fro

m S

WS

0 1 2 3 4 5 6 7 8

0.00.20.40.60.81.0

0 1 2 3 4 5 6 7 8

0.00.20.40.60.81.0

0 1 2 3 4 5 6 7 8

0.00.20.40.60.81.0

0 1 2 3 4 5 6 7 8

0.00.20.40.60.81.0

0 1 2 3 4 5 6 7 8

0.00.20.40.60.81.0

Fro

m R

EM

0 1 2 3 4 5 6 7 8

0.00.20.40.60.81.0

0 1 2 3 4 5 6 7 8

0.00.20.40.60.81.0

0 1 2 3 4 5 6 7 8

0.00.20.40.60.81.0

0 1 2 3 4 5 6 7 8

0.00.20.40.60.81.0

Time (hours)

Pro

bablit

y

IPPAS (Study 1)PAS (Study 2)

Paper III

10

Table 4. Stage Time Effect Estimated Model Parameter Values

However, summarizing the collective data

from PSG data into clinical endpoint

surrogates does not maximize the utility of the

rich information collected from each

individual. Thus, understanding the

probability of transitioning between sleep

stages employing the multinomial mixed-effect

Markov-chain model between the sleep stages

provides an opportunity to better understand

the sleep architecture. This can provide a

pathway for better predictions of drug efficacy

and future clinical trial execution that use

phase advanced sleep as a surrogate for

chronic sleep disorders as sleep promoting

drugs have shown efficacy in both the phase

advanced model and in insomniac patients (6,

9-10, 21-23).

The multinomial mixed-effect Markov-

chain model presented was developed with the

intent of providing a useful tool for

understanding sleep stage transition

probabilities in phase advanced subjects after

placebo dosing. The current population

analysis is similar to that previously reported

(12) using a non-homogenous multinomial

mixed-effect Markov-chain model where the

transition probabilities, dependent on nighttime

and stage time effects, were estimated as

piecewise linear logistic functions.

Furthermore, the current model was

expanded to allow additional break points to

accommodate the 13-hour observation period

and study-to-study differences in the transition

probabilities. The analyses presented allowed

quantitative examination of the phase

advanced subject data to demonstrate

differences between the two phase advanced

sleep studies on stage time effect and transition

probabilities for particular sub-models. These

significant differences in the transition

probabilities could be related to differences in

ethnicity; however, this hypothesis cannot be

confirmed by existing data because the

ethnicity in study1 was not available.

Examination of the clinical endpoint

surrogates and transition probabilities

displayed inherent similarities and differences

between phase advanced subjects and insomnia

patients, and indicates that phase advanced

subjects generally displayed a lower transition

frequency out of the current sleep state than

insomniac patients, as well as differing total

time spent in specific sleep stages. Although

the total time spent awake was not that

dissimilar, phase advanced subjects displayed a

higher tendency to awaken during the early

portions of the night compared to insomniac

patients. This could be based on falling asleep

quicker, thus inducing a tendency to awaken

after an initial rest period. Although the subject

Sub-model Parameters Parameter labels and values

From AW

STE

at stage time BP

ST1

(BPsb)

ST2

(BPsb)

REM

(BPsb)

ST1

(BPsc)

ST2

(BPsc)

REM

(BPsc)

-3.26 -5.29 -4.01 -4.63 -10 FIX -5.35

BP BPsa BPsb BPsc

1 FIX 13.4 480 FIX

From ST1

STE

at stage time BP

AW

(BPsb)

ST2

(BPsb)a

ST2

(BPsb)b

REM

(BPsb)

AW

(BPsc)

ST2

(BPsc)

REM

(BPsc)

-0.659 1.13 -0.913 -1.26 0.803 0.602 -1.46

BP BPsa BPsb BPsc

1 FIX 3.22 87 FIX

From ST2

STE

at stage time BP

AW

(BPsb)

ST1

(BPsb)

SWS

(BPsb)

REM

(BPsb)a

REM

(BPsb)b

AW

(BPsc)

ST1

(BPsc)

SWS

(BPsc)

REM

(BPsc)

-0.750 -1.65 0.643 -0.769 -4.44 0.874 -1.19 2.44 -1.67

BP BPsa BPsb BPsc

1 FIX 19.0 100 FIX

From SWS

STE

at stage time BP

AW

(BPsb)

ST2

(BPsb)

AW

(BPsc)

ST2

(BPsc)

-0.240 -1.23 -0.602 -0.365

BP BPsa BPsb BPsc

1 FIX 6.44 133 FIX

From REM

STE

at stage time BP

AW

(BPsb)

ST1

(BPsb)

ST2

(BPsb)

AW

(BPsc)

ST1

(BPsc)

ST2

(BPsc)

-1.07 -0.998 -1.51 0.902 -3.19 -0.511

BP BPsa BPsb BPsc

1 FIX 2.22 86 FIX a Study 1

b Study 2

Paper III

11

Figure 6: Observed clinical endpoint surrogates for phase advanced subjects (PAS) and insomnia patients (IP).

Statically significant difference indicated by “*”.

numbers were significantly unbalanced

between phase advanced subjects and insomnia

patients (27 and 116, respectively), it was

observed that the phase advanced subjects and

insomniac patients display different sleep

architecture for some transition probabilities

and clinical endpoint surrogates.

One of the advantages of multinomial

mixed-effect Markov-chain model to explore

this kind of data is that it provides additional

information on understanding the effects of

drugs on different sleep stages and

extrapolation from the phase advanced model

to insomniac patients could be beneficial. With

model based analysis, new dosing regimens

can be explored for designing new studies in

sleep data from phase advanced subjects to

insomniac patients. Simulations may be

performed to predict the outcome of

compounds with similar mechanism of action

but different pharmacokinetic profiles.

Additional work aimed to further

investigate the differences observed between

the current studies explored here, might

include identifying significant covariates that

affect the transition probabilities and reducing

the model structure. For this particular data,

several potential covariates were available,

such as smoking habits, weight, alcohol habits

and gender, as these have demonstrated some

pronounced effect in sleep patterns in previous

investigations (12,24-25).

In conclusion, the proposed multinomial

mixed-effect Markov-chain model in phase

advanced subjects resulted in a robust

estimation of transition probabilities between

sleep stages and effect of the duration of

contiguous sleep after placebo dosing. The

internal validation procedures, sPPC and VPC,

demonstrated that the proposed models

suitably predicted aggregated clinical endpoint

surrogates characteristics of PSG data, and

confining some transition probabilities to

essentially zero, either by removing their

estimation or assigning the value to -10 for the

logit in the model, was appropriate. In

addition, the general model structure is easily

adaptable to allow significant changes, as was

done to accommodate the 13-hour

observational period and quantitative

assessment of study-to-study differences.

Finally, phase advanced subjects and

insomniac patients displayed different sleep

architecture.

050

100

150

PAS IP

LSO*

050

100

200

PAS IP

LPS*

01

00

200

300

400

PAS IP

WASO*

020

40

60

80

PAS IP

WASO1

020

60

100

PAS IP

WASO2

020

60

100

PAS IP

WASO3

020

60

100

PAS IP

WASO4

100

200

300

400

PAS IP

TST

01

00

200

300

400

PAS IP

tAW

100

200

300

400

PAS IP

tNREM

050

100

150

PAS IP

tREM*

050

100

150

PAS IP

tST1

50

150

250

PAS IP

tST2*

050

150

250

PAS IP

tSWS*

Min

ute

s20

40

60

80

PAS IP

SEI

020

40

60

80

PAS IP

SEI1

20

40

60

80

PAS IP

SEI2

020

40

60

80

PAS IP

SEI3

20

40

60

80

100

PAS IP

SEI4

%20

40

60

80

PAS IP

nAW*

010

20

30

40

50

PAS IP

nREM

020

60

100

PAS IP

nST1*

20

60

100

PAS IP

nST2*

010

30

50

PAS IP

nSWS

Num

ber

Paper III

12

References

1. American Sleep Disorders Association

(ASDA). EEG arousals: scoring rules and

examples. A preliminary report from the

Sleep Disorders Atlas Task Force of the

American Sleep Disorders Association.

Sleep. 1992;19:174-184.

2. American Sleep Disorders Association

(ASDA). Recording and scoring leg

movements. The Atlas Task Force. Sleep.

1993;16:748-759.

3. American Sleep Disorders Association

(ASDA). Sleep-related breathing disorders

in adults: recommendations for syndrome

definition and measurement techniques in

clinical research. Sleep. 1999;22:667-689.

4. Lavie P. Ultrashort sleep-waking schedule.

III. 'Gates' and 'forbidden zones' for sleep.

Electroencephalogr Clin Neurophysiol.

1986;63(5):414-425.

5. Walsh JK, Deacon S, Dijk DJ, Lundahl J.

The selective extrasynaptic GABAA

agonist, gaboxadol, improves traditional

hypnotic efficacy measures and enhances

slow wave activity in a model of transient

insomnia. Sleep. 2007;30:593–602.

6. Walsh JK, Schweitzer PK, Sugerman JL,

Muehlbach MJ. Transient insomnia

associated with a 3-hour phase advance of

sleep time and treatment with zolpidem. J

Clin Psychopharmacol. 1990;10(3):184-

189.

7. A. Rechtschaffen, A. Kales. A manual of

standardized terminology, techiques and

scoring system for sleep stages of human

subjects. In U.B.I.S.B.R. Institute (ed)

(U.B.I.S.B.R. Institute, ed), Los Angles,

1968.

8. Kemp B, Kamphuisen HA. Simulation of

human hypnograms using a Markov chain.

Sleep.1986;9:405-414.

9. Karlsson MO, Schoemaker RC, Kemo B,

Cohne AF, van Gerven JM, Tuk B, Peck

CC, Danhof M. A pharmacodynamic

Markov mixed-effects model for the effect

of temazepam on sleep. Clin Pharmacol

Ther. 2000;68:175-88.

10. Kjellsson MC, Ouellet D, Corrigan B

Karlsson MO. Modeling Sleep Data for a

New Drug in Development using Markov

Mixed-Effects Models. Pharm Res.

2011;28(10):2610-2627.

11. Bizzotto R, Zamuner S, De Nicolao G,

Karlsson MO, Gomeni R. Multinomial

logistic estimation of Markov-chain

models for modeling sleep architecture in

primary insomnia patients. J

Pharmacokinet Pharmacodyn. 2010;

7:137–155.

12. Bizzotto R, Zamuner S,Mezzalana E, De

NicolaoG, Gomeni R, Hooker AC,

Karlsson MO. Multinomial Logistic

Functions in Markov Chain Models of

Sleep Architecture Internal and External

Validation and Covariate Analysis. The

AAPS Journal, 2011; 13(3):445-463.

13. Beal SL, Sheiner LB. NONMEM user’s

guides. San Francisco: NONMEM Project

Group; 1998.

14. Jonsson EN, Karlsson MO. Automated

covariate model building within

NONMEM. Pharm Res. 1998;15:1463–

1468.

15. R version 2.11.1, 2010 The R Foundation

for Statistical Computing.

16. Hirshkowitz M. Normal human sleep: an

overview, Med Clin North Am. 2004;88:

551-565.

17. Feinsilver SH. Sleep in the elderly. What

is normal?, Clin Geriatr Med.

2003;19:177-188.

18. Roehrs T. Sleep physiology and

pathophysiology. Clin Cornerstone.

2000;2:1-15.

19. Carskadon MA, Dement WC, Mitler MM,

Roth T, Westbrook PR, Keenan S.

Guidelines for the Multiple Sleep Latency

Test (MSLT): a standard measure of

sleepiness, Sleep. 1986; 9:519–524.

20. Horne J. Why we sleep. Oxford University

Press, Oxford, UK; 1988.

21. Svetnik V, Ferri R, Ray S, Ma J, Walsh

JK, Snyder E, Ebert B, Deacon S.

Alterations in cyclic alternating pattern

associated with phase advanced sleep are

differentially modulated by gaboxadol and

zolpidem. Sleep. 2010;33(11):1562-1570.

22. Erman MK, Loewy DB, Scharf MB.

Effects of temazepam 7.5 mg and

temazepam 15 mg on sleep maintenance

and sleep architecture in a model of

transient insomnia. Curr Med Res Opin.

2005;21(2):223-230.

23. Berger M, Vollmann J, Hohagen F, König

A, Lohner H, Voderholzer U, Riemann D.

Sleep deprivation combined with

consecutive sleep phase advance as a fast-

acting therapy in depression: an open pilot

trial in medicated and unmedicated

Paper III

13

patients. Am J Psychiatry. 1997

;154(6):870-2.

24. Nakade M, Takeuchi H, Kurotani M,

Harada T. Effects of meal habits and

alcohol/cigarette consumption on

morningness-eveningness preference and

sleep habits by Japanese female students

aged 18-29. J Physiol Anthropol.

2009;28(2):83-90.

25. Ma J, Dijk DJ, Svetnik V,

Tymofyeyev Y, Ray S, Walsh JK, Deacon

S. EEG power spectra response to a 4-h

phase advance and gaboxadol treatment in

822 men and women. Clin Sleep Med.

2011;7(5):493-501A.