Study Design and Study Design and EfficiencyEfficiency

Margarita SarriMargarita Sarri

Hugo SpiersHugo Spiers

We will talk about:We will talk about:

What kinds of designs are out there? -What kinds of designs are out there? -Blocked vs event-related designsBlocked vs event-related designs

How can I order my events?How can I order my events? What is estimation efficiency?What is estimation efficiency? Which designs are more efficient?Which designs are more efficient? Spacing of eventsSpacing of events Sampling issuesSampling issues Filtering issuesFiltering issues

Event related vs Blocked Event related vs Blocked designsdesigns

Blocked / Epoch/ Box design Blocked / Epoch/ Box design Types of trials are ‘blocked’ together e.g. AAAAA Types of trials are ‘blocked’ together e.g. AAAAA

BBBBB AAAAA.BBBBB AAAAA.

Event related design Event related design Types of trials are interleaved and each trial is modelled Types of trials are interleaved and each trial is modelled

separately as an ‘event’ e.g. AABABBABseparately as an ‘event’ e.g. AABABBAB

In general In general 22 blocks more efficient than blocks more efficient than 44.. Ideal modulation frequency being approximately Ideal modulation frequency being approximately 16sec 16sec

but you may not be able to test certain things with such a but you may not be able to test certain things with such a design…design…

So you may want to go for an So you may want to go for an event related design…event related design…

Blocked designBlocked design typically used in experiments where the detection of activation is typically used in experiments where the detection of activation is the primary goal.the primary goal. e.g localise a specific brain region showing a differential response to e.g localise a specific brain region showing a differential response to one type of stimulus (e.g. faces vs houses)one type of stimulus (e.g. faces vs houses)

Why should I use efMRI ? Why should I use efMRI ?

Flexibility and randomizationFlexibility and randomization eliminate predictability of block designseliminate predictability of block designs avoid practice effects/strategy useavoid practice effects/strategy use

Post hoc sorting Post hoc sorting e.g. classification of correct vs. incorrect, subjective e.g. classification of correct vs. incorrect, subjective

perception: aware vs. unaware, remembered vs. perception: aware vs. unaware, remembered vs. forgotten items, parametric scores: e.g. fast vs. slow RTsforgotten items, parametric scores: e.g. fast vs. slow RTs

Measuring novelty: Measuring novelty: Rare or unpredictable eventsRare or unpredictable events e.g. oddball designs.e.g. oddball designs.

Allows to look at events on a shorter time scale.Allows to look at events on a shorter time scale.

P

L

H

A

K

But you can also combine block and But you can also combine block and efMRI…efMRI…

A block can be treated as a continuous train of A block can be treated as a continuous train of event-trialsevent-trials

E.g Otten, Henson & Rugg, Nature Neuroscience 2002E.g Otten, Henson & Rugg, Nature Neuroscience 2002

‘‘Subsequent memory’ experiment separating transient (events) Subsequent memory’ experiment separating transient (events) and sustained (blocks) neural activity. and sustained (blocks) neural activity.

At the beginning of each trial a cue instructed subjects to make At the beginning of each trial a cue instructed subjects to make an phonological or semantic judgement.an phonological or semantic judgement.

83sec83sec restrest 83sec83sec

Hmmm I think I like efMRI.

But how do I order my trials?

efMRI: SefMRI: Sequencing of eventsequencing of events

Deterministic Deterministic designs:designs:

the occurrence of the occurrence of events is pre-events is pre-determined e.g. a determined e.g. a blocked design or blocked design or alternating design alternating design (all (all the probabilities are zero or the probabilities are zero or oneone ) )

StochasticStochastic

designs:designs:

the occurrence of the occurrence of an event an event depends on a a depends on a a specified specified probability e.g. probability e.g. random or random or permuted designpermuted designStochastic Stochastic designs can be designs can be stationary or stationary or dynamicdynamic

BlockedBlocked

AlternatinAlternatingg

1 2 3 4 5 6 7 8

10

20

30

40

50

60

70

80

RandomRandom

PermutedPermuted

How do I do I create a permuted order of How do I do I create a permuted order of events?events?

ensure mini-runs of same stimuli…ensure mini-runs of same stimuli…

i.e. modulate the probability of different event-types over i.e. modulate the probability of different event-types over experimental timeexperimental time

Permutation methods continued…Permutation methods continued…

So what is So what is Efficiency?Efficiency?

Efficiency is…Efficiency is… Efficiency is a numerical value Efficiency is a numerical value

which reflects the ability of your design to detect the effect which reflects the ability of your design to detect the effect of interestof interest

General Linear Model: General Linear Model:

Y = X Y = X .. ββ + + e e

DataData Design Matrix Design Matrix Parameters errorParameters error

Efficiency is the ability to estimate Efficiency is the ability to estimate ββ, given the design , given the design matrix Xmatrix X

Efficiency can be calculated because the variance of Efficiency can be calculated because the variance of ββ is is proportional to the variance of Xproportional to the variance of X

What is variance?What is variance?

Standard Deviation

Variance = Standard Deviation Variance = Standard Deviation 22

High Variance

Low Variance

Standard Deviation

Testing a Hypothesis Testing a Hypothesis

T- Test for the difference between 2 T- Test for the difference between 2 conditionsconditions

Lower ability to detect a difference

Higher ability to detect a difference

Standard Deviation

Standard Deviation

• By reducing the variance in the design we can maximize our T values

How do we calculate it?How do we calculate it?

Efficiency Efficiency Inverse( Var( Inverse( Var(ββ) ) ) )

Inverse( Var(Inverse( Var(ββ) )) ) Var(X) Var(X)

Var(X) Var(X) Inverse( X Inverse( XTTX )X )

A B C DA B C D 1 0 0 01 0 0 0 1 0 0 01 0 0 0 1 0 0 01 0 0 0 1 0 0 01 0 0 0 1 0 0 01 0 0 0 0 1 0 00 1 0 0 0 1 0 00 1 0 0 0 1 0 00 1 0 0 0 1 0 00 1 0 0 0 1 0 00 1 0 0 0 0 0 00 0 0 0 0 0 0 00 0 0 0 0 0 1 00 0 1 0 0 0 1 10 0 1 1 0 0 1 10 0 1 1 0 0 1 10 0 1 1 0 0 1 10 0 1 1 0 0 0 10 0 0 1 0 0 0 00 0 0 0 0 0 0 00 0 0 0

X XT

A 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0A 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0B 0 0 0 0 0 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0B 0 0 0 0 0 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0C 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 0 0 0C 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 0 0 0D 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 0 0D 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 0 0

. = A B C DA B C DA 5 0 0 0A 5 0 0 0B 0 5 0 0B 0 5 0 0C 0 0 5 4C 0 0 5 4D 0 0 4 5D 0 0 4 5

XT

X

Non-Non-overlappinoverlappin

ggconditionsconditions

OverlappinOverlappingg

conditionsconditions

A B C DA B C DA 5 0 0 0A 5 0 0 0B 0 5 0 0B 0 5 0 0C 0 0 5 4C 0 0 5 4D 0 0 4 5D 0 0 4 5

XT

X inverse (XT

X) A B C DA B C DA 0.2 0 0 0A 0.2 0 0 0B 0 0.2 0 0B 0 0.2 0 0C 0 0 0.6 -0.4C 0 0 0.6 -0.4D 0 0 -0.4 0.6D 0 0 -0.4 0.6

The efficiency is related to the specific The efficiency is related to the specific contrast you are interested incontrast you are interested in

Efficiency = inverse(σ2 cT Inverse(XTX) c)

Where c = contrast σ2 = noise variance

But if we assume that noise variance σ2 is constant then:

Efficiency = inverse (cT Inverse (XTX) c)

When c is Simple Effect,

e.g. main effect of A c = [1 0 0 0]

inverse(XT

X) A B C DA B C DA 0.2 0 0 0A 0.2 0 0 0B 0 0.2 0 0B 0 0.2 0 0C 0 0 0.6 -0.4C 0 0 0.6 -0.4D 0 0 -0.4 0.6D 0 0 -0.4 0.6

Efficiency = Inverse( cT Inverse(XTX) c)

A, B: Efficiency = 1 / 0.2 = 5 C, D: Efficiency = 1 / 0.6 = 1.7

1000

1 0 0 0

CTC

When c is contrast difference,

e.g. For A – B c = [1 -1 0 0]

inverse(XT

X) A B C DA B C DA 0.2 0 0 0A 0.2 0 0 0B 0 0.2 0 0B 0 0.2 0 0C 0 0 0.6 -0.4C 0 0 0.6 -0.4D 0 0 -0.4 0.6D 0 0 -0.4 0.6

Efficiency = Inverse( cT Inverse(XTX) c)

A-B: Efficiency = 1 / 0.4 = 2.5 C-D: Efficiency = 1 / 2 = 0.5

1-1 0 0

1 -1 0 0

CTC

0.5 1 1.5 2 2.5 3 3.5 4 4.5

100

200

300

400

500

600

700

800

900

Variable No. of Trials

X inv(XT

X)

4.24.2RandomRandom::

Events Events = = 2525

2.12.1

Relative EfficiencyRelative Efficiency

RandomRandom::

Events Events = = 5050

0.5 1 1.5 2 2.5 3 3.5 4 4.5

0.5

1

1.5

2

2.5

3

3.5

4

4.5

How does trial order effect How does trial order effect Efficiency?Efficiency?

ExampleExample

ORDER 1 Inte

rleaved st

imuli

ORDER 2 Block

s of s

timuli

A B C D E FA B C D E F 1 0 0 0 0 01 0 0 0 0 0 1 0 1 0 0 11 0 1 0 0 1 1 0 0 0 0 11 0 0 0 0 1 1 0 0 1 0 01 0 0 1 0 0 1 0 0 0 0 01 0 0 0 0 0 0 1 1 0 0 00 1 1 0 0 0 0 1 0 0 0 00 1 0 0 0 0 0 1 0 1 1 00 1 0 1 1 0 0 1 0 0 1 00 1 0 0 1 0 0 1 1 0 0 10 1 1 0 0 1 0 0 0 0 0 00 0 0 0 0 0 0 0 0 1 0 10 0 0 1 0 1 0 0 0 0 1 00 0 0 0 1 0 0 0 1 0 1 00 0 1 0 1 0 0 0 0 0 0 00 0 0 0 0 0 0 0 0 1 0 00 0 0 1 0 0 0 0 0 0 1 00 0 0 0 1 0 0 0 1 0 0 00 0 1 0 0 0 0 0 0 0 0 10 0 0 0 0 1 0 0 0 1 0 00 0 0 1 0 0

Different Designs – Boxcar Events

inv(XT

X) A B C D E FA B C D E FA 0.2488 0.0377 -0.0297 -0.0396 -0.0012 -A 0.2488 0.0377 -0.0297 -0.0396 -0.0012 -

0.08730.0873B 0.0377 0.2862 -0.0941 -0.0421 -0.0873 -B 0.0377 0.2862 -0.0941 -0.0421 -0.0873 -

0.02630.0263C -0.0297 -0.0941 0.2871 0.0495 -0.0297 -C -0.0297 -0.0941 0.2871 0.0495 -0.0297 -

0.09410.0941D -0.0396 -0.0421 0.0495 0.2327 -0.0396 -D -0.0396 -0.0421 0.0495 0.2327 -0.0396 -

0.04210.0421E -0.0012 -0.0873 -0.0297 -0.0396 0.2488 E -0.0012 -0.0873 -0.0297 -0.0396 0.2488

0.03770.0377F -0.0873 -0.0263 -0.0941 -0.0421 0.0377 F -0.0873 -0.0263 -0.0941 -0.0421 0.0377

0.28620.2862

1 2 3 4 5 6

1

2

3

4

5

6

X

BlockedBlocked

Fixed Fixed InterleaveInterleave

dd RandomRandom

1.51.5

Different Designs

BlockedBlocked

Fixed Fixed InterleaveInterleave

dd Random-Random-UniformUniform

1 2 3 4 5 6 7 8

10

20

30

40

50

60

70

80

Random-Random-SinusoidaSinusoida

ll

1 2 3 4 5 6 7 8

1

2

3

4

5

6

7

8

inv(XT

X)X

55

2.82.8

3.53.5

Relative Efficiency

Relative Efficiency

1 2 3 4 5 6 7 8

1

2

3

4

5

6

7

8

Different Designs

1.51.5

BlockedBlocked

X

55

2.82.8

3.53.5

inv(XT

X)

10

20

30

40

50

60

70

80

Relative Efficiency

Relative Efficiency

0 5 10 15 20 25 30 35 40-1

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

0 5 10 15 20 25 30 35 40-1

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

0 5 10 15 20 25 30 35 40-1

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

Sequencing of eventsSequencing of events

Stochastic designs: at each point at which an event could occur there is a specified probability of that event occurring. The timing of when the events occur is specified. Non-occurrence = null event.

Deterministic designs: the occurrence of events is pre-determined.

The variable deterministic design i.e. a blocked design, is the most efficient.

Joel’s example of different stimulus Joel’s example of different stimulus presentationspresentations

Blocked design

Fully randomised

Dynamic stochastic

A B CTasks

0102030405060708090

100

Block Dynamicstochastic

Randomised

Efficiency calculation

different designsdifferent designs {{

minimum SOA (inter-stimulus minimum SOA (inter-stimulus interval)interval)

probability of occurrenceprobability of occurrence

How fast can I present my trials?

The absolute minimum…The absolute minimum… Early event-related fMRI studies used a long Early event-related fMRI studies used a long

Stimulus Onset Asynchrony (SOA) to allow BOLD Stimulus Onset Asynchrony (SOA) to allow BOLD response to return to baseline (20-30s).response to return to baseline (20-30s).

However, if the BOLD response is explicitly However, if the BOLD response is explicitly modelledmodelled, overlap between successive responses , overlap between successive responses at short SOAs can be accommodatedat short SOAs can be accommodated… (assuming … (assuming that successive responses add up in a linear that successive responses add up in a linear fashion)fashion)

The lower limit on SOAs is dictated by nonlinear The lower limit on SOAs is dictated by nonlinear interactions among eventsinteractions among events that can be though of as that can be though of as saturation phenomena or ‘‘refractoriness’’ at a neuronal saturation phenomena or ‘‘refractoriness’’ at a neuronal or hemodynamic level.or hemodynamic level.

But, very short SOAs (< 1s) are not advisable as But, very short SOAs (< 1s) are not advisable as the predicted additive effects upon the HRF of two the predicted additive effects upon the HRF of two closely occurring stimuli break down. closely occurring stimuli break down.

BriefStimulus

Undershoot

InitialUndershoot

Peak

So you can have events occurring even every 1-2 sec! So you can have events occurring even every 1-2 sec! But think of psychological validity! But think of psychological validity!

max. max. oxygenation: oxygenation: 4-6s post-4-6s post-stimulusstimulus

And how should my events be spaced?

optimal SOA

Choosing the best SOAChoosing the best SOA Optimal SOA depends on:Optimal SOA depends on: Probability of occurrence (design)Probability of occurrence (design) Whether one is looking for evoked responses Whether one is looking for evoked responses

per se or differences in evoked responses.per se or differences in evoked responses.

Generally SOAs that are small and randomly distributed are the most efficient.

Rapid presentation rates allow for the maintenance of a particular cognitive or attentional set, decrease the latitude that the subject has for engaging alternative strategies, or incidental processing.

Random SOAs ensure that

preparatory or anticipatory

factors do not confound event-

related responses and ensure a

uniform context in which events are

presented.

Probability

SOA

ONE TRIAL TYPE TWO TRIAL TYPES

Main effectDifferential responses

the most efficient SOA for differential responses is very small. the most efficient SOA for differential responses is very small. longer SOAs of around 16 s are necessary to estimate the responses longer SOAs of around 16 s are necessary to estimate the responses themselves.themselves.

Stationary Stochastic designs

What should I do if I am interested What should I do if I am interested in the main effects (‘evoked in the main effects (‘evoked

responses’)?responses’)?

You can use long SOA’s (around 16 You can use long SOA’s (around 16 secs!). But behaviourally this may be secs!). But behaviourally this may be inefficientinefficient

So you can introduce ‘null’ events and So you can introduce ‘null’ events and keep your SOA short.keep your SOA short.

These null events now provide a baseline These null events now provide a baseline against which the response to either trial against which the response to either trial type 1 or 2 can be estimated even using type 1 or 2 can be estimated even using a very small SOA.a very small SOA. (p=0.5 0.3) (p=0.5 0.3)

to identify areas that are activated by both event types

Here is what happens when you add null Here is what happens when you add null events…events…

Random

Note that although null events increase efficiency for main Note that although null events increase efficiency for main effects (at short SOA’s), they slightly decrease efficiency for effects (at short SOA’s), they slightly decrease efficiency for differential effectsdifferential effects

What should I do if I am interested in the differential effects?What should I do if I am interested in the differential effects?

For very short SOA’s use a randomised designFor very short SOA’s use a randomised designBut for medium SOA’s a permuted (4-6sec) or an alternating (8sec) design is betterBut for medium SOA’s a permuted (4-6sec) or an alternating (8sec) design is better

To sum up: Remember To sum up: Remember that…that…

Blocked designs generally more efficientBlocked designs generally more efficient Some random event-related designs are Some random event-related designs are

much better than others.much better than others. Different design is appropriate depending Different design is appropriate depending

on what you want to optimize. on what you want to optimize.

Critical properties to optimizeCritical properties to optimize Ordering of trials Ordering of trials spacing between stimulispacing between stimuli

Timing of the SOAs in relation to the Timing of the SOAs in relation to the TRTR

If the TR (Repetition Time of slice collection) is divisible by the SOA then data If the TR (Repetition Time of slice collection) is divisible by the SOA then data collected for each event will be from the same slices, at the same points along the collected for each event will be from the same slices, at the same points along the HRF.HRF.

Therefore, either choose a TR and SOA that are not divisible or introduce a ‘jitter’ Therefore, either choose a TR and SOA that are not divisible or introduce a ‘jitter’ such that the SOA is randomly shifted.such that the SOA is randomly shifted.

Scans TR = 4s

Stimulus (synchronous) SOA=8s

Stimulus (asynchronous) SOA=6s

Stimulus (random jitter)

Temporal Filtering: The High Pass Temporal Filtering: The High Pass FilterFilter

A temporal filter is used in fMRI to get A temporal filter is used in fMRI to get rid of noise, thus increasing the rid of noise, thus increasing the efficiency of the data.efficiency of the data.

Non-neuronal noise tends to be of Non-neuronal noise tends to be of low-frequency, including ‘scanner low-frequency, including ‘scanner drift’ and physiological phenomenon. drift’ and physiological phenomenon.

Applying a high pass filter means that Applying a high pass filter means that parameters that occur at a slow rate parameters that occur at a slow rate are removed from the analysis.are removed from the analysis.

The default high pass filter in SPM is The default high pass filter in SPM is 128s, thus if you have experimental 128s, thus if you have experimental events occurring less frequently than events occurring less frequently than once every 128s then the associated once every 128s then the associated signal will be removed by the filter!! signal will be removed by the filter!!

SourcesSources

SummarySummary

Blocked designs are generally the most efficient, but Blocked designs are generally the most efficient, but blocked designs have restrictions.blocked designs have restrictions.

For event-related designs, dynamic stochastic presentation For event-related designs, dynamic stochastic presentation of stimuli is most efficient.of stimuli is most efficient.

However, the most optimal design for your data depends on However, the most optimal design for your data depends on the SOA that you use. The general rule is the smaller your the SOA that you use. The general rule is the smaller your SOA the better, but sometimes a small SOA may not be SOA the better, but sometimes a small SOA may not be possible. possible.

Also, the most optimal design for one contrast may not be Also, the most optimal design for one contrast may not be optimal for another e.g. the inclusion of null events optimal for another e.g. the inclusion of null events improves the efficiency of main effects at short SOAs, at the improves the efficiency of main effects at short SOAs, at the cost of efficiency for differential effects.cost of efficiency for differential effects.

Finally, there is no point scanning two tasks to look for Finally, there is no point scanning two tasks to look for differences between them if they are too different or too differences between them if they are too different or too similar.similar.