Design and analysis of experiments with a laboratory phase subsequent to an initial phase

Chris BrienUniversity of South Australia

Bronwyn Harch

Ray CorrellCSIRO Mathematical and Information Sciences

[email protected]

Design and analysis of experiments with a laboratory phase subsequent to an initial phase

2

Outline

1. Designing two-phase experimentsa) A biodiversity example

b) When first-phase factors do not divide lab factors

2. Trend adjustment in the biodiversity example

3. Taking trend into account in design

4. Duplicates

3

Notation

Factor relationshipsA*B factors A and B are crossedA/B factor B is nested within A

Generalized factorAB is the ab-level factor formed from the combinations of A with a levels and B with b levels

Symbolic mixed modelFixed terms : random terms (A*B : Blocks/Runs)

A*B = A + B + ABA/B/C = A + AB + ABC

Sources in ANOVA tableA#B a source for the interaction of A and BB[A] a source for the effects of B nested within A

4

1. Designing two-phase experiments

Two-phase experiments as introduced by McIntyre (1955):Consider special case of second phase a

laboratory phase

5

General considerations Need to randomize laboratory phase so involve two

randomizations: 1st-phase treatments to 1st-phase, unrandomized factors latter to unrandomized, laboratory factors

Often have a sequence of analyses to be performed and how should one group these over time.

Fundamental difference between 1st and 2nd randomizations 1st has randomized factors crossed and nested 2nd has two sets of factors and all combinations of the two sets are

not observable; within sets are crossed or nested tendency to ignore 1st phase, unrandomized factors.

Categories of designs Lab phase factors purely hierarchical or involve crossed rows and

columns; Two-phase randomizations are composed or randomized-inclusive

(Brien & Bailey, 2006); related to whether 1st-phase, unrandomized factors divide laboratory unrandomized factors

Treatments added in laboratory phase or not Lab duplicates included or not

6

1a) A Biodiversity example

Effect of tillage treatments on bacterial and fungal diversity

Two-phase experiment: field and laboratory phase

Field phase: 2 tillage treatments assigned to plots using

RCBD with 4 blocks 2 soil samples taken at each of 2 depths

2 4 2 2 = 32 samples

(Harch et al., 1997)

7

Laboratory phase: Then analysed soil samples in the lab using Gas

Chromatography - Fatty Acid Methyl Ester (GC-FAME) analysis

2 preprocessing methods randomized to 2 samples in each PlotDepth

All samples analysed twice — necessary? once on days 1 & 2; again on day 3

Day

1 2 am of 3 pm of 3

Int1 1 1 2 2

Int2 1 2 1 2

Blocks 1 & 2 3 & 4 1 & 2 3 & 4

In each Int2, 16 samples analyzed

8

Processing order within Int1Int2Analysis Method Block Plot Depth Analysis Method Block Plot Depth

1 ground A 1 0-5cm 9 sieved A 1 0-5cm




5 ground B 1 0-5cm 13 sieved B 1 0-5cm




Logical as similar to order obtained from field But confounding with systematic laboratory effects:

Preprocessing method effects Depth effects

Depths assigned to lowest level ─ sensible?

9

Int3 Int4 Int5 Int6 Method Block Plot Depth

2 1 1 1 sieved A 1 0-5cm

2 1 1 2 sieved A 1 5-10cm

2 1 2 1 sieved A 2 0-5cm

2 1 2 2 sieved A 2 5-10cm

2 2 1 1 sieved B 1 0-5cm

2 2 1 2 sieved B 1 5-10cm

2 2 2 1 sieved B 2 0-5cm

2 2 2 2 sieved B 2 5-10cm

Towards an analysis

Dashed arrows indicate systematic assignment

Int3 Int4 Int5 Int6 Method Block Plot Depth

1 1 1 1 ground A 1 0-5cm

1 1 1 2 ground A 1 5-10cm

1 1 2 1 ground A 2 0-5cm

1 1 2 2 ground A 2 5-10cm

1 2 1 1 ground B 1 0-5cm

1 2 1 2 ground B 1 5-10cm

1 2 2 1 ground B 2 0-5cm

1 2 2 2 ground B 2 5-10cm

2 Samples in B, P, D 4 Blocks

2 Plots in B2 Depths

32 samples

2 Tillage

2 field treats

2 Methods

2 lab treats

64 analyses

2 Int12 Int3 in I1, I2

2 Int2 in I12 Int4 in I1, I2, I3

2 Int5 in I1, I2, I3, I42 Int6 in I1, I2, I3, I4, I5

2 B1

2 B2 in B1

Int1 Int2 Block

1 1 1&2

1 2 3&4

2 1 1&2

2 2 3&4

64 analyses divided up hierarchically by 6 x 2-level factors Int1…Int6 of size 32, …, 2 analyses, respectively.

10

Analysis of example for

lab variability

Variability for: Int4 >Int5 > Int6 8 > 4 >2

Analyses Int1, Int2, Int3

small (< Int4)

Source df SSq MSq F p Int1 1 0.72 0.72 19.98 0.140 Int2[Int1] 2 58.84

Block 1 58.80 58.80 1621.46 0.016 Residual 1 0.04 0.04 0.02

Int3[Int1 Int2] 4

Sample[Block Plot Depth] 2 121.82

Method 1 121.28 121.28 224.89 0.042 Residual 1 0.54 0.54

Residual 2 3.19 1.59 0.43

Int4[Int1 tInt2 Int3] 8 64.54

Block 2 37.68 18.84 5.05 0.080

Sample[Block Plot Depth] 2 11.94 5.97 1.60 0.308

Residual 4 14.91 3.73 2.43 0.133

Int5[Int1 Int2 Int3 Int4] 16 56.72

Plot[Block] 4 43.22 Tillage 1 1.60 1.60 0.12 0.757 Residual 3 41.62 13.87


Tillage#Method 1 3.04 3.04 56.26 0.005 Residual 3 0.16 0.05

Residual 8 12.30 1.54 1.58 0.207

Int6[Int1 Int2 Int3 Int4 Int5] 32 209.67

Depth 1 137.66 137.66 141.75 <0.001 Block#Depth 3 13.74 4.58 4.72 0.015 Plot}#Depth[Block] 4 15.87

Tillage#Depth 1 3.87 3.87 0.97 0.398 Residual 3 12.00 4.00 1.9


Depth#Method 1 13.22 13.22 6.28 0.046 Tillage#Depth#Method 1 1.01 1.01 0.48 0.514 Residual 6 12.63 2.10

Residual 16 15.54 0.97

11

Alternative blocking for the biodiversity example

Want to assign the 32 samples to 64 analyses Consider with the experimenter:

1 Uninteresting effects — Blocks

2 Large effects — Depth?

3 Some treatments best changed infrequently —

Methods?

4 Period over which analyses effectively homogeneous

— 16 analyses? 4 analyses? 2 analyses?

12

Occasion 1 2 Analysis 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

Time Time 1 1 2 2 3 3 4 4

Alternative blocking for the biodiversity example For now, divide 64 analyses into 2 Occasions = Int1,

4 Times = Int2Int3, 8 Analyses = Int4Int5Int6

Blocks of 8 would be best as 2 Plots x 2 Depths x 2 Methods, but Blocks = Times too variable. Best if pairs of analyses in a block. Also Times are similar could take 4 Times x 2 Analyses.

Many other possibilities: e.g. blocks of size 4 with Depths randomized to pairs of blocks.

13

Proposed laboratory design

Organise 64 analyses into blocks of 8:

Randomization of field units ignores treats Two composed randomizations (Brien and Bailey, 2006)

Field treats to samples to analyses Two independent randomizations (Brien and Bailey, 2006)

Field and lab treats to samples

Experiment with hierarchical lab phase, composed randomizations, duplicates and

treatments added at laboratory phase.

4 Blocks2 Depths

2 Plots in B2 Samples in B, P, D

32 samples

2 Methods

2 lab treats

2 Tillage

2 field treats

64 analyses

2 Occasions4 Intervals in O

8 Analyses in O, I

Occasion 1 Interval 1 2 3 4 Time

1 1 2 2 3 4 3 5 6 4 7 8

14

Decomposition table for proposed design

Randomization-based mixed model (Brien & Bailey, 2006): Till*Meth*Dep : ((Blk/Plot)*Dep)/Sample – Dep + Occ/Int/Anl

Each of the 15 lines is a separate subspace in the final decomp-osition

Note Residual df determined by field phase

analyses tier

source df

Occasion 1

Interval[Occasion] 6

Analysis[O I] 56

samples tier

source df

Block 3

Residual 3

Depth 1

Block#Depth 3

Plot[Block] 4

Depth#Plot[Block] 4

Samples[B P D] 16

Residual 28

treatments tiers

source df

Tillage 1

Residual 3

Tillage#Depth 1

Residual 3

Method 1

Tillage#Method 1

Method#Depth 1

Tillage#Method#Depth 1

Residual 12

Important for design: shows confounding and apportionment of variability

Or Till*Meth*Dep : ((Blk/Till)*Dep)/Meth – Dep + Occ/Int/Anl

15

1b) When first-phase factors do not divide lab factors

Need to use a nonorthogonal design and two randomized-inclusive randomizations (Brien and Bailey, 2006)

Willow experiment (Peacock et al, 2003) Beetle damage inhibiting rust on willows? Glasshouse and lab phases Example here same problem but different details Will be an experiment with

hierarchical lab phase, randomized-inclusive randomizations, no duplicates and no treatments added at laboratory phase

16

Glasshouse: 60 locations each with a plant 12 damages to assign to locations. Only 6 locations per bench:

— Damages does not divide no. locations or benches so IBD— Use RIBD with v = 12, k = 6, E = 0.893, bound = 0.898.

Randomize between Reps, Benches within Reps and Locations within Benches.

Willow experiment (cont’d)

Rep I II III IV V Bench 1 2 1 2 1 2 1 2 1 2 1 2 3 4 1 3 2 4 1 4 2 3 1 2 3 4 1 3 2 4 6 7 8 5 5 7 6 8 8 7 5 6 5 6 7 8 7 5 8 6 11 12 9 10 9 11 10 12 10 9 11 12 12 9 10 11 9 11 10 12

60 locations

5 Reps6 Locations in R, B

2 Benches in R

12 treatments

12 Damages

17

Lab phase: disk/plant put onto 20 plates, 3 disks /plate Plates divided into 5 groups for processing on an Occasion Locations does not divide Cells

— divide 6 Locations into 2 sets of 3: cannot do this ignoring Damages RIBD related to 1st-phase (v = 12, k =3, r = 5, E = 0.698, bound = 0.721)

— In fact got this design using CycDesgN (Whittaker et al, 2002) and combined pairs of blocks to get 1st-phase.

— To include Locations, read numbers as Locations with these Damages.— Renumber Locations to L1 and L2 to identify those assigned same Plate.

Willow experiment (cont’d)

Rep III II V IV I Bench 2 1 1 2 1 2 2 1 1 2 1 2 3 4 1 3 2 4 1 4 2 3 1 2 3 4 1 3 2 4 6 7 8 5 5 7 6 8 8 7 5 6 5 6 7 8 7 5 8 6 11 12 9 10 9 11 10 12 10 9 11 12 12 9 10 11 9 11 10 12

Plate 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 Cell 2 3

5 Occasions3 Cells in O, P4 Plates in O

60 cells

Sometimes better design if allow for lab phase in designing 1st

60 locations

5 Reps6 Locations in R, B

2 Benches in R

12 treatments

12 Damages

2 L1

3 L2

L1 1 2 1 2 1 2 1 2 1 2 1 2 1 2 1 2 1 2 1 2

18

Decomposition table for proposed design

Each of the 6 lines is a separate subspace in the final decomposition. Note Residual df for Locations from 1st phase is 39 and has been

reduced to 29 in lab phase. s are strata variances or portions of E[MSq] from cells and s from

locations. Four estimable variance functions: O + R, OP + RB, OP + RBL, OPC +

RBL, although 2nd may be difficult. Randomization-based mixed model (Brien & Bailey, 2006) that corresponds

to estimable quantities: Damages : Rep/Benches/L1 + OccasionsPlatesCells. Must have Locations in the form of L1 in this model ─ i.e. cannot ignore

unrandomized factors from 1st phase.

cells tier

source df

Occasion 4

Plates[Occasion] 15

Cells[O P] 40

locations tier

source df

Rep 4

Benches[R] 5

L1[R B] 10

L2[R B] 40

treatments tiers

source df

Damages1 5

Damages2 8

Residual 2

Damages3 11

Residual 29

E[MSq]

O + R

OP + RB + q(D1)

OP + RBL + q(D2)

OP + RBL

OPC + RBL + q(D3)

OPC + RBL

21

2. Trend in the biodiversity example Trend can be a problem in laboratory phase. Is it here? Plot of Lab-only residuals in run order for 8 Analyses within

Times

Linear trend that varies evident Proposed design (4 x 2) is appropriate ( trend & low Times

variability) smallest Analysis Residual

Occasion 1 Analysis 1 2 3 4 5 6 7 8 Times

1 2 3 4

22

Trend adjustment for example REML analysis with vector of 1…8 for each Occasion Significant different linear trends (p < 0.001) Effect on fixed effects

Not adjusted Adjusted Sources Wald statistic p Wald statistic p Depth 30.73 <0.001 57.36 <0.001 Tillage 0.12 0.726 7190.13 <0.001 Method 214.24 <0.001 252.04 <0.001 Depth#Tillage 0.95 0.331 0.96 0.327 Depth#Method 10.46 0.001 8.27 0.004 Tillage#Method 2.82 0.093 12.53 <0.001 Depth#Tillage#Method 0.80 0.371 0.75 0.385 Trend adjustment reduced

Tillage effect from 0.99 to 0.07 Plot[Block] component from 13.25 to 0.001.

Low Plot[Block] df makes this dubious.

23

4. Taking trend into account in design Cox (1958, section 14.2) discusses trend

elimination: concludes that, where the estimation of trend not

required, use of blocking preferred to trend adjustment; Yeh, Bradley and Notz (1985) combine blocking for

trend and adjustment & provide trend-free and nearly trend-free designs with blocks allow for common quadratic trends within blocks minimize the effects of adjustment

Look at design of laboratory phase for field phase with RCBD, b = 3, v = 18 3 Occasions in lab phase to which 3 Blocks randomized allow for different linear & cubic Trends within each

Occasion

24

Different designs for blocks of 18 analyses

RCBD for this no. treats relatively efficient when adjusting for trend Blocks assigned to 3 Occasions × 6 Analyses (blocking perpendicular

to trend?) Use when Occasions variability low e.g. recalibration

Nearly Trend-Free (using Yeh, Bradley and Notz , 1985) worse than RCBD for different trends: optimal for common linear trend.

Still to investigate designs that protect against different trends.

Trend Different linear Different cubic Design E L E L mean (min, max) mean (min, max) v k 18

RCBD 0.91 (0.39, 0.64) 0.73 (0.20, 0.47) Blocks = 3 O x 6 A 0.96 (0.93, 0.97) 0.78 (0.23, 0.51) Nearly Trend-Free 0.90 (0.32, 0.50) 0.66 (0.08, 0.31)

Analysis 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 Occasions

1 2 3

25

Comparing RCBD with RIBDs for k = 6,9

2 2OIA O OI

2 2OI BP

OI BP2 2OIA OIA

set 1, 5

vary , ,

Use Relative Efficiencies = av. pairwise variance of RCBD to

RIBD for sets of generated data Generate using random model:

Y = Occasion + Interval[Occasion] +

Analyses[OccasionsInterval] + Plots[Blocks]

Interval 1 2 Analysis 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9

Occasions 1 2 3

Expect efficiency

k = 6 > k = 9 RIBD > RCBD

provided BP not dominant and OI is non-zero.

How much? BP < 10 OI ≥ 0.5 (very little extra

required, but after trend adjustment)

26

Resolvable design with cols & latinized rows [using CycDesgN (Whittaker et al, 2002), Intra E = 0.49]

Expect LRCD > RIBD if IA ≠ 0 and BP not dominant; How much? If IA > 1 irrespective of

OI. Again only small s.

Expect LRCD > RCD if Occasions different.

REs as BP (LRCD/RCD < 2 if BP ≥ 2.5).

Interval 1 2 3 Analysis 1 2 3 4 5 6 1 2 3 4 5 6 1 2 3 4 5 6 Occasion

1 5 14 12 18 2 7 4 15 1 16 10 9 11 13 8 3 6 17 2 8 6 15 10 16 3 13 17 2 11 5 7 9 14 18 4 1 12 3 9 17 4 11 13 1 8 18 14 3 12 6 15 10 7 2 5 16

27

4. Duplicates Commonly used, but only need in two-phase experiments

if Lab variation large compared to field. Possibilities:

Separated: analyze all & then reanalyze all in different random order

Nested: some analyzed & then these reanalyzed in a different random order

Crossed: some analyzed & then these reanalyzed in same order Consecutive: duplicate immediately follows first analysis Randomized: some analysed & everything randomized

From ANOVAs and REs to randomized, when adjusting for different cubic trends, conclude Separated duplicates superior, with nested duplicates 2nd best;

little gain in efficiency if OI ≤ 0.5 and BP is considerable; Crossed and consecutive duplicates perform poorly with RE < 1

often

28

5. Summary for lab phase design Two-phase: initial expt & lab phase

Leads to 2 randomizations: composed or r-inclusive related to whether 1st –phase, unrandomized factors divide laboratory, unrandomized factors

Use of pseudofactors with r-inclusive does not ignore field terms and makes explicit what has occurred

Adding treatments in lab phase leads to more randomizations Cannot improve on field design but can make worse

Important to have some idea of likely laboratory variation: Will there be recalibration or the like? Are consistent differences between and/or across Occasions likely? How does the magnitude of the field and laboratory variation compare? Are trends probable: common vs different; linear vs cubic?

Will laboratory duplicates be necessary and how will they be arranged? If yes, separated duplicates best but other arrangements may be OK.

RCBD will suffice if field variation >> lab variation, in which case duplicates unnecessary. after adjustment for trend, no extra laboratory variation, except Occasions can block across occasions when no Occasion differences

If Intervals differences, RIBD better than RCBD ─ not much needed. LRCD better than RIBD provided, after trend adjustment, moderate

consistent differences between Analyses across Occasions.

29

References Brien, C.J., and Bailey, R.A. (2006) Multiple randomizations (with

discussion). J. Roy. Statist. Soc., Ser. B, 68, 571–609. Cox, D.R. (1958) Planning of Experiments. New York, Wiley. John, J.A. and Williams, E.R. (1995) Cyclic and Computer Generated

Designs. Chapman & Hall, London. Harch, B.E., Correll, R.L., Meech, W., Kirkby, C.A. and Pankhurst, C.E.

(1997) Using the Gini coefficient with BIOLOG substrate utilisation data to provide an alternative quantitative measure for comparing bacterial soil communities. Journal of Microbial Methods, 30, 91–101.

McIntyre, G. (1955) Design and analysis of two phase experiments. Biometrics, 11, p.324–34.

Peacock, L., Hunter, P., Yap, M. and Arnold, G. (2003) Indirect interactions between rust (Melampsora epitea) and leaf beetle (Phratora vulgatissima) damage on Salix. Phytoparasitica, 31, 226–35.

Whitaker, D., Williams, E.R. and John, J.A. (2002) CycDesigN: A Package for the Computer Generation of Experimental Designs. (Version 2.0) CSIRO, Canberra, Australia. http://www.ffp.csiro.au/software

Yeh, C.-M., Bradely, R.A. and Notz, W.I. (1985) Nearly Trend-Free Block Designs. J. Amer. Statist. Assoc., 392, 985–92.

Documents

Design and analysis of experiments with a laboratory phase subsequent to an initial phase