Upload
sharleen-underwood
View
217
Download
1
Embed Size (px)
Citation preview
Analyzing the Power and Error of Listeria monocytogenes Growth
Challenge StudiesMark Powell
U.S. Department of Agriculture
Washington, DC
IAFP 2009, Grapevine, TX, 12-15 July
Introduction
• For ready-to-eat (RTE) foods that do not support growth of L. monocytogenes, food safety criteria limit of 100 colony forming units (cfu)/g.– EC Regulation 2073/2005– FDA (2008) draft compliance policy guide– Codex (2009) microbiological criteria
• For RTE foods that do support growth of L. monocytogenes, “zero tolerance” (i.e., not detected in a regulatory sample).
• Design and interpretation of challenge studies to determine whether RTE are unable to support growth of L. monocytogenes.
Introduction
• Type I (F+) error (α): probability that H0 is rejected when true.
• Type II (F-) error (β): probability that H0 is not rejected when Ha is true.
• Power = (1-β). • By convention, α ≤ 0.05 and (1-β) ≥ 0.8
Fixed Exceedance Values
• To distinguish real growth from measurement uncertainty, L. monocytogenes challenge study protocols apply a fixed exceedance value: difference (δ) < M.
• EU/CRL (2008): difference between the initial and final sample median concentrations < 0.5 log10 cfu/g for all batches tested (Mm = 0.5 log).
• CCFH (2009): ≤ (on average) 0.5 log10 cfu/g increase for at least the expected shelf life (Mxbar = 0.5 log).
• FDA (2008): < 1 log10 increase during replicate trials (assume Mxbar = 1 log).
Fixed Exceedance Values
• M ~ ISO “expanded uncertainty” (U)
• x ± U = x ± 2σx
– where σx = std. error of meas. uncertainty
• 2 (k factor) ≈ z(1-0.05/2) = 1.96
– α = 0.05; 2-sided interval
• If σx = 0.25 log, → M = 0.5 log
• If σx = 0.50 log, → M = 1.0 log
Variance of a difference
• If two means are independent:
– where:
– Assuming equal σx and n:
222
00 txtxtxtx
nxx /22
2n if ONLY 2 2
0 x
xtxtx n
Quantitative Measurement Uncertainty
• σx ≠ 0.25 or 0.5 or any other fixed value.• EC, FDA, and CCFH reference ISO
Method 11290-2 for enumerating L. monocytogenes in RTE foods.
• Scotter et al (2001): std dev reproducibility (sR) = 0.17 - 0.45 log cfu/g in food samples.
• sR: an intra-laboratory measure of quantitative measurement uncertainty.
Challenge Study Designs Differ
• Number of sampling times• Number of batches• Experiment-wise α depends on:
– Number of comparisons– Whether multiple comparisons are
independent or dependent.• Independent: (μfinal – μinitial) X multiple
batches• Dependent: μ(t) – μ(t0) within a batch
Challenge Study Designs Differ
• EU/CRL (2008): k = 2 sampling times (initial and final), b ≥ 3 batches, sample size (n) = 3 samples per sampling time.– c ≥ 3 multiple, independent pair-wise comparisons.– std dev w/in batch < 0.3 log at t0.
• FDA cites Scott et al. (2005): k = 5-7 sampling times, sample size (n) = 2-3 samples per sampling time.– c = k-1 dependent pair-wise comparisons per trial
(μ(t) – μ(t0)).– No minimum number of batches.
Type I error for fixed exceedance value (Mxbar)
• For a single comparison test of H0: δ ≤ 0:
• For multiple independent comparisons:
• For multiple dependent comparisons, Monte Carlo simulation, with α = proportion (F+)
• Based on Scotter et al (2001), consider σx from 0.15 log cfu/g to 0.50 log cfu/g
nMMtxtxp x
xx
2
0
2,0|1
c 11
Type I error for difference in means fixed exceedance value (Mxbar) = 0.5 log cfu/g
** p<0.01
std. dev. (log
cfu/g)
sample size (n) = 2 sample size (n) = 3
independent comparisons (c) independent comparisons (c)
1 2 3 4 5 6 1 2 3 4 5 6
p(type I error) ≤ α
0.15 ** ** ** ** ** ** ** ** ** ** ** **
0.20 0.01 0.01 0.02 0.02 0.03 0.04 ** ** ** ** 0.01 0.01
0.25 0.02 0.04 0.07 0.09 0.11 0.13 0.01 0.01 0.02 0.03 0.04 0.04
0.30 0.05 0.09 0.14 0.18 0.22 0.25 0.02 0.04 0.06 0.08 0.10 0.12
0.35 0.08 0.15 0.21 0.27 0.33 0.38 0.04 0.08 0.12 0.15 0.19 0.22
0.40 0.11 0.20 0.28 0.36 0.43 0.49 0.06 0.12 0.18 0.23 0.28 0.32
0.45 0.13 0.25 0.35 0.44 0.51 0.58 0.09 0.17 0.24 0.30 0.36 0.42
0.50 0.16 0.29 0.40 0.50 0.58 0.65 0.11 0.21 0.30 0.37 0.44 0.50
Type I error for difference in means fixed exceedance value (Mxbar) = 0.5 log cfu/g
** p<0.01
std. dev. (log
cfu/g)
sample size (n) = 2 sample size (n) = 3
dependent comparisons (c) dependent comparisons (c)
1 2 3 4 5 6 1 2 3 4 5 6
p(type I error) ≤ α
0.15 ** ** ** ** ** ** ** ** ** ** ** **
0.20 0.01 0.01 0.02 0.02 0.03 0.03 ** ** ** ** 0.01 0.01
0.25 0.02 0.04 0.06 0.07 0.08 0.10 0.01 0.01 0.02 0.02 0.03 0.03
0.30 0.05 0.09 0.11 0.14 0.16 0.18 0.02 0.04 0.05 0.06 0.08 0.09
0.35 0.08 0.13 0.17 0.21 0.23 0.26 0.04 0.07 0.09 0.12 0.14 0.15
0.40 0.11 0.18 0.23 0.27 0.31 0.33 0.06 0.11 0.14 0.17 0.20 0.22
0.45 0.13 0.22 0.28 0.33 0.36 0.39 0.09 0.14 0.19 0.23 0.26 0.29
0.50 0.16 0.25 0.32 0.37 0.41 0.45 0.11 0.18 0.24 0.28 0.32 0.34
Power of F-test for One-Way ANOVA
• SAS© PROC Power
– where:– Fω = non-central F dist– Fcrit = critical value of the F dist with k-1 and k(n-1) df
– ω (non-centrality parameter) = – H0: μi = μ for all i– Ha: μmax – μmin = δ– Power depends on δ and growth pattern under Ha
,,|111 21 dfdfFF crit
2
1
2 /.
k
iin
Pattern that maximizes power for δ = 1 log
0
0.5
1
1.5
2
2.5
1 2 3 4 5 6
time
log1
0 cf
u/g
Pattern that minimizes power for δ = 1 log
0
0.5
1
1.5
2
2.5
1 2 3 4 5 6
time
log1
0 cf
u/g
Power curves for one-way ANOVA F-test (α = 0.05, δ = 1 log cfu/g)
with sample size n = 2 and sampling times k = 2-7
0.0
0.10.2
0.3
0.4
0.50.6
0.7
0.80.9
1.0
0.0 0.1 0.2 0.3 0.4 0.5 0.6
Std Dev (log10 cfu/g)
Po
we
r
n=2 k=7
n=2 k=6
n=2 k=5
n=2 k=4
n=2 k=3
n=2 k=2
max
min
Power curves for one-way ANOVA F-test (α = 0.05, δ = 1 log cfu/g)
with sample size n = 3 and sampling times k = 2-7
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
0.0 0.1 0.2 0.3 0.4 0.5 0.6
Std Dev (log10 cfu/g)
Po
we
r
n=3 k=7
n=3 k=6
n=3 k=5
n=3 k=4
n=3 k=3
n=3 k=2
max
min
Conclusions
• Applying any fixed acceptance criteria exceedance value (e.g., less than a 0.5 log or 1 log increase) to distinguish real growth from quantitative measurement uncertainty over different experimental designs and/or measurement uncertainty values implies highly inconsistent type I error probabilities.
Conclusions
• None of the L. monocytogenes growth challenge study designs currently being considered are likely to provide an F-test with α = 0.05 and power ≥ 0.8 to detect a 1 log increase in mean concentration over the entire range of measurement uncertainty values for enumeration of L. monocytogenes reported in food samples in a validation study of ISO Method 11290-2.
Conclusions
• Satisfying these conventional experimental design criteria would require a larger sample size, lower measurement uncertainty, or both.
Thank you