Biomarkers: the Good, the Bad and the Ambiguous Enrique F. Schisterman, PhD Epidemiology Branch – DESPR – NICHD 1

Biomarkers: the Good, the Bad and the AmbiguousEnrique F. Schisterman, PhD

Epidemiology Branch – DESPR – NICHD

1

In Memory of My Friend Sholom Wacholder

2

“I never feel I truly understand something if I can’t explain it to someone else, orally or in writing,”

Outline

Background

Limit of Detection

Pooling Biomarkers – Hybrid Design

Calibration Curves

Lipid Standardization (if we have time)

Conclusions

3

Background

Biomarker: A specific physical trait used to measure or indicate the effects or progress of a disease or condition

Newly developed laboratory methods expand the number of biomarkers on a daily basis Cost Measurement Error Causal Link to Disease

4

Motivation

Preliminary analysis of salivary concentrations of cortisol from the LIFE study

P=0.04Shipment 1 n Mean SDMichigan 85 0.40 0.19Texas 142 0.57 0.79

5

Cortisol by Site & Plate6

02

46

8

Ship1 Ship2 Ship1 Ship2 Ship3

1 2 3 4 5 1 2 3 1 2 3 4 5 1 2 1 2 3 4 5 6 7 8

Michigan Texas

ug/

dL

Graphs by Site

ROC curve—background

Before a marker is used its discriminating accuracy needs to be evaluated

Most commonly used tool - Receiver Operating Characteristic (ROC) curve.

We consider only continuous markers

Distribution of marker in Healthy and Disease

c c c

* ROC curve – graph of (1-p,q) for all c.

ROC(1-p)=1-FD(FH-1(p))

* ROC defined over all possible thresholds gives the entire range of possible sensitivities and specificities.

* Diagnostic accuracy evaluated using measures including the area under the curve (AUC AUC = Prob(XH<XD)) and Youden’s index (J)

ROC curve example Complete Separation

q(c)

1 – p(c)0 1

1

Chanc

e lin

e

Df

c q(c)p(c)

q(c) = 1 and p(c) = 1 for some c

ROC curve example Complete Overlap

q(c) = 1- p(c), for all c

q(c)

1 – p(c)0 1

1

Chanc

e lin

e

c q(c)p(c)

ROC curve example Partial Separation

Sensitivity by 1- Specificity

P( True Pos.) by P( False Pos.) across all c

q(c)

1 – p(c)0 1

1

Chance

lineDf

c q(c)p(c)

q(c)

1 – p(c)0 1

1

ROC: AUC

Area Under the Curve Overall discriminatory ability Area under the ROC curve

x

dxdyygxf

YXPAUC

)()(

)(

Do Common Laboratory Practices Affect our Estimates of Risk?

Limit of Detection

Measurement Error

Calibration Curves

Lipid Standardization

14

Outline

Background

Limit of Detection


Calibration Curves


Conclusions

15

Reporting of Biomarker Data

ID Z1 3.12 1.53 8.44 0.85 5.46 3.27 2.08 5.89 13.410 2.511 1.912 6.1

• Reporting threshold is equal to 2.2

ID Z1 3.12 ND3 8.44 ND5 5.46 3.27 ND8 5.89 13.410 2.511 ND12 6.1

Report values < threshold as ‘not detected’

ID Z1 3.12 1.13 8.44 1.15 5.46 3.27 1.18 5.89 13.410 2.511 1.112 6.1

Report values < threshold as one half the value of the threshold

16

Conventional Determination of the Limit of Detection (LOD)

6.15)3( blanksblanksLOD

BLANK SERIES• 10.0• 5.0• 8.1• 7.1• 4.0• 11.3• 12.0• 8.0• 7.7• 7.0Mean = 8.02Std Dev = 2.53

0 5 10 15 20

17

Example of LOD left-censored data

0 5 10 15 20 25 30 35

Blanks

“True” biomarker

Better LOD?

18

Example of LOD left-censored data

0 5 10 15 20 25 30 35

Blanks

“True” biomarkerObserved biomarker (samples)

19

CONTROLS

CASES

Why is this a problem?Comparisons of PCBs in cases and controls

Controls—mean PCB Cases—mean PCB

Effect size

LOD

Blanks

20

Approaches for LOD/ missing data

Simplest approach is substitution Under certain circumstances yield minimal bias Conventionally, values below the LOD are usually

1. replaced by zero, LOD, LOD/2, LOD/√22. excluded 3. retained

Model based approaches Likelihood models (Perkins et al., AJE 2007)

Multiple imputation (Harel et al., Epidemiology 2010)

Schisterman EF, Vexler A, Whitcomb BW, Liu A. AJE 2006

21

CONTROLS

CASES

Why is this a problem?Comparisons of PCBs in cases and controls

LOD

Impute what? 0 LODLOD/2

22

LOD Simulation

Purpose: To evaluate the effect of the handling of values below the LOD on risk estimates

Simulated data from a normal and log normal distribution and varied:

Effect size Variance of PCBs in the exposure group LOD level Measurement error mean and variance

23

Effect of Handling of Values < LOD on %Bias

*LOD “low” indicates 1.6 SDs below the mean of controls, resulting in imputed values for a small number of data points. LOD “high” indicates 1 SD above the mean of the controls, resulting in imputed values for a large number of both controls and cases

24

Effect size = .25 SD

Method for values < LOD LOD high LOD low 1. Replace by a. Zero -59.0 -25.1 b. LOD -187.1 -40.8

c. LOD/2 -19.3 -18.1 d. LOD/2 -17.1 -15.9

2. Exclude (truncated) -314.2 -265.3 3. Retain (observed) -11.5 -11.7

LOD—Conclusions

Choice of how to handle values below the LOD can result in a loss of accuracy in estimating risk

Retaining observed values below the LOD produces the least biased estimates

Substitution of LOD/√2 for values below the LOD produces not terribly biased estimates

25

How the LOD Affects the ROC Curve?26

LOD: ROC curve

Properly accounting for the mass yields a ROC curve bias toward the null that is identical for any choice of a<d.

Xf

a

ROC

* Schisterman, et al.(2005)

MLE to correct for LOD

Normal Case

MLE’s of dist.parameters

Yk

jYYYYjY

Y

YYYYY

knz

kCzL

1

2

22

1)(log)()(

log)|,(log

YYY d /)( where

* Gupta(1952)

Normal Case


MLE’s of ROCmeasures

InvarianceProperty

22 ˆˆ

ˆˆˆ

YX

YXCUA


Normal Case



CI’s for ROCmeasures

2nd Deriv. and Fisher Info.

22

22

1

YX

X

YX

X

AUC

22

22

1

YX

Y

YX

Y

AUC

NzCUA A

2

2

/

ˆ 1

22

2

1

)(1)ˆ(ˆ2

)(1)(

)(1)(

)()(

][)ˆ,ˆ()ˆ(

qp

q

qqp

ICovCov


Gamma Case


where

,)(log)(log)1(

log)(log)|,(log

1 1

**

Y

YY

Y

YY

n

knjYYY

n

knjjYjYY

YYYYYY

Gknzz

kCzL

Y

jYjY

zz

*

Y

dyeyG y

yY

0

1

)(

1)(

YY

d

* Harter and Moore(1967)


dtttCUA XY

Q

YX

YX 1

0

1 1

ˆ

ˆ

ˆ )()ˆ()ˆ(

)ˆˆ(ˆ

Gamma Case



InvarianceProperty


,)(

);1()ln(

)(1

)()1()(

)(

);1(21

)(

);1();1();1())(ln(

**

**

2112

*****

*

222

2

2***

2

2

11

ddq

q

dhdpII

q

dhddqdhd

dpI

q

ddd

d

dI

Gamma Case



CI’s for ROCmeasures

2nd Deriv. and Fisher Info.

NzCUA A

2

2

/

ˆ

X

X

X AUC

AUC

AUC

Y

Y

Y AUC

AUC

AUC

1

2221

12111][)ˆ,ˆ()ˆ(

II

IIICovCov θ


LOD: ROC curve

Corrected!

Yf

Xf

a

ROC

* Schisterman, et al.(2005)

Evaluation of MLE

0

1

10

Example

PCB114 levels of 28 cases 51 controls classifying women with and without endometriosis.

Empirical 0.701 (0.576, 0.827)

MLE 0.777 (0.655, 0.899)

*assuming gamma distributions and d=0.005

AUC

Outline

Background

Limit of Detection


Calibration Curves


Conclusions

36

What is pooling?

Physically combining several individual specimens to create a single mixed sample

Pooled samples are the average of the individual specimens

1

2 p

37

Random Sample of Biospecimens

RANDOM SAMPLE

Randomly select 20 samples

FULL DATA

N = 40 Individual Biospecimens

38

Pooling Biospecimens

POOLED DATA

40 samples in groups of 2

FULL DATA

N = 40 Individual Biospecimens

39

Un-pooled

0

0.2

0.4

0.6

0.8

1

-4 -3 -2 -1 0 1 2 3 4

Effect of Pooling on Markers

Pooled

0

0.2

0.4

0.6

0.8

1

-4 -3 -2 -1 0 1 2 3 4

40

Un-pooled

0

0.2

0.4

0.6

0.8

1

-4 -3 -2 -1 0 1 2 3 4

Pooled

0

0.2

0.4

0.6

0.8

1

-4 -3 -2 -1 0 1 2 3 4

0

20

40

60

80

100

120

0 2 4 6 8 10

# of Pooled Samples

# o

f A

ssa

ys o

f P

oo

led

Sa

mp

les

20

50

100

200

Number of assays required to reach equivalency on ROC Curves

Limit of Detection and Pooling

/i i

ii

x if x LODZ

N A if x LOD

Unpooled Specimens

Pooled Specimens

( ) ( )( )

( )

, if LOD

N/A, if LOD

p pp i i

i pi

x xZ

x

Un-pooled

0

0.2

0.4

0.6

0.8

1

-4 -3 -2 -1 0 1 2 3 4

Effect of Pooling on Markers Affected by an LOD

Pooled

0

0.2

0.4

0.6

0.8

1

-4 -3 -2 -1 0 1 2 3 4

43

Un-pooled

0

0.2

0.4

0.6

0.8

1

-4 -3 -2 -1 0 1 2 3 4

Pooled

0

0.2

0.4

0.6

0.8

1

-4 -3 -2 -1 0 1 2 3 4

Estimation Based on Pooling Using Likelihood Function

dXi

i

k

i

XfdXkNk

NL

:

)(Pr)!()!1(

!

0ˆ

);ˆ,ˆ(log,0

ˆ);ˆ,ˆ(log

LL

Differentiating with respect to

ˆ,ˆ

,

To obtain estimates of

Schisterman et al. Bioinformatics 2006

Efficiency of the Mean and Variance

Variance of Estimated Mean

Variance of Estimated Variance

FULL DATA POOLED RANDOMFULL DATA POOLED RANDOM

45

LOD below Mean

LOD below Mean

LOD above Mean

LOD above Mean

Pooling and Random Sampling

Pooling advantages Reduces the number of assays we need to test Efficiently estimates the mean Cost-effective

Random sampling advantages Reduces the number of assays we need to test Efficiently estimates the variance Cost-effective & easy to implement

46

Hybrid Design: Pooled—Unpooled

Creates a sample of both pooled and unpooled samples Takes advantage of the strengths of both the pooling and

random sampling designs

Reduces number of tests to perform Cuts overall costs Gains efficiency (by using pooling technique) Accounts for different types of measurement error without

replications

– Pooling error– Random measurement error– LOD

47

Unpooled: X1,…,X5

Pooled: Z1,…,Z15

Hybrid Sample S: X1,…,X5,Z1,…,Z15

Setup of Hybrid Design

Unpooled: X1,…,X[αn]

Pooled: Z1,…,Z[(1-α)n]

In GeneralHybrid Sample S: X1,…,X[αn],Z1,…,Z[(1-α)n]

α is the proportion of unpooled samples

48

Maximum Likelihood Estimators

Random Sampling Pooling

In order to estimate the variance, α cannot be zero.

Schisterman EF et al, Stat Med 2010

pn

zxni

xi

p

2)1(

2

2 ˆ

)1(

)ˆ(

ˆ

22

22

ˆ)1(ˆ

ˆ)1(ˆˆ

xz

xzx nn

znxn

n

xni

xi

x

2

2

)ˆ(ˆ

2

)1(

2

2

2

2

)(

ln)1(2

)(ln),,(

z

nixi

zx

nixi

xpxx

z

nx

n

49

Hybrid Design: Pooled—Unpooled

Create a sample of both pooled and un-pooled samples that takes advantage of the strengths of the pooled and random sample designs Reduce number of tests to perform Cut overall costs Gain efficiency (by using pooling technique) Accounts for different types of measurement

error without replicationsPooling errorRandom measurement error Limit of detection

Mathematical Setup

),(~ 2xxi NX NiX i 1

),0(~ 2p

pi Ne

pi

ip

pijii eX

pZ

1)1(

1

2

2

,~ px

xi pNZ

Individual Samples

Pooled Samples

p is the pooling group size

Average of p individual samples pooled together

Pooling error

Maximum Likelihood Estimators

22

(1 )

2 2

( )( )( , , ) ln (1 ) ln

2 2

i xi xi ni n

x x p x zx z

zxn n

Random Sampling Pooling

22

22

ˆ)1(ˆ

ˆ)1(ˆˆ

xz

xzx nn

znxn

n

xni

xi

x

2

2

)ˆ(ˆ

2

2(1 )2

ˆ( )ˆ

ˆ(1 )

i xi n x

p

z

n p

In order to estimate the variance, α cannot be zero.

Add in Measurement Error

),0(~ 2M

Mi Ne

Individual Samples + ME

Pooled Samples + ME

p is the pooling group size

Measurement error

* 2 2~ ( , )i x x MX N

22

2* ,~ Mp

xxi p

NZ

Hybrid Design Example: IL-6

Measured IL-6 on 40 MI cases and 40 controls

Biological specimens were randomly pooled in groups of 2, for the cases and controls separately, and remeasured

We want to evaluate the discriminating ability of this biomarker in terms of AUC

54

Hybrid Design Example: IL-6

n αx αy AÛC Var(AÛC)

Empirical 40 1.00 1.00 0.640 0.0036

Hybrid design: Optimal α 20 0.40 0.35 0.621 0.0049

Random sample: α=1 20 1.00 1.00 0.641 0.0071

Hybrid design reduced the variability of Var(AÛC) by 32% as compared to taking only a random sample

55

Summary—Hybrid Design

Hybrid design is a more efficient way to estimate the mean and variance of a population

Cost-effective

Yields estimate of measurement error without requiring repeated measurements

Here we focus on normally distributed data, but can be applied to other distributions as well

56

Outline

Background

Limit of Detection


Calibration Curves


Conclusions

57

Measurement of G-CSF

Chemiluminescence assays 96-well plate Antibody against the biomarker of interest Set of standards of known biomarker concentration

included in each batch Set of samples (concentration unknown) Light emitting molecule binds to bound biomarker

58

59

Measurement of Cytokines

Cytokines are not measured directly Antibodies against analyte(s) coat wells

60


Samples added, analyte binds to antibodies Unbound proteins are washed away

61


A ‘tag’ is added to the assay that binds to the protein – antibody complex that produces color

62


A ‘tag’ is added to the assay that binds to the protein – antibody complex that produces color

The intensity of the color is measured

63

ELISA/Multiplex Layout

Step 1: prepare antibodies mixture and add to plate Step 2: prepare calibrators, add to plate Step 3: prepare unknowns, add to plate

64

Use of Chemiluminescence Assays for Measuring Protein Concentrations

Use calibration to convert relative measures to the desired unit of concentration From optical density in relative fluorescence units (RFU)

to concentration in pg/mL

Current practice is per assay calibration Results in potentially large calibration datasets used only

minimally in current practice

65

Calibrating the Assay: The Standard Curve

66

Calibrating the Assay: The Standard Curve

The human G-CSF standard curve is provided only for demonstration

A standard curve must be generated each time an assay is run, utilizing values from the Standard Value Card included in the Base Kit

Potential variation in the relation between relative fluorescence and concentration Chromophore potentially affected by temperature, humidity,

etc.

67

G-CSF and Miscarriage in the CPP

Case-control study nested in the Collaborative Perinatal Project study cohort 462 miscarriage cases 482 non-miscarriage controls

Serum biospecimens from early pregnancy, prior to miscarriage onset

For n = 944, 24 assays were used

68

69

Unadjusted model Adjusted model OR [95% CI] OR [95% CI] Factor G-CSF 0.84 [0.72, 0.99] 0.78 [0.64, 0.95]

This estimate is based on the conventional batch specific approach

70

Objective

Question: Is the current practice of standard batch-specific calibration the best use of information?

To evaluate the effect of different approaches for calibration models on risk estimation

To assess bias associated with different approaches

71

Data from the calibration experiments

24 batches, each with 7 known concentrations measured in replicate Batches varied by

Shape Location Agreement between replicates Presence of outliers

72

l ogod

1

2

3

4

l ogcon

0 1 2 3 4

Batch 1 Calibration Curve – G-CSF

Standard 1 – undiluted (conc = 6000 pg/mL)

Mea

sure

d op

tical

den

sity

Fixed ‘known’ concentration

*All calibration data (in log10)

73


l ogod

1

2

3

4

l ogcon

0 1 2 3 4

Standard 2 – 1/3rd dilution (conc = 2000 pg/mL)

Mea

sure

d op

tical

den

sity


74


l ogod

1

2

3

4

l ogcon

0 1 2 3 4

Mea

sure

d op

tical

den

sity


75


l ogod

1

2

3

4

l ogcon

0 1 2 3 4

Mea

sure

d op

tical

den

sity


76


l ogod

1

2

3

4

5

l ogcon

0 1 2 3 4

Mea

sure

d op

tical

den

sity


77


l ogod

1

2

3

4

l ogcon

0 1 2 3 4

Mea

sure

d op

tical

den

sity


78


l ogod

1

2

3

4

l ogcon

0 1 2 3 4

Mea

sure

d op

tical

den

sity


79


l ogod

1

2

3

4

l ogcon

0 1 2 3 4

Mea

sure

d op

tical

den

sity


80


l ogod

1

2

3

4

l ogcon

0 1 2 3 4

Mea

sure

d op

tical

den

sity


81


l ogod

1

2

3

4

l ogcon

0 1 2 3 4

Mea

sure

d op

tical

den

sity


82

All Calibration Curves Collapsed – G-CSF

l ogod

1

2

3

4

l ogcon

0 1 2 3 4

s i mpl e l i near

l ogod

1

2

3

4

5

l ogcon

0 1 2 3 4

Mea

sure

d op

tical

den

sity


83

Effect of Calibration Method on Logistic Regression Results

Calibration models As observed Outliers removed aOR 95%CI aOR 95%CI Forward Collapsed Linear 0.34 (0.13, 0.90) 0.27 (0.10, 0.73) Batch specific Linear 0.73 (0.46, 1.17) 0.60 (0.33, 1.10) Mixed model Linear 0.67 (0.39, 1.14) 0.56 (0.30, 1.06) Collapsed Curvilinear 0.21 (0.05, 0.84) 0.25 (0.09, 0.71) Batch specific Curvilinear 0.81 (0.57, 1.16) 0.98 (0.93, 1.02) Mixed model Curvilinear ~ ~ ~ ~ Reverse Collapsed Linear 0.37 (0.15, 0.91) 0.28 (0.11, 0.73) Batch specific Linear 0.63 (0.37, 1.11) 0.58 (0.32, 1.07) Mixed model Linear 0.43 (0.19, 0.94) 0.53 (0.27, 1.02) Collapsed Curvilinear 0.37 (0.15, 0.91) 0.29 (0.11, 0.74) Batch specific Curvilinear 0.86 (0.53, 1.41) 0.67 (0.38, 1.16) Mixed model Curvilinear 0.50 (0.27, 0.95) ~ ~

84

Simulation Study

1. Generate dataset with:

True biomarker concentration True effect on risk Overall relation between concentration and RFU Batch variability Occasional outliers

2. Simulate calibration experiments to estimate RFU – concentration relation according to each approach

3. Assess bias and variance of estimators from risk models

85

Simulation Study:The Biomarker

Biomarker: exp(X ~ N(5,1))

Miscarriage risk: OR = 1.05, 1.15 or 1.65 β={0.05, 0.14, 0.50}

Conc. and OD: OD determined througha single function

86

Summary of simulation study resultsComparison of shape, model for β = 0.14

CollapsedMixedBatch-specific

Linear Curvilinear Linear CurvilinearFORWARDS REVERSE

β̂

0.14

Whitcomb et al, Epidemiology 2010

87

Conclusions

Underestimation of effects due to calibration approach has broad implications

Use of conventional batch-specific approaches performed poorly Greatest bias to estimates in simulations Most prone to loss of data for batches with failure of

some calibration points

88

Outline

Background

Limit of Detection


Calibration Curves

Lipid Standardization (later if we have time)

Conclusions

89

Outline

Background

Limit of Detection


Calibration Curves


Conclusions

90

Do Common Laboratory Practices Affect our Estimates of Risk?

Limit of Detection Request the observed values Design away using hybrid methods and overcome cost, LOD and ME

Calibration Curves Study Design should include a calibration curve plan

Standardization Don’t do it!

YES!91

Questions?

Thank you!

Future Direction

Limit of detection: Linking the CV, AUC and calibration.

22

2

x

xR

x

xCV

CV

RAUC

x

X

x

X

2222

),(~ 2xXNX ),0(~ 2

NW

Reliability index could be applied such that

Coefficient of variation is used by labs during calibration

For normally distributed biomarkers, we can use these two summaries such that

93

Limit of detection: Linking the CV, AUC and calibration.

22

2

x

xR

x

xCV

CV

RAUC

x

X

x

X

2222

),(~ 2xXNX ),0(~ 2

NW

Reliability index could be applied such that

Coefficient of variation is used by labs during calibration

For normally distributed biomarkers, we can use these two summaries such that

94

We can investigate the relation between these factors easily because R and CV are relative measures.

95

This is of particular importance because for certain types of measurement processes, the Limit of detection is based on the CV and a more appropriate cut-point might be better generated using ROC methods for differentiating between signal and noise.

96

modeling calibration data

dependent/independent variables

Assay yields OD for observations in the main dataset

‘Forward’ (regression calibration) Estimate & use calibration equations in the same order, i.e.:

‘Reverse’ Estimate calibration equations, flip for calibration, i.e.:

)(ODfionConcentrat

)(ODfionConcentrat

)( ionconcentratfOD

97


treatment of batch

Approach to batch, OD as independent variable

use calibration data to model:

conijk = α + βODijkCollapsed

conijk = αk + βkODijk Batch-specific effects

conijk = α* + β*ODijk + aj + bjODijkMixed effects

i = standard 1 – 7

j = replicate 1 – 2

k = batch 1 – 24

98


treatment of batch

Approach to batch, concentration as dependent variable

use calibration data to model:

ODijk = α + βconijkCollapsed

ODijk = αk + βkconijk Batch-specific effects

ODijk = α* + β*conijk + ak + bkconijkMixed effects

i = standard 1 – 7

j = replicate 1 – 2

k = batch 1 – 24

99

Data calibration

Parameters estimated for linear and curvilinear models

For concentration as dependent Unknown concentrations calibrated from estimated f(OD)

)(ˆˆ ODionconcentrat

)(ˆˆ ODionconcentrat

100

Data calibration


For concentration as dependent Unknown concentrations calibrated from estimated f(OD)

221 )(ˆ)(ˆˆ ODODionconcentrat

221 )(ˆ)(ˆˆ ODODionconcentrat

101

Data calibration


For OD as dependent Unknown concentrations calibrated as:

ˆˆ

OD

con

)(ˆˆ ionconcentratOD

102

Data calibration


For OD as dependent Unknown concentrations calibrated as:

22

221

ˆ2

ˆ)ˆ(4ˆ

OD

con

221 )(ˆ)(ˆˆ conconOD

103

Simulation study the calibration relation

7 known concentrations in replicate/batchMeasured OD for each true concentration

True overall relation by 4 parameter logistic

4

3

212

1

i

ix

y

34

213

2

01

at slope2)(at

)(lim

)(lim

xfx

xf

xf

x

x

34

213

2

01

at slope2)(at

)(lim

)(lim

xfx

xf

xf

x

x

CONCx

ODy

104

Simulation study the calibration relation

7 known concentrations in replicate/batchMeasured OD for each true concentration

True overall relation by 4 parameter logistic

Random batch variability Random failures with frequency 0.01

4

3

212

1

ixOD

105

Calibration approaches

Calibration models varied as regards:1. Dependent/independent variable

– Measured OD and fixed known amount

2. Shape (linear or quadratic in log)3. Treatment of batch

Disregard (collapsed) Model as fixed effect (batch-specific) Model as random effect

106

Results

Calibration models Plots of cross-classification

Effects on risk estimation Simulation study

107

s i mpl e l i near

l ogod

1

2

3

4

5

l ogcon

0 1 2 3 4

All batches combined – outliers in the calibration set

0 1 2 3

1

108

s i mpl e l i near

l ogod

1

2

3

4

5

l ogcon

0 1 2 3 4

All batches combined – outliers in the calibration set

0 1 2 3

1

109

Bat ch number of assay 1 2 3 4 5 6 7 89 10 11 12 13 14 15 16

17 18 19 20 21 22 23 24

A A A B B B C C C D D D E E E F F F G G G H H HI I I J J J K K K L L L M M M N N N O O O P P PQ Q Q R R R S S S T T T U U U V V V W W W

l ogod

1

2

3

4

5

l ogcon

0 1 2 3 4 5

AA

A

A

AA

AA

AA

AA

A

A

BB

B

B

BB

BB

BB

B

B

BB

CC

CC

CC

CC

CC

CC

CC

DD

DD

DD

DD

D

D

DD

D

D

E

E

EE

EE

EE

E

E

EE

EE

F

F

FF

FF

FF

FF

FF

FF

GG

GG

GG

GG

GG

GG

GG

HH

HH

HH

HH

HH

HH

HH

I

I II

II

II

II

I

I

II

JJ

JJ

JJ

JJ

J

JJJ

J

J

KK

K

K

KK

KK

K

K

K

K

KK

LL

LL

LL

LL

LL

LL

LL

MM

MM

MM

MM

MM

MM

MM

NN

NN

NN

NN

NN

NN

NN

OO

O

O

OO

OO

O

O

O

O

OO

Q

Q

QQ

QQ

QQ

QQ

QQ

QQ

RR

R

RR

R

R

RR

RR

RR

SS

SS

SS

SS

SS

SS

S

S

TT

TT

TT

TT

TT

TT

TT

UU

UU

U

U

U

U

UU

UU

U

U

VV

VV

VV

VV

VV

VV

V

V

WW

WW

WW

WW

WW

WW

WW

s i mpl e l i near


17 18 19 20 21 22 23 24


l ogod

1

2

3

4

5

l ogcon

0 1 2 3 4

AA

A

A

AA

AA

AA

AA

A

A

BB

B

B

BB

BB

BB

B

B

BB

CC

CC

CC

CC

CC

CC

CC

DD

DD

DD

DD

D

D

DD

D

D

E

E

EE

EE

EE

E

E

EE

EE

FF

FF

FF

FF

FF

FF

FF

GG

GG

GG

GG

GG

GG

GG

HH

HH

HH

HH

HH

HH

HH

I

I II

II

II

II

I

I

II

JJ

JJ

JJ

JJ

J

JJJ

JJ

KK

K

K

KK

KK

K

K

K

K

KK

LL

LL

LL

LL

LL

LL

LL

MM

MM

MM

MM

MM

MM

MM

NN

NN

NN

NN

NN

NN

NN

OO

O

O

OO

OO

O

O

O

O

OO

Q

Q

QQ

QQ

QQ

QQ

QQ

QQ

RR

R

RR

R

R

RR

RR

RR

SS

SS

SS

SS

SS

SS

S

S

TT

TT

TT

TT

TT

TT

TT

UU

UU

U

U

U

U

UU

UU

U

U

VV

VV

VV

VV

VV

VV

V

V

WW

WW

WW

WW

WW

WW

WW

All calibration data (in log10), with batch-specific regression

All batches combined outliers in the calibration set

110


17 18 19 20 21 22 23 24


l ogod

1

2

3

4

5

l ogcon

0 1 2 3 4 5

AA

A

A

AA

AA

AA

AA

A

A

BB

B

B

BB

BB

BB

B

B

BB

CC

CC

CC

CC

CC

CC

CC

DD

DD

DD

DD

D

D

DD

D

D

E

E

EE

EE

EE

E

E

EE

EE

F

F

FF

FF

FF

FF

FF

FF

GG

GG

GG

GG

GG

GG

GG

HH

HH

HH

HH

HH

HH

HH

I

I II

II

II

II

I

I

II

JJ

JJ

JJ

JJ

J

JJJ

J

J

KK

K

K

KK

KK

K

K

K

K

KK

LL

LL

LL

LL

LL

LL

LL

MM

MM

MM

MM

MM

MM

MM

NN

NN

NN

NN

NN

NN

NN

OO

O

O

OO

OO

O

O

O

O

OO

Q

Q

QQ

QQ

QQ

QQ

QQ

QQ

RR

R

RR

R

R

RR

RR

RR

SS

SS

SS

SS

SS

SS

S

S

TT

TT

TT

TT

TT

TT

TT

UU

UU

U

U

U

U

UU

UU

U

U

VV

VV

VV

VV

VV

VV

V

V

WW

WW

WW

WW

WW

WW

WW

s i mpl e l i near


17 18 19 20 21 22 23 24


l ogod

1

2

3

4

5

l ogcon

0 1 2 3 4

AA

A

A

AA

AA

AA

AA

A

A

BB

B

B

BB

BB

BB

B

B

BB

CC

CC

CC

CC

CC

CC

CC

DD

DD

DD

DD

D

D

DD

D

D

E

E

EE

EE

EE

E

E

EE

EE

FF

FF

FF

FF

FF

FF

FF

GG

GG

GG

GG

GG

GG

GG

HH

HH

HH

HH

HH

HH

HH

I

I II

II

II

II

I

I

II

JJ

JJ

JJ

JJ

J

JJJ

JJ

KK

K

K

KK

KK

K

K

K

K

KK

LL

LL

LL

LL

LL

LL

LL

MM

MM

MM

MM

MM

MM

MM

NN

NN

NN

NN

NN

NN

NN

OO

O

O

OO

OO

O

O

O

O

OO

Q

Q

QQ

QQ

QQ

QQ

QQ

QQ

RR

R

RR

R

R

RR

RR

RR

SS

SS

SS

SS

SS

SS

S

S

TT

TT

TT

TT

TT

TT

TT

UU

UU

U

U

U

UU

UU

U

U

VV

VV

VV

VV

VV

VV

V

WW

WW

WW

WW

WW

WW

WW

All calibration data (in log10), with batch-specific regression

All batches combined outliers in the calibration set

111

Impact of calibration on risk estimationoutput from logistic regression Assay response data were calibrated to convert OD

to concentration according to each of the previously described methods

Logistic regression relating miscarriage risk to log10 levels of GCSF were performed

112

Documents

Biomarkers: the Good, the Bad and the Ambiguous Enrique F. Schisterman, PhD Epidemiology Branch – DESPR – NICHD 1