A Bayesian Approach toParallelism Testing in BioassaySteven Novick, GlaxoSmithKlineHarry Yang, MedImmune LLC
Manuscript co-author
John Peterson, Director of statistics, GlaxoSmithKline
Warm up exercise
When are two lines parallel?Parallel: Being everywhere equidistant and not intersecting
Slope
Horizontal shift places one line on top of the other.
When are two curves parallel?Parallel: Being everywhere equidistant and not intersecting?
Can you tell by checking model parameters?
Horizontal shift places one curve on top of the other.
Where is parallelism important?Gottschalk and Dunn (2005)Determine if biological response(s) to two substances are similarDetermine if two different biological environments will give similar doseresponse curves to the same substance.
Compound screeningAssay development / optimizationBioassay standard curve
Screening for compound similar to gold-standardE.g., seeking new HIV compound with AZT-like efficacy, but different viral-mutation profile.
Desire for dose-response curves to be parallel.
Change to assay procedureE.g., change from fresh to frozen cells.
Want to provide same assay signal window.Desire for control curves to be parallel.
Assess validity of bioassayused for relative potency
Dilution must be parallel to original.Callahan and Sajjadi (2003)OriginalDilution
Replacing biological materials used in standard curveE.g., ELISA (enzyme-linked immunosorbent) assay to measure protein expression.Recombinant proteins used to make standard curve.Testing clinical sampleNew lot of recombinant proteins for standards. Check curves are parallelCalibrate new curve to match the old curve.
Potency often determined relative to a reference standard such as ratio of EC50Only meaningful if test sample behaves as a dilution or concentration of reference standard
Testing parallelism is required by revised USP Chapter and European Pharmacopeia
Linear: Two lots of Protein AEstimated ConcentrationsLot 2 is 1.4-fold higher than Lot 1=0.14Log10 Signal
If the lines are parallel
Shift Lot 2 line to the left by a calibration constant .
is log relative potency of Lot 2.Draft USP 2010
Testing for Parallelism in bioassay
Typical experimental designLog10 SignalSerial dilutions of each lot
Several replicates
Fit on single plate (no plate effect)
Tests for parallel curvesLinear model
Hauck et al, 2005; Gottschalk and Dunn, 2005
H0: b1 = b2H1: b1 b2May lack power with small sample sizeMight be too powerful for large sample sizeANOVA: T-test F goodness of fit test 2 goodness of fit test
A better ideaCallahan and Sajjadi 2003; Hauck et al. 2005Slopes are equivalent
H0: | b1 b2 | H1: | b1 b2 |<
Nonlinear: Two lots of protein BEstimated ConcentrationsLot 2 is 1.6-fold higher than Lot 1=0.21
Tests for parallel curves4-parameter logistic (FPL) model
Jonkman and Sidik 2009F-test goodness of fit statistic
H0: A1 = A2 and B1 = B2 and D1 = D2 H1: At least one parameter not equal
May lack power with small sample sizeMight be too powerful for large sample size
Calahan and Sajjadi 2003; Hauck et al. 2005; Jonkman and Sidik (2009)
Equivalence test for each parameter = intersection-union test
H0: |1 / 2| 1 or |1 / 2| 2H1: 1
Our proposalParallel Equivalence
Definition of ParallelTwo curves and are parallel if there exists a real number such that for all x.
Definition of Parallel Equivalence
Two curves and are parallel equivalent if there exists a real number such that for all x [xL, xU].
It follows that two curves are parallel equivalent if there exists a real number such that
It also follows that
Are these two lines parallel enough when xL < x < xU ?< ?
Linear-model solutionLinear model:
Just check the endpoints
Parallel equivalence = slope equivalenceSame as testing: | b1 b2 |< wlog
Parallel EquivalenceFPL model:
No closed-form solution.Simple two-dimensional minimax procedure.
< ?Are these two curves parallel enough when xL < x < xU ?
Testing for parallel equivalenceH0:
H1:
Proposed metric (Bayesian posterior probability):
Computing the Bayesian posterior probabilityFor each curve, assume
Data distribution:Prior distribution:i =1, 2 = reference or samplej = 1, 2, , N = observationsPosterior distribution proportional to:
Draw a random sample of the i of size K from the posterior distribution (e.g., using WinBugs).
The posterior probability
is estimated by the proportion (out of K) that the posterior distribution of:
= 0.14 (100.14=1.4-fold shift)= 0.07 90% probability to call parallel equivalent
= 0.33 (100.33=2.15-fold shift)= 0.52 90% probability to call parallel equivalent
Simulation: FPL ModelBased on protein B data
Simulation: FPL modelSimilar to Protein B protein-chip data.
Concentrations (9-point curve + 0):0, 102(=100), 102.5625, 103.125, , 106.5(=3,200,000)Three replicates
xL = 3.5=log10(3162) & xU = 5=log10(100,000) = 0.2. = 0.02, 0.04, 0.11, and 0.21 (%CV = 5, 10, 25, 50)
For each Monte Carlo run, I computed:
5,000 Monte Carlo Replicates
ScenarioOriginalNewMax DiffA1B1C1D1A2B2C2D2124.54-124.54-10224.54.7-1032.4454.9454.7-10.15424.54.7-1.3120.1552.615.114.7-1=0.2062.254.754.7-1.50.30
Example data (CV=10%)Diff: 0 0 0.15 0.15 0.20 0.30
Diff: 0 0 0.15 0.15 0.20 0.30
ScenarioFrequentist statistical powerMax Diff%CV=5%CV=10%CV=25%CV=50101.001.001.000.50201.001.001.000.5030.15 (shape 1)1.000.990.28< 0.0140.15 (shape 2)1.000.880.360.145 = 0.200.100.02< 0.01< 0.0160.300.000.00< 0.01< 0.01
SummaryStraight-forward and simple test method to assess parallelism.
Yields the log-relative potency factor.
Easily extended.
ExtensionsInstead of f(, x), could use
f(, x) / x = instantaneous slope
f-1(, y) = estimated concentrations
Whats next?Head-to-head comparison with existing methods
Choosing test level and , possibly based on ROC curve? Harry Yang paper
Guidance for prior distribution of 1 and 2.
ReferencesCallahan, J. D. and Sajjadi, N. C. (2003), Testing the Null Hypothesis for a Specified Difference - The Right Way to Test for Parallelism, Bioprocessing Journal Mar/Apr 1-6.Gottschalk P.J. and Dunn J.R. (2005), Measuring Parallelism, Linearity, and Relative Potency in Bioassay and Immunoassay Data, Journal of Biopharmaceutical Statistics, 15: 3, 437 - 463.Hauck W.W., Capen R.C., Callahan J.D., Muth J.E.D., Hsu H., Lansky D., Sajjadi N.C., Seaver S.S., Singer R.R. and Weisman D. (2005), Assessing parallelism prior to determining relative potency, Journal of pharmaceutical science and technology, 59, 127-137. Jonkman J and Sidik K (2009), Equivalence Testing for Parallelism in the Four- Parameter Logistic Model, Journal of Biopharmaceutical Statistics, 19: 5, 818 - 837.
Thank you!Questions?
Parallel indicates a change in potency only.*CD8 T cells, HIV ELISPOT assay**USP = United States PharmacopeiaUSP sets standards for the quality, purity, strength, and consistency of productscritical to the public health.*Depending on the response to back-calculate, the fold-change between lots 1 and 2 can range from as little as 1.5-fold to as much as 5.5-fold.If we examine the difference in estimated concentration for lot 2 from the x-range, we get a delta.x = 0.07 -> 1.17-fold maximum shift in estimated conc. Frequentist model point-estimate maximum shift = 1.07-fold. *If we examine the difference in estimated concentration for lot 2 from the x-range, we get a delta.x = 0.35 -> 2.15-fold maximum shift in estimated conc. Frequentist model point-estimate maximum shift = 1.67-fold. *