Upload
others
View
2
Download
0
Embed Size (px)
Citation preview
©2016, Genentech
Recommendations to Challenges for
Appropriate Selection and Characterization
of Calibrator Material
Kyra J. Cowan, Ph.D.Biomarker Theme, AAPS NBC 2016:
Characterization of Protein Biomarker Reference Material
©2016, Genentech
Background
Consensus to date
Questions from Scientists
Biomarker Assay Protein calibrators Subteam
Scope of work for cross-industry team: BAPS
Approach to date
Recommendations and Solutions for Challenges
Across protein biomarker assay calibrators
Fit-for-purpose (FFP) approach
Commutability
Summary
Acknowledgements
Outline 2
©2016, Genentech
Publications Cross-Industry Meetings/Conferences 2013 FDA Draft Guidance
Written Consensus papers:
Lee et al. 2005, 2006, 2009
Biomarker assay validation
O’Hara et al. 2012
Critical Reagents characterization
King et al. 2014
GBC Harmonization white paper on
critical reagents for LBAs
Bower et al. 2014
Commentary paper on
reference standards and
reagents in BMV
Building Industry Consensus:
AAPS NBC 2014, 2015, 2016
- Biomarker themes, topics
Crystal City V 2013
FDA draft guidance discussion
Crystal City VI 2015
Biomarker assay discussion
Lowes, Ackermann 2016
2nd Upcoming publication
WRIB 2014, 2015, 2016
Biomarker assay discussions
Gap Identified: Best practices for the selection and characterization
of calibrator material.3
©2016, Genentech
Key Difference: PK vs. Biomarker Assay Calibrators
Calibrator Analyte
PK Assay example
•Unlikely that recombinant material will be the same as the endogenous analyte for biomarker assays.
•Recombinant material for protein biomarker assays often not well characterized, or readily available
Recombinant Calibrator Analyte
Biomarker Assay example
Glyc
OR
Glyc Glyc
Monomer No PTM
Glyc
Difficult to express Misfolded, aggregates
4
©2016, Genentech
What characteristics are we looking for in a recombinant material for our assays?
Challenge is to match recombinant material with endogenous
Potential post-translation modifications, depending on disease-state, matrix, treatment regimen, genetics, environment…
Are we measuring what we think we’re measuring?
Specificity vs. Interference
Is the reagent we’re using reliable as a calibrator for our assay?
Parallelism – must be assessed early on
Lot-to-lot variability, stability
Commutability
What is the “best” calibrator material for protein
biomarker assays? 5
©2016, Genentech
BAPS: Biomarker Assay Protein calibrators Subteam
LBABFG Biomarker Discussion Group Need recommendations on calibrator selection, characterization
Need for an AAPS subteam to establish consensus to address challenges
Scope of Subteam Recommend best practices for bioanalytical scientists
FFP approach for selection, characterization, and assay acceptance.
Based on experience, challenges faced, important case studies, and relevant literature.
Define the important characteristics that a scientist should look for in identifying a reliable recombinant material.
Evaluate the choices for different types of calibrator protein depending on the challenges faced during assay development (from full-length proteins to truncated forms or peptides).
Introduce and incorporate appropriate commutability approaches used in clinical chemistry and approaches for available NIST standards into industry-wide recommendations.
6
©2016, Genentech
Kyra Cowan (TL), Genentech
Lakshmi Amaravadi, Sanofi
Mark Cameron, Beckman-Coulter
Damien Fink, Janssen
Darshana Jani, Pfizer
Medha Kamat, Eurofins
Lindsay King, Pfizer
RJ Neely, Bristol-Myers Squibb
Yan Ni, Bristol Myers-Squibb
Paul Rhyne, Q2 Solutions
Renee Riffon, CiToxLAB North America
Yuda Zhu, Genentech
BAPS 7
©2016, Genentech
List of challenges each team member has faced
Develop recommendations for:
Challenges common to all protein biomarkers
Challenges unique to commonly tested, available proteins
Challenges unique to novel targets/uncommon proteins
FFP Table
Recommendations across proteins for characterization
Risks and caveats
Commutability approach for bioanalytical scientists
Apply to recombinant material for biomarker assays
Approach 8
©2016, Genentech
Know your endogenous protein!• Amino acid sequence; Tertiary, quaternary structure?
• Is the protein monomeric/dimeric/oligomeric in nature?
• Does the protein of interest undergo cleavage or alternative splicing?
• How is that represented in the “total” protein?
• Does it have a binding partner, or bind with drug?
• Are “misfolded” proteins reactive in the assay? Are they bioactive?
• Endogenous and exogenous match, but do the modifications or post-translational modifications match? Glycosylation? Isoforms? His/Flag/etc. tagged.
How much must the exogenous protein have similarity to the endogenous protein?
Further Understanding:• Look Into Protein Database Resources, published biology
• Take home message: Calibrator material should be as close to the endogenous form of the protein as possible, to provide confidence in true quantitation
• Also, should know your assay and what it can detect.
Case Study: State of endogenous “heterodimer” versus calibrator material
Solutions/Recommendations Across Proteins 9
©2016, Genentech
BIOLOGY:
IL-23 is a heterodimer protein consisting of a p19 and p40 subunit.
Shares p40 subunit with IL-12 cytokine
KIT:
Capture Ab is IL-23-specific, does not bind IL-12.
Calibrator material is a p19 subunit fused to the p40 subunit.
Company used calibrator material to generate the capture Ab.
Results from samples:
Kit gave higher than expected levels of IL-23 (based on previous publications and in-house data).
Case Study: IL-23 Luminex Commercial Kit
RJ Neely, BMS
10
©2016, Genentech
Troubleshooting: Compared Luminex kit to a later run, in-house BMS IL-23 assay
- Substantially different levels (<BLQ)
- Established 3rd party vendor assay agreed with the in-house, BMS assay.
Likely Issues:Luminex kit used a fused p19/p40 heterodimer as calibrator material - used to generate capture Ab
In-house and third party vendors used ex vivo formed
p19/p40 heterodimers in assay
- Fused p19/p40 heterodimer most likely different from endogenous
- Ex vivo material better mimics endogenous
Case Study: IL-23 Luminex Commercial Kit
Jennifer Postelnek, RJ Neely, BMS
Lessons Learned
•Calibrator material should be as close to the
endogenous form of the protein as possible
•Make sure the analyte biology supports the results generated.
Sample
LuminexOOR =
out of range
BMS
pg/mL
3rd
Party
Vendor
pg/mL pg/mL
1 283 <9.6 0.047
2 44.5 <9.6 0.111
3 7004 13.4 0.084
4 OOR < <9.6 0.033
5 102 <9.6 0.035
6 396 <9.6 0.065
7 60 <9.6 0.116
8 1529 <9.6 0.011
9 715 <9.6 0.247
10 OOR < <9.6 0.071
11
©2016, Genentech
• Vendors provide:
• Concentration of the protein
• Purity (such as a percentage, based on a silver stain or HPLC analysis)
• Source or origin (E. coli derived, for example)
• An accession number (providing basic information about the protein)
• Formulation (phosphate buffered saline, for example)
• Storage and stability information.
What Is Standard Practice for Protein Manufacturers?
• Vendors may, or may not, provide the method(s) used to assign a concentration to their protein product.
• These types of assays are typically plate-based colorimetric assays or absorption spectroscopy.
• As part of pre-purchase due diligence, information about how the vendor has assigned a concentration value to the protein produce should be gathered.
• Recommendation:
• Contact technical support key to understand this information, including why the vendor selected a particular method over another.
12
©2016, Genentech
FFP Table for calibrator material of protein biomarker assays• To define recommended extent of characterization depending on the intended use
of the biomarker assay data.
• To provide a consistent approach to aid in the selection and characterization of a calibrator
• To define what characterization criteria are "must" vs ”good to have” from vendors
• To provide a framework for how much characterization is "enough”
FFP Approach for Characterization Team recommendations for selection and characterization of biomarker assay protein calibrator material• Need to define “well-characterized”, for in-house and commercially obtained:
• What is the characterization of the endogenous, and what are acceptance/rejection criteria?
• What, if any, additional characterization should be done in-house if purchased?
• Risks associated with varying degrees of characterization.
Table recommends the bare minimum for what we recommend depending on the purpose of the data, regardless of whether the assay/material is purchased from a vendor or generated in-house.
• Caveat for table: highly dependent on whether the endogenous protein is well characterized.
13
©2016, Genentech
FFP Table: Identity
Recommendations for Biomarker Calibrator Material Characterization
Assay Type/Inten
ded Purpose
Relative quantitative assays used in exploratory
setting*
All quantitative assays used to justify dose, toshow safety or efficacy, and/or to support drug
registration**
Identity CoA from Producer
(external or internal)CoA plus Additional
Internal Characterization1
1 Including, but not limited to: Sequencing confirmation,
Peptide mapping, Mass Spec, or comparison to other family
members*“Exploratory” on next slides**“Treatment Decision-Making” on next slides
14
©2016, Genentech
FFP Table: Quantity and Purity
Recommendations for Biomarker Calibrator Material Characterization
Assay Type/Intended
PurposeExploratory Treatment Decision-Making
Quantity/Concentration
Carrier-free calibrators - A2802, BCA, Bradford, Performance in
Assay3, or CoA from Producer4
Carrier-protein containing calibrator - Performance in assay3, or CoA from Producer4
Purity, impurities and
contaminantsCoA from Producer4 Electrophoresis, or SEC, or Mass
Spec
2 Ensure proper extinction coefficient used3 Given there is one available; performance in orthogonal method optional4 Consider internal characterization (eg. SDS-PAGE/Electrophoresis) for new
vendors and for incomplete characterizations.
15
©2016, Genentech
FFP Table: Physicochemical, Expression
Recommendations for Biomarker Calibrator Material CharacterizationAssay
Type/Intended Purpose
Exploratory Treatment Decision-Making
Physicochemical properties5 CoA from Producer4 Electrophoresis, SEC, Mass Spec,
Maldi-TOF, DLS, and/or DSCExpression
SystemsMammalian or other6 Preferably mammalian expression
in appropriate cell types6
4 Consider internal characterization (eg. SDS-PAGE/Electrophoresis) for new
vendors and for incomplete characterizations.5 Molecular weight confirmation, post-translational modification, oligomerization,
folding, etc.6 Alternative expression systems may be deemed suitable depending on the
specific biomarker.
16
©2016, Genentech
FFP Table: Activity, Stability
Recommendations for Biomarker Calibrator Material Characterization
Assay Type/Intende
d PurposeExploratory Treatment Decision-Making
Biological activity
/Tertiary structure
Confirm activity7 or binding to Ab reagents
Confirm activity in an existing method7
Stability of calibrator
Short term (i.e. Freeze/Thaw, 4°C, Room Temperature, etc.)
Short & Long Term (multi-year -70°C)
7 If available; performance in orthogonal method if available.
17
©2016, Genentech
FFP Table: Parallelism, Lot-Lot Var.Recommendations for Biomarker Calibrator Material
CharacterizationAssay
Type/Intended Purpose
Exploratory Treatment Decision-Making
Parallelism Calibrator and endogenous
should pass with fewer samples tested (n > 3)
Calibrator should match endogenous analyte using more
samples (n > 10) 8, disease-
state samples required
Lot-to-lot variability
Determine comparability of new lot to previous lot. Normalize (if
needed) new material to old material or obtain a replacement
lot.
Test multiple lots from same producer to determine
variability. Consider value assignment.
8 If 10 samples are not available, scientists should consider the risks if fewer samples are tested. This number also applies to pre-clinical samples.
18
©2016, Genentech
Risk: Calibrator Mismatch to Endogenous
Risks/Impact Associated With Unacceptable Biomarker Calibrator Material
Assay Type/Intende
d PurposeExploratory Treatment Decision-Making
Impact
Calibrator performance significantly different than
endogenous biomarker, impacting data interpretation
and decisions based on data,
plus timelines, resources.
Data predicts incorrect dose. Patient safety impacted.
Incorrect efficacy
assessment. Therapeutic
approval affected.
19
©2016, Genentech
Additional Notes
Goal of FFP Table:
• Set threshold recommendations
• List risks associated with missing characterization of calibrator material, depending on intended use of biomarker assay data.
Must understand the biology of the analyte:
• To help ensure recombinant material will match endogenous analyte
Parallelism should always be performed early on• During method development
• Bridge surrogate matrix/recombinant protein vs. matrix with endogenous protein.
• Parallelism means calibrator and endogenous analyte behave in the same manner in the assay.
20
©2016, Genentech
BAPS: Biomarker Assay Protein calibrators Subteam
LBABFG Biomarker Discussion Group Need recommendations on calibrator selection, characterization
Need for an AAPS subteam to establish consensus to address challenges
Scope of Subteam Recommend best practices for bioanalytical scientists
FFP approach for selection, characterization, and assay acceptance.
Based on experience, challenges faced, important case studies, and relevant literature.
Define the important characteristics that a scientist should look for in identifying a reliable recombinant material.
Evaluate the choices for different types of calibrator protein depending on the challenges faced during assay development (from full-length proteins to truncated forms or peptides).
Introduce and incorporate appropriate commutability approaches used in clinical chemistry and approaches for available NIST standards into industry-wide recommendations.
21
©2016, Genentech
Commutability Approach
Definition: Any mathematical relationship between the results of different assays for a biomarker calibrator and for representative samples.
• Commutability is a property of the calibrator with respect to a defined set of assays and samples.
• Used in Clinical Chemistry settings to compare calibrators with samples.
• Any assay calibrator material from different sources, or even a different lot from same source, may not be commutable.
Importance of commutability:
• To manage a biomarker assay over years to support clinical trials
• To support assay on different platforms as needed
22
©2016, Genentech
Commutability MethodologyWe wanted to define:
• The appropriate mathematical relationships
• How to assess these relationships
Deming Residuals1. Plotted sample data with calibrator:
●
●●
●
●
●
●
●
●●
●
●●
●●
●
●●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
● ● ●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●●
●
●
●
●●●
●
●
●
0
5
10
15
0 5 10 15
Assay.1
Assay.2 Type
●
●
Reference
Sample
6 8 10 12 14
46
810
12
14
Deming Regression Fit
Method1
Me
tho
d2
Pearson's r = 0.845
The 0.95−confidence bounds are calculated with the analytical method.
Deming RegressionFit (n=100)
−0.68 + 1.05 * Method1identity
2. Fit Deming regression, and
compute Deming residuals:
23
©2016, Genentech
Commutability Methodology
● ●
●●
●●
●●
●●
●● ●
●●
●●
●
●●
●
●
●
●
●
●
●●
●●
●●
●●
●●
●●
● ●●●
● ●●●
●
●●● ●
● ●
●●
●●
●●
●
●
● ●●●
●
●
●
●●
●
●●●
●
●●
●●●
●●
●
●●●
●●
● ●●● ● ●
●
●●
●
●●
●
●●●
●
0.0
0.2
0.4
0.6
0.0 0.5 1.0 1.5 2.0
Residuals
de
nsity Data
●
●
Ref. Material
Samples
Outlier Screening via Deming residuals
DEMING RESIDUALS:
Using residuals, we can
determine a 99%
quantile based on
samples and examine
the location of the
calibrator material’s
Deming residuals.
Calibrator, at different
levels, fits within the
range of samples.
Calibrator is
commutable.
3. Estimate 99% cutpoint Deming residuals:
24
©2016, Genentech
Summary of RecommendationsRecommendations across proteins:
Know your endogenous protein
– Binding partners, PTM, multi-merization, etc.
FFP Table
– Baseline selection and characterization
– Dependent on intended use of data
– Conduct parallelism early
Commutability
– Suggested statistical approach for long-term
support of clinical assays: Deming residuals
25
©2016, Genentech
Acknowledgements
Cross-Industry:
Lakshmi Amaravadi
Mark Cameron
Damien Fink
Darshana Jani
Medha Kamat
Lindsay King
RJ Neely
Yan Ni
Paul Rhyne
Renee Riffon
Yuda Zhu
AAPS NBC Session Moderators
Yan Ni
Steve Piccoli
AAPS NBC Session Speakers
Dave Bunk
Hubert Vesper
Genentech BioAnalytical
Sciences:
Eric Wakshull
An Song
Patty Siguenza
26