View
223
Download
1
Category
Tags:
Preview:
Citation preview
Some comments on the 3 Some comments on the 3 paperspapers
Robert T. O’Neill Ph.DRobert T. O’Neill Ph.D
Comments on G. AndersonComments on G. Anderson
WHISH is a nice exampleWHISH is a nice example
Randomiztion (Zelen) but using different sources of Randomiztion (Zelen) but using different sources of data for outcomedata for outcome
Outcome data: self reported, adjudicated for medical Outcome data: self reported, adjudicated for medical records, Medicare claim (hybrid-ability to estimate SE records, Medicare claim (hybrid-ability to estimate SE and SPand SP
Impact of outcome misclassificationImpact of outcome misclassification
Event data not defined by protocol-you depend on the Event data not defined by protocol-you depend on the health care systemhealth care system
Claims data DO NOT provide standardized data – see Claims data DO NOT provide standardized data – see Mini-Sentinel and OMOPMini-Sentinel and OMOP
Comments on A J CookComments on A J Cook
Key component is randomization at Key component is randomization at patient or clinic level and use of electronic patient or clinic level and use of electronic health record for data capture (cluster health record for data capture (cluster randomization addresse different issues) randomization addresse different issues)
Missing data, informative censoring, switching, Missing data, informative censoring, switching, measuring duration of exposure (repeat Rx, gaps) , measuring duration of exposure (repeat Rx, gaps) , different answers depending upon definitiondifferent answers depending upon definition
Validation of outcomes makes the pragmatic trial less Validation of outcomes makes the pragmatic trial less simplesimple
Only some outcomes (endpoints) , populations, Only some outcomes (endpoints) , populations, questions are addressable before complexities of questions are addressable before complexities of interpretation overwhelminterpretation overwhelm
Comments on M GaffneyComments on M Gaffney
Precision and Eagle are not large simple Precision and Eagle are not large simple trials – they are large difficult trials trials – they are large difficult trials
Outcome adjudication, monitoring Outcome adjudication, monitoring strategiesstrategies
Non-inferiority poses significant challenges Non-inferiority poses significant challenges for pragmatic trials – generally no assay for pragmatic trials – generally no assay sensitivitysensitivity
Margin selection based upon evidence Margin selection based upon evidence vs. based upon close enough but not vs. based upon close enough but not sure if both are equally good or badsure if both are equally good or bad
Other comments on NI studiesOther comments on NI studies
Pre-specifying the margins – why and what is the Pre-specifying the margins – why and what is the difference in these two situationsdifference in these two situations
What treatment difference is detectable and credible What treatment difference is detectable and credible with the playoff of bias and huge sample size with the playoff of bias and huge sample size
When pre-specification is not possible because there is When pre-specification is not possible because there is no historical information, the width of the confidence no historical information, the width of the confidence interval makes sense – but two conclusions – both interval makes sense – but two conclusions – both treatments the same and comparably effective vs. both treatments the same and comparably effective vs. both the same but both ineffectivethe same but both ineffective
What endpoints are eligible : Hard endpoints (y), patient What endpoints are eligible : Hard endpoints (y), patient symptoms(n) symptoms(n)
Other commentsOther comments
Are NI designs appropriate for claims data of EHR Are NI designs appropriate for claims data of EHR without independent all case adjudication – without independent all case adjudication – implications for poor sensitivity and specificity to implications for poor sensitivity and specificity to drive estimate to null – what does a null result drive estimate to null – what does a null result meanmean
Experience suggests that Exposure (drugs) has Experience suggests that Exposure (drugs) has better accuracy than diagnoses or procedure in better accuracy than diagnoses or procedure in claims data bases(outcomes)claims data bases(outcomes)
Duration of exposure dependent upon Duration of exposure dependent upon algorithms for repeat prescriptions – different algorithms for repeat prescriptions – different results depending upon definitions of gaps results depending upon definitions of gaps between repeated RXbetween repeated RX
Can randomization overcome lack Can randomization overcome lack of blinding and personal choices of blinding and personal choices
after randomizationafter randomization
Use of observational methods of Use of observational methods of accounting for unmeasured accounting for unmeasured confounding of assigned treatment confounding of assigned treatment and time to event outcomes subject to and time to event outcomes subject to censoringcensoring
Directed Acylic Graphs to explore Directed Acylic Graphs to explore the confounding-censoring the confounding-censoring problem - diagnosticsproblem - diagnostics
Instrumental variablesInstrumental variables
Lessons learned from Mini-Sentinel and the Lessons learned from Mini-Sentinel and the Observational Medical Outcomes Partnership Observational Medical Outcomes Partnership
(OMOP)(OMOP)
Distributed data modelsDistributed data models
Common data modelsCommon data models
Limits of detectability of effect sizes of two or more Limits of detectability of effect sizes of two or more competing agents – calibration, interpretation of p-competing agents – calibration, interpretation of p-values for non randomized studiesvalues for non randomized studies
Not all outcomes and exposures can be dealt with Not all outcomes and exposures can be dealt with in similar mannerin similar manner
Know the limitations of your data base – is this Know the limitations of your data base – is this possible in advance of conducting the study – part possible in advance of conducting the study – part of the intensive study planning, protocol and of the intensive study planning, protocol and prospective analysis planprospective analysis plan
An example of Medicare data use but An example of Medicare data use but not a RCTnot a RCT
Some other views and opinions on CER using the learning health care system
Lessons Learned from OMOP Lessons Learned from OMOP and Mini-Sentinel About and Mini-Sentinel About
observational studies using observational studies using health care claims data or EHR – health care claims data or EHR –
but no randomizationbut no randomization
Lessons about the limitations of Lessons about the limitations of the data bases, outcome the data bases, outcome capturing, ascertainment , missing capturing, ascertainment , missing data (confounders) are relevant to data (confounders) are relevant to the RCT use of the same data the RCT use of the same data sourcesource
Lessons about data models, and Lessons about data models, and challenges for data (outcome challenges for data (outcome standardization)standardization)
http://www.mini-sentinel.org/http://omop.org/
The Observational Medical Outcomes Partnership – many findings
Some ideas on what to evaluate about a given data source before committing to conducting a study – focus on observational studies – but also relevant to pragmatic RCTs
How do these presentations relate How do these presentations relate to pragmatic trials within a health to pragmatic trials within a health
care systemcare system
Two or more competing therapies on a formulary – Two or more competing therapies on a formulary – never compared with each othernever compared with each other
Randomize patients under equipoise principle – do you Randomize patients under equipoise principle – do you need patient consent , physician consent if health plan need patient consent , physician consent if health plan and no data to think ‘I or We don’t know but want to and no data to think ‘I or We don’t know but want to find out ‘find out ‘
Collect electronic medical record data, including Collect electronic medical record data, including exposures and outcomes – and decide if any additional exposures and outcomes – and decide if any additional adjudication is neededadjudication is needed
Analyze according to best practices – but with some Analyze according to best practices – but with some prospective SAPs – causal inference strategies ?prospective SAPs – causal inference strategies ?
Recommended