View
377
Download
1
Embed Size (px)
Citation preview
The Continuous Update Project: Novel approach to reviewing mechanistic evidence on diet, nutrition, physical activity and cancer 21 October 2015
Martin WisemanWorld Cancer Research Fund International & University of Southampton
The Panel emphasises the importance of not smoking and of avoiding exposure to tobacco smoke
Journal citations WCRF/AICR Reports
1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 20110
50
100
150
200
250
300
350
400
450
500
Doll & Peto 19811997 EXPERT REPORT 2007 EXPERT REPORT
Num
ber o
f cita
tions
KEY FEATURES OF PROCESS
• New method• Systematic reviews• Review of evidence separate from
judgement • Panel of international experts• Predetermined criteria for judgements• Flexibility
Systematic reviewsAnalysis
• Strength• Consistency• Specificity• Timing• Dose Response• Plausibility• Coherence• Experiment• Analogy
Inferring causality
Bradford Hill
• Strength• Consistency• Specificity• Timing• Dose Response• Plausibility and coherence• Coherence• Experiment• Analogy
Inferring causality
Bradford Hill
Systematic reviews
• Expert international Task Force for method• Nine centres - USA, UK, NL, Italy• SLR centre coordinator• Test of reproducibility• Standardised search, analysis and display • Epidemiology and mechanisms• Quality assessment• Peer review - protocol, report• Defined expertise required
– Nutrition, epidemiology, systematic review, cancer biology, statistics
GRADING CRITERIAPredefined requirements for:
–Number and types of studies–Quality of exposure and outcome assessment–Heterogeneity within and between study
types–Exclusion of chance, bias or confounding–Biological gradient–Evidence of mechanisms–Size of effect
• To help identify causal links• Information from mechanistic studies-narrative reviews • Evidence on mechanisms-predefined requirement for
grading criteria• Considerations:
reviews not systematiccould select a mechanism to explain epidemiological associations
2007 Second Expert Report and Continuous Update Project
29 MARCH 2012 | VOL 483 | NATURE | 531
• Reproducibility• Relevance of model• Relevance of exposure• Relevance of dose• Route of administration• Publication bias
• < 10% highly promising basic science discoveries enter clinical use
• Evidence distorted – Weak internal validity (randomisation, allocation concealment,
blinding, drop outs, co-morbidities) – Publication bias (98% of animal studies ‘significant’)
Having confidence in the evidence
van Luijk J et al (2014) Systematic Reviews of Animal Studies; Missing Link in Translational Research?. PLoS ONE 9(3): e89981.
Internal validity of animal intervention studies
available in principle(e.g. thesis, obscure journal)
easily available(Medline-indexed)
activelydisseminated(e.g. reprint fromdrug company)
unavailable(unpublished)
Publication bias
Meta-analysis of the association between TP53 status and the risk of death at 2 years
Kyzas P A et al. JNCI J Natl Cancer Inst 2005;97:1043-1055
• To develop a method for conducting systematic reviews of mechanistic evidence linking food, nutrition and physical activity exposures to cancer– Multidisciplinary team (informatics, statistics,
epidemiology, systematic reviews, cancer biology, pathology, nutrition, cancer site of interest)
– Search terms/inclusion-exclusion criteria– How to manage vast number of papers– What information to be extracted– How to analyse/display results– Identify criteria for grading the evidence
WCRF mechanisms Project at Bristol
Importance• Extensive mechanistic data from animals & cell lines linking diet & cancer• Systematic reviews
•Allow objective appraisal of evidence •Reduce false-positive & false-negative results•Identify sources of bias, improving study quality•Rigorous methods for conducting & reporting systematic reviews of mechanistic studies are lacking
• This project should increase the value of mechanistic data•Enable rigorous reviews •Increased precision of estimated effects •Identify gaps in the research evidence•Reduce selective citation of mechanistic evidence•Inform generalisability to humans (e.g. heterogeneity across species & models)
• A potential tool in the translation of basic sciences into policy & practice
Challenges• Developing a one size fits all template
although we expect that when the template is developed research groups will choose to search for mechanistic targets relevant to their own question• Finding the relevant studies
this relies on having a good search strategy and validating this• Determining study quality
some quality criteria could be adopted from epidemiology• Determining the strength of evidence for different study types
by discussions within the team, tapping into to our multidisciplinary team• Determining the relevance to humans
consensus following examination of the studies and discusses within the team• Publication bias
We will use recognised methods to quantify the extent to which this is likely to have occurred• Collating the evidence
Key issues• Mechanisms discovery
_ Validation (not missing important studies)_ Inter-relationships between mechanisms _ Stand-alone automation
• Risk of bias / internal validity– Animal and in-vitro studies
• Relevance / external validity – What relevance criteria form part of eligibility?– How to incorporate into evidence synthesis?
• Evidence synthesis– Publication bias, consistency (heterogeneity), precision in absence
of meta-analyses• Categorising overall conclusions
- magnitudes of effect and causality
Summary of the process
Research question
• Identify
• Appraise individual studies
• Integrate body of evidence
• Risk of bias• Relevance
• Mechanism discovery (unbiased)
• Specific mechanisms (targeted)
• Within an evidence stream (human, animal, in vitro)
• Across evidence streams
• Confidence conclusion- High- Moderate- Low
Eligibility criteriaRelevance
Step 9: Synthesis of supporting evidence from in vitro and xenograft models underpinning biological plausibility
Step 8: Integrate human and animal studies to develop an evidence based conclusion
Step 7: Assess strength of overall body of evidence for human and animal studies separately (based on study design, risk of bias, relevance, imprecision, inconsistency, publication bias,
magnitude of effect, dose-response & confounding)
Step 6: Synthesis of data from individual studies
Step 5: Assess the quality of individual studies (risk of bias)
Step 4: Extract data
Step 3: Apply inclusion/exclusion criteria, including an assessment of relevance
Step 2: Search for studies
Step 1: Specify research objectives
Step 1
• Identifying potential mechanisms by which an exposure causes an outcome
summarise the literature on all potential mechanisms linking a modifiable exposure to cancer outcomes (may be bypassed if the objective is to
review the evidence underlying a pre-specified mechanism )
Step 1, Stage 1
• Defining the research question– specify the modifiable exposure and the cancer outcomes of interest. – develop a comprehensive list of possible mechanisms or intermediate
phenotypes
A diverse range of studies provide evidence on mechanisms of explaining an exposure – cancer outcome relationship, including cell-line, animal and human
Step 1, Stage 2
Searching for studies
-Develop of list of search terms-Carry out searches in Medline, Embase and Biosis
Automated mechanisms discovery
Tom Gaunt, University of Bristol
Exposure terms
Linking mechanisms
Outcome terms
Tom Gaunt, University of Bristol
Relevance (external validity)
• Mechanistic studies inform plausibility: Bradford-Hill
• Animal studies vs human studies– Population: homogeneity (genetic, clinical, environment)– Exposure: biological window– Outcome: induced or transplanted cancers
• Need clarity over role of relevance in: – Setting eligibility criteria (only assess bias if relevant?)– Integrating the body of evidence (hierarchy?)
Heterogeneity in animal studies exploring effect of statins
Evidence integration
• Within an evidence stream (human, animal, in vitro)– Assess confidence about data quality– Summarise magnitude of effects where possible
• Across evidence streams - causal conclusions• Integrate the highest level of evidence from
each of the evidence streams (human RCT, observational, animal, in-vitro)
Integrate evidence to develop causal conclusions
• Integrate the highest level of evidence from each evidence stream (human, animal, in-vitro)
• Consider evidence from human data with animal evidence, then in-vitro data– E.g. If human level of evidence high – conclusion based on
human data only– If human level of evidence moderate / low – conclusion based
on animal evidence– Upgrade if in vitro data provide strong evidence of biological
plausibility (downgrade if weak?)• Develop a categorisation schema (high,
moderate, low evidence of causality)
Level of evidence
in human studies
High Convincing
ModerateSuggestive Probable
Low
No conclusion Suggestive Probable
Low Moderate High
Level of evidence in animal studies
Integrating evidence
Discussion points
• Eligibility criteria (retrieval stage) vs stratification– design / conduct / reverse causation / relevance considerations?
• Risk of bias for in vitro studies• Discussion over role of relevance in:
- Setting eligibility criteria (only assess RoB if relevant)- Integrating the body of evidence
• Signalling questions for relevance • Evidence integration • Publication bias • Causal conclusions: categorisation and relative weights
– animal studies upgrade confidence if human evidence moderate or low?– in-vitro studies: biological plausibility to support recommendations
• Testing of the tool by others – utility, generalisability and replicability
• Increasing the sensitivity and specificity of mechanism discovery algorithm
• Incorporation of methodological developments– e.g. risk of bias in animal & in-vitro studiesunderstanding of relevance
• Review of other templates: – Office of Health Assessment & Translation, US National
Toxicology Program
Future work
Research question (PECO)
• Identify
• Appraise individual studies
• Integrate body of evidence
• Risk of bias• Relevance
• Mechanism discovery (unbiased)
• Specific mechanisms (targeted)
• Within an evidence stream (human, animal, in vitro)
• Across evidence streams
• Confidence conclusion- High- Moderate- Low
Eligibility criteria relevance
Integrating causal
conclusions about
mechanisms into pathways
Summary of the process
The TeamUniversity of Bristol: PI- Dr Sarah Lewis –Genetic epidemiology/systematic reviews of genetic studies Co-PI- Prof Richard Martin –Epidemiology/systematic reviews Dr Mona Jeffreys- Cancer Epidemiology/systematic reviews Dr Mike Gardner – Animal biology/systematic reviews Prof Jeff Holly- Molecular biology – IGF and cancer Dr Tom Gaunt – Genetic epidemiology/bioinformatics Prof Jonathan Sterne- Meta-analysis and systematic review methodology Professor Julian Higgins – Meta-analysis and systematic review methodology Prof George Davey Smith – Epidemiology Prof Christos Paraskeva –Molecular biology Prof Steve Thomas –Epidemiology of head and neck cancer Dr Pauline Emmett - Nutritional epidemiology Dr Kate Northstone – Nutritional Epidemiology Cath Borwick – Librarian/ Search strategies
University of Cambridge: WCRF InternationalDr Suzanne Turner- Animal models Prof Martin Wiseman
Dr Pierre Hainut (advisor to WCRF Int)
Dr Panagiota Mitrou
Dr Rachel Thompson
IARC:Dr Sabina Rinaldi- Hormones and cancer