CAMARADES: Bringing evidence to translational medicine Stanford visit 8 Feb 2011

CAMARADES: Bringing evidence to translational medicine

Stanford visit

8 Feb 2011


Starting points

• As a community we thought we knew how to do clinical trials, but when we looked we found lots of confounding by bias

• As a community we think we know how to do animal experiment in stroke, but when we looked we found lots of confounding by bias

• As a community we think we know how to do in vitro biological research


Caveats

• We have looked in domains where we know there is translational failure – it might just be stroke researchers

• Most of our evidence is observational not experimental

• We are not experts in other fields, and experts may know best

• There are broader issues of how we know what we know – hierarchies of evidence


Not just stroke researchers

• EAE modelling of MS• Dopamine agonists in PD• Receptor Ligand Binding Assays


Systematic review in EAE


EAE modelling of MS


Systematic review in PD


Dopamine Agonists in PD

Blinded outcome assessment Composite quality


Receptor Ligand Binding Assays

McPartland et al 2007


Assertions

• Poor study quality is highly prevalent in laboratory research

• The effectiveness of laboratory research could be substantially improved if study quality were improved

• Improved study quality would reduce the risks of translational failure


Is it possible to study this systematically?

• Cannabinoid receptors• NOS transgenics in stroke• Pathophysiology of EAE


NOS transgenics in focal ischaemia

Grey bar represents the 95% confidence limits of the pooled effect size, bar width reflects the number of animals contributing to each analysis and vertical error bars represent 95% confidence intervals.

nexp nani

m

es 95% CI

eNOS -/- 16 308 -17.3 -22.3 to -12.4

iNOS +/-19 320

10.9 8.1 to 13.6

iNOS -/- 17.9 12.2 to 23.5

nNOS -/- 23 402 24.0 13.0 to 35.0


TH1 and TH17 cells in EAE

• Concordance is observed when a decreased number of circulating Th1 or Th17 cells is associated with decreased EAE severity, and when an increase in numbers of circulating Th1 or Th17 cells is associated with increased EAE severity. (a),(b) Concordance (green) and discordance (red) between change in the number of circulating (a) Th1 or (b) Th17 cells and change in the severity of observed EAE. Concordance is greater for the Th17-EAE pathway. (c) “Meta-blot” showing this relationship. Dots on the line represent concordance, those off the line show discordance. The size of each dot reflects the number of studies contributing to that point. Dot colour reflects the presence of factors which might bias the results of that group of experiments (Red: bias likely; Green: bias unlikely). The width of the arrow reflects the number of experiments contributing data to the analysis.


Contrasting strategies

• Random sampling “brute force” attack

• Focused study of particular area of biology


Random Sampling

• Decide what we want to look for• Sample at random (?PMID) against

inclusion and exclusion criteria• Categorise publications and experiments• Extract high level quality metrics• Extract data where possible (quantatative/

qualitative)• Seek evidence for influence of quality

metrics on outcome• Seek evidence for publication bias


..but

• no accepted taxonomy of laboratory research

• would require iterative approach• PMID selection will lead to

incomplete ascertainment


Focussed studyThe pathophysiology of NMDA

activation

• Start with the NMDA receptor as a “seed node”

• Systematically identify evidence for things which modulate NMDA-R activity

• Systematically identify evidence for things which are modulated by NMDA-R activity

• Weigh the evidence (taxonomy, quality, magnitude)

• Identify each “thing” as a secondary node and iterate until a pathway is created


…but

• Number of nodes will increase geometrically with each iteration

• Can the strength of evidence be condensed into only a few dimensions?


Opportunities

• Funding application in progress (7 years, statistician, post docs, multi-centre animal studies)

• IUPHAR database project• AHRQ “Mechanistic Frameworks”

project

Documents

CAMARADES: Bringing evidence to translational medicine Stanford visit 8 Feb 2011