The Death of Mortality: Epistemic Problems in Critical Care Research

There Is (No) Evidence For That: Epistemic Problems in Critical Care Medicine

Scott K. Aberegg, MD, mphSalt Lake City, Utahwww.medicalevidenceblog.comwww.statusiatrogenicus.blogspot.com@medevidenceblog

EpistemologyA branch of philosophy concerned with knowledge and its:Essence and OriginsMethodsValidityScope and Limitations

There is (or is not) evidence for is a glib statement and should be taken with a grain of salt

Intelligence versus RationalityBook SmartsThe algorithmic mindLearn and apply rules to solve well defined problemsMeasured precisely by IQ testsPredicts success at calculusCommon SenseEpistemic Rationality: How well do your beliefs map onto reality; sound judgment and decision makingInstrumental Rationality: Optimization of goal selection and fulfillmentNot measured (at all) by intelligence tests (need RQ test says Stanovich)

LUDIC FALLACY: The misuse of games to model real world problems

Dr. John the StatisticianThe coin has no memory of past flips. The current flip is independent of them. Thus, the probability of the 101st flip landing on heads is 50%Dr. John has book smartsTony the Options TraderThe probability that in my lifetime I will ever see a fair coin land on heads 100 times in a row is so low that I reject the premise of the problem that you have presented me. The coin is rigged.Tony has common sense

A fair coin is flipped and lands on heads 100 times in a row. What is the probability that it will land on heads on the 101st flip?

No Evidence Condition (There is No Evidence for)

Parachutes for Gravitational Challenge(You dont need a parachute to skydive. You need a parachute to skydive twice.)Mechanical VentilationAntibiotics for sepsis IVF for dehydration Insulin for DKAKnee Replacement

Knowledge without Formal Evidence

ARR high [NNT low]Visible & immediate effectsCausal Pathways ObviousType I diabetes DKAInsulin Resolution of DKATrials unethical No Equipoise High Prior Probability for HaImplicit Bayesian ApproachCategory 1 Therapies (Parachute Therapies)

Knowledge without formal evidence?Efficacy of parachute A versus parachute BDose of mechanical ventilationDuration/spectrum of antibiotics for sepsisDose of IVF for resuscitationAnytensive insulin therapy in the ICUArthroscopy debridement for osteoarthritis

No knowledge without Formal EvidenceCategory 2 TherapiesARR low(er) [NNT high(er)]Invisible & delayed effectsAssociations Prevalent, CPs ObscureICU Hyperglycemia ??? Insulin correction of hyperglycemiaTrials imperative Equipoise - Low(er) Prior Probability for Ha

There Is (Formal) Evidence for

Category II Therapies: The Trial As a Diagnostic Test of a HypothesisThere is Evidence for..True PositivesFalse PositivesType I Errors: A Specificity Issue The alpha bet a product of the unreflective mindJournal Club Biases Ioannidis - FLEXIBILITYFRAUD you can build an entire career (or product line) on itBayesian Interpretations

There is No Evidence for..True NegativesStochastic Dominance of the Null HypothesisBayesian InterpretationsFalse NegativesType II Errors: A Sensitivity IssueInadequate Study Power: SensitivityDelta Inflation

False Positive Evidence

False Positive Evidence: COS

False Positive Evidence

False Positives: The alpha betThreshold for statistical significance ( = 0.05) based on conventionConvention established by Fisher in 1925 in Statistical Methods for Research WorkersFisher was suggesting alpha 0.05 for the science of the 1920sStudy Population: Frog legs bathed in RingersStudy Outcome: Action PotentialsInvestigators on Earth studying the same topic: A handfulStudy Cost: PeanutsForward to the 21st Century:Study Population: Tens of thousands of patients with coronary diseaseStudy Outcome: Death or non-fatal MIInvestigators on Earth studying the problem: many handfulsStudy Cost: $250 Million

Statistical Methods for Research WorkersThe value for which P=0.05, or 1 in 20, is 1.96 or nearly 2; it is convenient to take this point as a limit in judging whether a deviation ought to be considered significant or not..Using this criterion we should be led to follow up a false indication only once in 22 trials, even if the statistics were the only guide available.

If one in twenty does not seem high enough odds, we may, if we prefer it, draw the line at one in fifty (the 2 per cent point), or one in a hundred (the 1 per cent point). Personally, the writer prefers to set a low standard of significance at the 5 per cent point, and ignore entirely all results which fail to reach this level. A scientific fact should be regarded as experimentally established only if a properly designed experiment rarely fails to give this level of significance. - Sir R.A. Fisher

What would Fisher say?TherapyStudyYearp-valueIntensive InsulinVan den Berghe2001