Evidence Based EMS

Evidence Based EMS2013 Advanced EMT Course

Definitions

Definitions: Research

• Research: noun \ri-ˈsərch, ˈrē-ˌ\• 1 : careful or diligent search• 2 : studious inquiry or examination;

• especially : investigation or experimentation aimed at the discovery and interpretation of facts, revision of accepted theories or laws in the light of new facts, or practical application of such new or revised theories or laws

• 3 : the collecting of information about a particular subject

Definitions: Science

• Science: noun \ˈsī-ən(t)s\• 1 : the state of knowing : knowledge as distinguished from ignorance or

misunderstanding• 2 a department of systematized knowledge as an object of

study <the science of theology>• 3 knowledge or a system of knowledge covering general truths or the

operation of general laws especially as obtained and tested through scientific method

• b such knowledge or such a system of knowledge concerned with the physical world and its phenomena - i.e. natural science

• 4 : a system or method reconciling practical ends with scientific laws• “cooking is both a science and an art”

“Science is neither a philosophy nor a belief system. It is a combination of mental operations that has become increasingly the habit of educated peoples, a culture of illuminations hit upon by a fortunate turn of history that yielded the most effective way of learning about the real world ever conceived.”

Edward O. Wilson

Consilience: The Unity of Knowledge

So then, what is Science?

Definition: Evidence

• Evidence: noun \ˈe-və-dən(t)s, -və-ˌden(t)s\• 1 : an outward sign• 2 : something that furnishes proof :

• specifically :something legally submitted to a tribunal to ascertain the truth of a matter

• 3: to be seen

Definitions: Evidence Based Medicine (EBM)• “ “…The conscientious, explicit and judicious use of current best

evidence in making decisions about the care of individual patients.”• Sackett DL, Rosenberg WM, Gray JA, Haynes RB, Richardson WS (January 1996).

"Evidence based medicine: what it is and what it isn't". BMJ 312 (7023): 71–2.PMC 2349778. PMID 8555924.

• “…The use of mathematical estimates of the risk of benefit and harm, derived from high-quality research on population samples, to inform clinical decision-making in the diagnosis, investigation or management of individual patients.“

• Greenhalgh, Trisha. How To Read a Paper: The Basics of Evidence-Based Medicine. Wiley-Blackwell, fourth edition, 2010, p. 1.

Background: Why EBM?

• Prior to the 1970’s, it was assumed that basic medical education, clinical updates, and interaction with other providers was adequate to guide good decision making.

• Research in the 1970’s found HUGE variance on how providers would treat similar clinical conditions.

• Most patient care was based on tradition and/or experience rather than evidence and research

Background: Why EBM?

• Most patient care was based on tradition and/or experience rather than evidence and research

• “That’s the way we always did it, why change”• “Practicing at the level of graduation”

• Institute of Medicine (IOM) Report-1985 Estimate: Only 15% of medical practices based upon solid evidence.

• Probably much much less for EMS

BENCH TEST vs. BEDSIDE TEST

EBM: What it is…

• A means to take individual biases and common cognitive errors out of the equation

• A means to keep up to date and evolve as the art and science of medicine evolves.

• A means to apply a rigorous review to what we previously just accepted as truth.

EBM: What it isnt

• “Cook-book” medicine• Tends to require more thinking to apply it correctly

• “Cost-cutting” medicine• Sometimes the cost of care will increase, though the effectiveness will also

(More bang for the buck)

• A replacement for clinical expertise• A replacement for common sense• Absolute rules and regulations that leave no room for deviation

KEY POINT:

• External clinical evidence can inform, but can never replace, individual clinical expertise…..

EBM:

• The practice of evidence based medicine means integrating individual clinical expertise with the best available external clinical evidence from systematic research.

• “Individual Clinical Expertise”: What does this mean?• The proficiency and judgment that individual clinicians acquire through

clinical experience and clinical practice• Typically acquired over years (decades?) of actual practice….

• “Best Available Clinical Evidence”: What does this mean?• clinically relevant research, often from the basic sciences of medicine, but

especially from patient centered clinical

EBM: Focus

• Focuses on:• Accuracy of diagnostic tests including the clinical examination.

• Example: 80-70-60 Rule of blood pressure, 12 rhythm interpretation• The power of prognostic markers,

• Example: Glascow Coma Scale, Trauma levels, Mechanism of Injury /Co-Morbid factors• The efficacy and safety of therapeutic, rehabilitative, and preventive

regimens.• MAST Pants• Intubation

What does this mean to me?

• Good Medics/EMTs use both individual clinical expertise and the best available external evidence, and neither alone is enough.

• Without clinical expertise, protocols risks becoming tyrannised by evidence, becoming Cook Book medicine.

• Even excellent external evidence may be inapplicable to or inappropriate for an individual patient.

• Without current best evidence, practice risks becoming rapidly out of date, to the detriment of patients

Art or Science?

• Some health practices are more about art than science.

• Chiropractic, for example, has little science and a lot of art.

• Neurology today has a great deal of science and little art.

• Have you ever met a neurosurgeon with personality?

Art or Science?

• Neurology today has a great deal of science and little art.

Is EMS Art or Science?

• The knowledge of EMS is science.• The way it is applied is art.• Excellent EMTs know the science

of EMS and use the art of EMS to apply the science.

“…Medicine is an Art, masquerading as a science”

Robert. S. Cole, NR-P, OCD, Resident Weirdo

Art or Science?

• When you have a life threatening illness or injury, would you seek out a health care provider whose practice was more art or more science?

Art or Science?

• EMS is the safety net of society.• We are often the first to provide

care to the injured and the infirm.• Our practices must be based on

science.

But we must apply them with art!

Understanding the Research……

Understanding the Research

• EBM is based n research…

• To appropriately apply EBM, you must understand the research it is based on, so you can apply it correctly…..

• Most research is based on the “Scientific Method”

The Scientific Method

The Scientific Method

• The great advantage of the scientific method is that it is unprejudiced. • Scientific research is reproducible

• One does not have to believe a given researcher, one can redo the experiment and determine whether his or her results are true or false.

• Scientific Research is unbiased• The conclusions will hold irrespective of the state of mind, or the religious persuasion, or

the state of consciousness of the investigator and/or the subject of the investigation. • It focuses on the results of the science, regardless of what they show.

• Scientific Research evolves as our understanding evolves• It assumes that its conclusions may one day be disproven by new science, and that is OK

(after all , that is science)

Observation

Hypothesis

Predictions

Theory/ Empirical LawTheory/ Empirical Law

Experiment

The Scientific Method: The Theory/ Empiracal Law• A theory is accepted not based on the prestige or convincing powers

of the proponent, but on the results obtained through observations and/or experiments which anyone can reproduce.

• In other words: the theory is not based on the credentials of the scientists, rather on the strength and reproducibility of the theory itself…

The Scientific Method: Observation

• Observation: During a discussion you and your coworkers bring up the idea that there are more psychiatric emergencies when the moon is full.

The Scientific Method: Predictions

• Prediction: You predict that there will be more EMS calls for psychiatric emergencies when the moon is full as compared to other times of the month.

The Scientific Method: The Experiment• You need a study group and a control group…

The Scientific Method: The Hypothosis• Hypothesis: Psychiatric emergencies are more common when the lunar cycle is in

the full moon phase.

The Scientific Method: Experiment

• Testing: You set up an experimental protocol to test your hypothesis.• You need:

• A study method• A study period• Study Parameters• A study group• A control group

The Scientific Method: Experiment

• A study method• You decide that you will review charts (AKA a retrospective review)

• A study period• You decide to review the charts for a period of a year

• Study Parameters• You will review all charts over the period• You will define the term “Psychiatric Emergency” • You will gather information over the course of the year that includes the date of all

psychiatric emergency patients.• A study group

• Al patients that occur in your “Full Moon Period”• Define “full Moon Period” as 2 days before and after a full moon.

• A control group• All other patient contacts outside of your full moon period….

The Scientific Method: The Experiment

Results:

Full Moon PeriodDays=65

Other PeriodDays = 300

Total Days = 365

Ψ patients = 10 Ψ patients = 74 Total Ψ patients = 84

Total Patients = 2,105

Total Patients = 9,300

Total Patients = 11,315

The Scientific Method: The Experiment

• Use of the scientific method in EMS.• Results: Your study finds:

• During full moon period 0.5% of patients were Ψ patients.• During other period 0.8% of patients were Ψ patients.• During the entire study period 0.7% of patients were Ψ patients.

The Scientific Method: The Experiment

• Use of the scientific method in EMS.• Results: Your study finds:

• The incidence of Ψ patients during the full moon period was 0.15 patients per day.• The incidence of Ψ patients during the other moon phases was 0.25 patients per

day.• The incidence of Ψ patients overall was 0.23 patients per day.

• What does this mean?????• Psych patients are less common during the full moon.

• HYPOTHESIS DISSPROVED!

• What Your Hypothesis didn’t work out? That’s ok, because its SCIENCE, and the result is what counts , not our assumptions going into the experiment….

• SO , now what do you do?

Answer: You change your hypothesis to match your results (unless you are a drug company, then you do it the other way around….)

• Revise hypothesis:• New hypothesis: Psychiatric emergencies are no more frequent during full

moon lunar phases than during other lunar phases.• Based on research, this HYPOTHESIS is now ACCURATE (until disproven)

Now what do you do with your new and proven/researched hypothesis?• 1- TELL PEOPLE SO THEY CAN BENEFIT!

• Report findings. Paper should be detailed enough that anybody can follow it and repeat your experiment in their own enviroment

• 2- Repeat study to determine whether findings can be repeated.• i.e. does your study hold true in other systems?• Are there any weakness in your study you can correct, and then avoid in a

new study?

EMS Research

• The more valid a study, the closer it is to the truth!

EMS Research

• Bias: • A mental leaning or inclination; partiality; prejudice; bent.• The more a study design adheres to the scientific method, the less chance for

bias to affect the outcomes.

EMS Research

• Bias (sometimes flat deception) has been a common practice in medicine and EMS.

• Quackery and bias still permeates many aspects of modern healthcare.

EMS Research

• Validity: • Whether the study measures what it was supposed to measure.• Validity refers to the appropriateness of the interpretation of the results of a study.

• Constant: • A characteristic or condition that is the same for all individuals in a study.

• Variable:• A characteristic that takes on different values or conditions for different individuals.

• Dependent Variable: the variable being affected or assumed to be affected my the independent variable.

• Independent Variables: the variables that affects (or is assumed to affect) the dependent variable under study.

• Experimental Variable: at least one independent variable being manipulated by the researcher.

Understanding the “evidence”

Levels of Evidence

Levels of Evidence

Levels of Evidence (pre 2010)

• American Heart Association1. Positive randomized controlled trials.2. Neutral randomized controlled trials.3. Prospective, non-randomized controlled trials.4. Retrospective, non-randomized controlled trials5. Case series (no control group)6. Animal studies7. Extrapolations8. Rational conjecture (common sense)

Levels of Evidence

• Used to be 7 (or 8 ) different ILCOR levels of Evidence (LOE).

• In 2010 ILCOR reduced these to 5. • The AHA has broken these into three

broad categories

• Level A – Highest standard of evidence. • Level B – Most common. • Level C – Anecdotal case reports,

consensus opinions, retrospective studies, small studies, previous standard of care without evidence to contrary

KEY POINT:

“ A recommendation with Level of Evidence B or C does not imply that the recommendation is weak. Many important clinical questions addressed in the guidelines do not lend themselves to clinical trials. Even though randomized trials are not available, there may be a very clear clinical consensus that a particular test or therapy is useful or effective.”

-2010 AHA ECC GuidelinesCirculation 2010;122;S657-S664

Classes of Recommendations

• Most previous classes retained with better clarifications and descriptions.

•“Class Indeterminate” recommendations, which were used in 2005, are not included in the 2010 AHA Guidelines for CPR and ECC. The elimination of the term “Class Indeterminate” is consistent with the ACCF–AHA Classes of Recommendation.

•When the AHA writing groups felt that the evidence was insufficient to offer a recommendation either for or against the use of a drug or intervention, no recommendation was given.

Types of Studies: Randomized Controlled Trials• Randomized Controlled Trial (RCT): Gold Standard

• A group of patients are randomized into an experimental group and a control group. These groups are followed up for the variables/outcomes of interest.

• RCTs most closely approximate the scientific method and are the most valid of studies.

• A “double blind study” a type of RCT is one in which neither the patient nor the physician knows whether the patient is receiving the treatment of interest or the control treatment.

Randomized Controlled Trial

Double Blind Study

Quasi-Experimental Studies

• Quasi-experimental studies use intact groups of subjects rather than assigning subjects to groups at random.

Quasi-Experimental Studies

• Non-randomized controlled trial:• 1 group receives intervention• 1 group receives no-intervention (control)• Subjects assigned to groups by methods other than randomization.

Cohort Studies

• A Cohort Study is a study in which patients who presently have a certain condition and/or receive a particular treatment are followed over time and compared with another group who are not affected by the condition under investigation.

Cohort Study

Case-Control Studies

• Case Control Studies: Case control studies are studies in which patients who already have a certain condition are compared with people who do not.

Case-Control Studies

Case Series/Report

• A case series is a report on series of patients with an outcome of interest.

• No control group is involved.• • “This one time, at band camp, I had a bunch of patients) with ABC

condition, we did XYZ treatment, and here is what happened as a result. Aint it cool!”

• A case report is a narrative report of a single interesting case.

Historical Studies

• Historical studies are systematized searches for the facts and then using the information to describe, analyze, and interpret the past.

Systematic Reviews

• A summary of the medical literature that uses explicit methods to systematically search, critically appraise, and synthesize the world literature on a specific issue.

Meta-Analysis

• A meta-analysis is a systematic review of multiple studies that have already been done, rather than conducting a new study, to draw conclusions based on the whole of these studies…..

Meta-Analysis

Meta-Analysis

• Meta-analysis of RCTs represents the highest level of scientific evidence.

• Based on the theory that multiple RCT’s will show more truth than a single RCT will….

• Allows for more objective appraisal of the evidence.• Reduces the possibility of false negative or false positive results.

Meta-Analysis (Observational Studies)

• Less valid than RCTs.• Still valid in that it gives better information of the population as a

whole.• Most common means of studying EMS related interventions.

• Many EMS interventions cannot be tested with an RCT as it might be unethical to withhold care from the control group.

Meta-Analysis (Observational Studies)

• Conclusions:• 2 out of 3 trauma patients transported from the scene to a trauma center

have minor injuries based on validated trauma scoring criteria.• 1 out of 4 patients transported are discharged from the emergency

department.

Animal Studies

• Animal studies provide insight into biology, particularly into specific systems.

• For ethical reasons, some studies cannot be carried out on humans. But can be carried out on animals

• Much resuscitation research conducted on pigs and dogs. • Much toxicology and neurological research conducted on mice.

• Computer modeling are replacing many animal studies.

Expert Opinion

• Expert opinions can take various forms:• Systematic reviews• Narrative reviews• Pure opinion pieces• Position Papers produced by organizations

• Consensus papers• AMA• ACEP• NAEMTs

Evidence vs. Expert Opinion?

• If CONSENSU Sis the basis for a guideline, how is this distinguishable from EVIDENCE-based guidelines?

• What are the implications of having these guidelines LOOK equally authoritative when they make it to the street, when they ar ereally two different levels of evidence?

Rational Conjecture

• Lowest level of scientific validity.• But, overall very important.• “You don’t have to run a Chi-

Square test on common sense”

What is Psuedo Science?

• Psuedo Science is weakly based conjecture often passed off as based on science, often intentionally, with ulterior motives.

• Herbal Supplements• “Tactical” medical products

• Just make it black and wityh tons of Velcro and someone will buy it!• Targeted marketing

• “Hero”/”Ricky Rescue” complex

• It will often be supported with flashy presentations and plenty of research, but when reviewed closely the research is conducted with bias and conflicts of interests, and the presentations are weakly interpreted extrapolations….

EBM: How does it fit together?

• Rare to find a body of knowledge that “writes the guideline for you.”

• Requires explicit cognitive steps that translate DIRECT evidence into DIRECT guidelines through INFERENCES.

• From “Bench Test” to “Bedside Test”

• Example: Animal studies -> Human studies in limited settings -> Guideline applied across a broad population in dramatically different settings.

• Inevitably requires judgment, extrapolation, experience, inference, and opinion

Summary

• EMS must be driven by science.• Science is based upon quality research.• EMS providers of the future must be able to understand, and in

certain cases, conduct valid research.

Summary

• Evidence-Based Medicine:• EMS must start adhering to the tenets of evidence-based medicine.• Third-party payers will soon stop paying for care and procedures not

supported by science.• Litigation may follow if non-evidence-based practices continue.

Evidence-Based Medicine

• Evidence-based medicine is not restricted to randomized controlled trials and similar studies. It involves tracking down the best external evidence with which to answer our clinical questions.

The End

• Don’t be afraid of research. It can actually be fun!