PTP 661

EVIDENCE ABOUT INTERVENTIONS

CRITICALLY APPRAISE THE QUALITY AND APPLICABILITY

OF AN INTERVENTION RESEARCH STUDY

Min Huang, PT, PhD, NCS

Applicability

Determining Applicability of an Intervention Study

Start from a specific patient or a more general question about types of patients with the shared problem

Search for the literature Use PICO analysis to identify the search terms

e.g. “For a 17-year-old swimmer with neck pain, is a combination of manual therapy and exercise effective for reducing pain and improving function?”

Determining Applicability of an Intervention Study

Q1: Is the study’s purpose relevant to my clinical question? Read the abstract

Q2: Is the study population (sample) sufficiently similar to my patient? To justify whether my patient would respond similarly

to the population

Q3: Are the inclusion and exclusion criteria clearly defined, and would my patient qualify for the study? Is the treatment feasible in my clinic?

Determining Applicability of an Intervention Study

Q4: Are the intervention and comparison/ control groups receive a realistic intervention?

Q5: Are the outcome measures relevant to the clinical question and were they conducted in a clinically realistic manner?

Bottom line is to …… weigh similarities and differences between the study

participants and your patient

the intervention proposed in the study and the feasibility of this intervention for you and your patient

Quality and Results

Determining Quality of an Intervention Study

Q1: Were the participants randomly assigned to intervention groups? RCT is the gold standard of research design for

intervention studies

RCT is designed to evaluate the efficacy of an intervention

Efficacy: test-tube, ideal condition

Effectiveness: typical clinical conditions

Practice-Based Evidence Study Design for Comparative Effectiveness Research

Although randomized controlled trials (RCTs) are important to confirm whether a new treatment causes an effect, they are unlikely to discover combinations of interventions or practices that are effective and efficient in routine care.

PBE study is a type of observational study often utilizes existing data from medical records

can uncover better practices more quickly than RCTs

Horn 2007, http://www.effectivehealthcare.ahrq.gov/repFiles/MedCare/s50.pdf

Determining Quality of an Intervention Study

Q7: Is the sampling procedure (recruitment strategy) likely to minimize bias? Sample size and study power

Recruiting a study sample

ALL patients from selected clinics vs. SOME patients referred from multiple clinics

Q8: Are all participants who enter the study accounted for? Intention to treat (ITT)?

Determining Quality of an Intervention Study

Q9: Was blinding optimized in the research design? Evaluator

Participants

Therapists

Q10: Aside from the treatment, were groups treated equally?

Interpreting Results of an Intervention Study

Q11: Were participants similar at the baseline?

Q12: Were outcome measures reliable and valid?

Q13: Were C.I., descriptive and inferential statistics (p values) reported and applied to the results?

Clinical Bottom Line

Summarizing the Clinical Bottom Line of an Intervention Study

Q13: Was there a treatment effect? If so, was it clinically relevant? Change on a measure that has value to the patient

Change of a magnitude that will make a real different in the patient’s life

Effect size

e.g. Cohen’s d

Large= >0.8; Medium= 0.5-0.8; Small= 0.2-0.5

Number Needed to Treat (NNT)

Minimally Clinically Important Difference (MCID)

Figure 12-2: Influence of Group Variability on Effect Size

Compare absolute effect size and standardized effect size between the groups

Example: NNT

NNT = 2. For every two smokers who quit, one life is saved.

Which intervention is more effective? NNT = 7. hot pack + interferential

NNT = 3. education + core stabilization (better, need to only treat 3 to save someone)