Click here to load reader

Critical Appraisal Therapy

  • View

  • Download

Embed Size (px)



Text of Critical Appraisal Therapy

  • *Critical AppraisalTherapyJ.EKO WAHONO. R.dr.SpS.M.KesSMF I.P.SarafRSUD Dr.Soetomo Surabaya

  • *Principal of critical appraisal-primary researchInvolves 3 overall question

    What is the PICO of the study, and is it closed enough to your PICO ?How well was the study done ?What do the results mean and could they have been due to chance ?

  • *Critical appraisal TxStudyYour question(PICO)What do the result mean ?How well wasThe study done ?Is the study questionthe same as your question ?

  • *Wheter wearing elastic stockings on long-haul flights helps to prevent deep vein trombosis (DVT)

    Population/problem= passengers on long-haul flights

    Intervention= wearing elastic compression stockings

    Comparator/control= no elastic stockings

    Outcome= symptomless DVT

  • *Critical appraisal Tx DVTClinical question :In passengers on long-haul flights, does wearing elastic compression stockings, compared to not wearing elastic stockings, prevent DVT ?Search terms :Based on the clinical question (PICO) (flight*OR travel*) AND stockings*AND (DVT OR trombosis)

  • *Critical appraisal Tx DVTSearch result :PubMed Clinical Queries therapy,broad), 20 hits(referring to 5 studies and several reviews, including 1 recent Cochrane review)For exercise :Scurr et al (2001). Frequency and prevention of symptomless DVT in long-haul flights:a randomised trialThe Lancet 357:1485-1489Authorsconclusion :Wearing of elastic compression stockings during long-haul air travel is associated with a reduction in symptomless DVT

  • *AnalyseHow do we know that the results are valid and real ?Wearing elastic stockings is an intervention --- RCT okIn the real life straight to the Cochrane systematic review

  • *Question 1 : Is the PICO of the study close enough to your PICO ?If you find a study that will answer your clinical question Study PICO match your PICO or not !Example your PICOP= In patients with rheumatoid arthritisI= Does taking anti-inflammatory drugsC= no treatment or simple analgeticsO= Increase or reduce fatique

  • *If study PIC = your PICbut study O # your O

    StudyO= reduce joint painYourO= fatiqueHowever :The study may report on some measure of fatique as secondary outcomePlease decide quickly wheter to critize or not

  • *Question 2 :How well was the study done ?The quality of an epidemiological studyInternal validityFree from bias & confounding factorsBiasThe degree to which the result is skewed away from the truthSelection biasTreament biasMeasurement biasTo overcome bias RCT & Blinding

  • *Question 2 :How well was the study done ?The quality of an epidemiological studyInternal validityFree from bias & confounding factorsConfounding factorsPatients features & causal factorsTo overcome CF Both Group are closely matched/similarThe management of the group is the same

  • *How well was bias & confounding factors were avoided ?Check each stage of the study How fairly were : the subjects recruitted (the P)the subjects allocated to groups ( the I and C )the study group maintained through equal management and follow up of subjects ( the I and C )the outcome measured ( the O )

  • *Stucture of a comparative health care research study

    Study questionAimStudy methods Critical appraisalPFair recruitmentSubject representative of the target popLarge enaugh sample + randomlyRIFair allocationRandomly allocatedAdjust confounding (statitical adjustment/matching)ACFair maintenanceManage grup equallyFollow up all subjectsMOFair measurementValid & unbiased outcome measureMeasure outcomeBlindedObjective measureMBo

  • *Steps in critical appraisal of primary research - RAMMBORecruitment :Were the subjects representative of the target pop ?Allocation or adjustmentWas the tx allocation concealed before randomisation and were the groups comparable at the start of the trialMaintenance :Was the comparable status of the study groups maintained through equal management and adequate follow up ?MeasurementWere the outcomes measured withBlinded subjects and assessors, and/orObjective outcomes ?

  • *RecruitmentWere the subjects representative of target population ?If the subject are not representative difficult to know to which pop may be applicableThe best way to ensure study group representative is to :Recruit potential subjects SequentiallyAt random from populationOnly apply exclusion criteria relevant for study methodsExcluding deaf people from study requiring subjects to listen the music

  • *RecruitmentWere the subjects representative of target population ?Prefer large study, because small study group imprecise estimate of the effects !Continuous outcomes 50 100 ptBinary outcomes :Common event hundreds ptRare event thousands pt

  • *RecruitmentWere the DVT trial subjects representative of target population ?Inclusion/exclusion criteriaFor RCT difficult random sampling due to inform consentClear idea who they do representDescribe the severity,duration and/or risk level of the patients recruited

  • *RecruitmentWere the DVT trial subjects representative of target population ?VolunteersWere recruited by placing advertisements in paperPassengers > 50 yo Economy class At least 8 hours flight within 6 weeksVarious exclusionsSize of study groups231 116 received stockings & 116 noThis seem smallAs a 10% DVT rate 12 events

  • *AllocationWere the study groups comparable ?It is vital the groups are matched except for the interventions ( or exposure/other indicator)Ways in which groups could differ AgeSexSmoker/nonsmokerDisease severityRandom allocation

  • *Allocationwere the DVT study groups comparable ?The paper states Volunteers were randomised by sealed envelopre to one of two groups

  • *Allocation Characteristic of DVT study groupsP < 0.01

    No stockingsStockingsNumber116115Pre-study Age62(56-68)61(56-66) Females61(53%)81(70%) Varicose veins4145 Hb142140During study Hours flying2224 Day of stay1716

  • *Maintenancewas the comparable status of the study groups maintained through equal management and adequate follow-up ?Once comparable groups have been set up stay a that wayEqual managementUnequal tx invalidates result !In a trial of vit E in preterm infantsVit E appeared to prevent retrolental fibroplasia ?It was not !Control groups 100% O2Tx groups not 100% O2 because the babies were removed from O2 for freq dose of vit E

  • *MaintenanceAdequate follow-upInevitably, some subjects drop out, change groups or variously lost to follow up during study uncomparable groups !Check :Subject at start = at the endSubject are analysed in the groups that they stated out in ( Intention-to-treat principle )

  • *MeasurementWere the outcomes measured with Blinded subjects and accessorrs and/or Objective measures ?Measurement biasHuman tendency to unfairly nudge resultsCan be overcome by BlindingObjective measurement

  • *.MeasurementBlindingBest Double-blind trial Moderate Single blindWorst Not blinded

    Placebo effect

  • *Question 3:What do the result mean ?Outcome measuresBinaryContinuous

    Are the results real and relevent ?Assesing change :P-values (hypothesting)Confidence interval (estimation)

  • *Outcome measures for binary outcomes

    MeasureMeaningExampleRelative risk Risk of outcome in the Tx group/risk in the control groupRR how many times more likely event in the tx group relative to control groupRR = 1 no diff between 2 groupsRR < 1 the tx reduces the risk of eventRR > 1 the Tx increass the risk of the eventRR = 0.1/0.15= 0.67

    RR < 1The Tx decrease the risk of death

  • *Outcome measures for binary outcomes


    ARRRisk of event in the control group risk event in the tx groupARR = 0 no diffARR + the Tx is beneficialARR - harmfullARR = 0.15 0.10 = 0.05 (5%)The absolute benefit of Tx is a 5% reduction in the death rate

  • *Outcome measures for binary outcomesRRR


    RRRARR/risk of event in the control groupRRR Reduction in the rate of event in the Tx group relative to control groupRRR =ARR / Risk of event control group

  • *Outcome measures for binary outcomesNNT


    NNT= 1 / ARRNNT =The number of pts we need to treat in order to prevent bad event

  • *P-valuesAre a measure of the probability that the result is purely due to chance

  • *Confidence interval (CIs)More informative > P valueAn estimate of the range of value that likely to include the real value95% means :The range of values that have a 95% chance of including the real valueIf the 95% CI for the diff between Tx & Control groupSmallNo overlap the no effect point

    The result is real ( that is, with a P-value < 0.05)

  • *Confidence interval (CIs)Diff betweenTx vs ControlNull hypothesis(no effect)A B C D= point estimate= CIA : statistically sig result (p < 0.05 ) but low precisionB : statistically sig result (p < 0.05 ) with high precisionC : not statistically sig result (p > 0.05 ) with low precisionD : not statistically sig result (no effect ) with high precision

  • *Intervention = usesful if The 95% CI includes clinically important tx effectsStatistically sigRelates to the size of the effect and the 95% CI in relation to the Null hypothesisClinical importanceRelates to the size of effect and the 95% CI in relation to a minimum effect that would be considered to be clinically importantce

  • *Clinical importanceMinimum clinicalImportant diffNull hypothesis(no effect)A B C D= point estimate= CI

Search related