The Cochrane Collaboration

and Cochrane Reviews

Cochrane HIV/AIDS GroupUniversity of California, San

Francisco

Reviews of scientific literature

Meta-analyses

Cochrane reviews / systematic reviews

Reviews(narrative/literature/traditional)

Narrative reviews

• Usually written by experts in the field• Use informal and subjective methods to

collect and interpret information• Usually narrative summaries of the

evidence

What is a systematic review?

• A review of the evidence on a clearly formulated question, that uses systematic and explicit methods to identify, select and critically appraise relevant primary research, and to extract and analyze data from the studies that are included in the review

Systematic vs. narrative reviews

Systematic review

• Scientific approach to a review article• Criteria determined at outset• Comprehensive search for

relevant articles• Explicit methods of appraisal and synthesis• Meta-analysis may be used to combine data

Narrative review

• Depend on authors’ inclination• Author gets to pick any criteria• Search any databases• Methods not usually specified• Narrative summary• Can’t replicate review

Systematic reviews: high quality

Structured, systematic process involving several steps :

– Plan the review (a priori protocol)– Formulate the question– Comprehensive search– Unbiased selection and abstraction process– Critical appraisal of data– Synthesis of data (may include meta-analysis)– Interpretation of results

– All steps are described explicitly in the review.

Advantages of systematic reviews

• Reduce bias• Replicability• Resolve controversy between conflicting

studies• Identify gaps in current research• Provide reliable basis for decision making

Limitations

• Results may still be inconclusive• There may be no trials/evidence• The trials may be of poor quality• The intervention may be too complex to be

tested by a trial• Practice does not change just because you

have the evidence of effect/effectiveness

Cochrane Reviews are seen as the “gold standard” for systematic reviews in such key publications as Lancet, NEJM, BMJ, and JAMA, and routinely appear there as well as in specialised medical journals for various specialty areas.

Cochrane Collaboration

• International non-profit organisation that prepares, maintains, and disseminates systematic up-to-date reviews of health care interventions

• Named for Archibald Leman Cochrane, CBE FRCP FFCM (1909-1988)

• Best known for his influential book, Effectiveness and Efficiency: Random Reflections on Health Services (1972)

~25,000 people involved in Cochrane

from >170 countries

Cochrane Collaborative Review Groups

Acute Respiratory Infections Group Airways Group Anaesthesia Group Back Group Bone, Joint and Muscle Trauma Group Breast Cancer Group Childhood Cancer Group Colorectal Cancer Group Consumers and Communication Group Cystic Fibrosis and Genetic Disorders Group Dementia and Cognitive Improvement Group Depression, Anxiety and Neurosis Group Developmental, Psychosocial and Learning Problems Group Drugs and Alcohol Group Ear, Nose and Throat Disorders Group Effective Practice and Organisation of Care Group Epilepsy Group Eyes and Vision Group Fertility Regulation Group Gynaecological Cancer Group Haematological Malignancies Group Heart Group Hepato-Biliary Group HIV/AIDS Group Hypertension Group Incontinence Group Infectious Diseases Group

Inflammatory Bowel Disease and Functional Bowel Disorders Group Injuries Group Lung Cancer Group Menstrual Disorders and Subfertility Group Metabolic and Endocrine Disorders Group Methodology Review Group Movement Disorders Group Multiple Sclerosis Group Musculoskeletal Group Neonatal Group Neuromuscular Disease Group Oral Health Group Pain, Palliative and Supportive Care Group Peripheral Vascular Diseases Group Pregnancy and Childbirth Group Prostatic Diseases and Urologic Cancers Group Renal Group Schizophrenia Group Sexually Transmitted Diseases Group Skin Group Stroke Group Tobacco Addiction Group Upper Gastrointestinal and Pancreatic Diseases Group Wounds Group

• Focused around health problems (n = 52)• Produce reviews• Editorial base facilitates review process• International and multidisciplinary

Systematic review process

• Well formulated question• Comprehensive data search• Unbiased selection and abstraction

process• Critical appraisal of data• Synthesis of data• Interpretation of results

Well-formulated questionCochrane reviews are generally framed in one of three

ways:

• INTERVENTION 1 for CONDITION• INTERVENTION 1 vs. INTERVENTION 2 for CONDITION• INTERVENTION 1 for CONDITION in POPULATION

e.g.

• Nutritional interventions for reducing HIV morbidity and mortality

• Immediate vs. deferred zidovudine for treating HIV• Iron supplementation for reducing morbidity and mortality

in children with HIV

Writing the protocol

• The Protocol is the a priori work-plan for the eventual review. It lays out your plan in detail.

• USE PICO– Population– Intervention– Comparison– Outcomes (primary / secondary)

Sometimes given as PICOT (T= Timeframe) Helps limit/better define scope of project Guides data extraction A priori PICO questions limit the risk of selection bias

Comprehensively search databases to find all relevant

studies• PubMed• EMBASE• Cochrane Central Register of Controlled Trials (CENTRAL)• CINAHL• Web of Science• PsycINFO• LILACS• Databases from LMIC• Dissertation abstracts• Clinical trials registers• Handsearching of journals• Reference lists• Conference abstracts• Personal communication• Anything else you can think of.

NO RESTRICTIONS ON LANGUAGE

PubMed search strategy, modified

as needed for use in other

databases

Un-biased selection and abstraction process

• Searches brings 1814 citations (gross).

• 331 duplicates are removed.• Applying inclusion criteria,

Author #1 reviews the remaining 1483 article titles (and occasionally abstracts) in EndNote, and excludes 1247 citations.

• Applying inclusion criteria, Authors #1 and #2 independently (separately) review 237 abstracts, and exclude 212.

• Authors obtain full-text articles of the remaining 23 citations, for closer scrutiny.

• Closer examination reveals that 16 of these don’t meet the inclusion criteria.

• Seven studies are included in the review.

Data abstractionUsing a pre-piloted data abstraction form, authors separately abstract the data

and study characteristics from each study determined to be eligible for inclusion.

SourceStudy IDReview author initialsCitation and contact details;MethodsStudy design;Total study duration;Sequence generation;Allocation sequence concealment;Blinding;Other concerns about bias;ParticipantsTotal number;Setting;Diagnostic criteria;Age;Sex;Country;[Co-morbidity];[Socio-demographics];[Ethnicity];[Date of study];

For each outcome of interest:Outcome definition (with diagnostic criteria if relevant);Unit of measurement (if relevant);For scales: upper and lower limits, and whether high or low score is good

ResultsNumber of participants allocated to each intervention group;For each outcome of interest:

Sample size;Missing participants;Summary data for each intervention group

(e.g. 2×2 table for dichotomous data; means and SDs for continuous data);[Estimate of effect with confidence interval; P value];[Subgroup analyses];MiscellaneousFunding source;Key conclusions of the study authors;Miscellaneous comments from the study authors;References to other relevant studies;Correspondence required;Miscellaneous comments by the review authors.

InterventionsTotal number of intervention groups;For each intervention and comparison group of interest:

Specific intervention;Intervention details (sufficient for replication, if feasible);[Integrity of intervention];OutcomesOutcomes and time points (i) collected; (ii) reported*;

Critical appraisal and synthesis of data

• Choose effect measure• Dealing with missing data• Assessment of heterogeneity• Data synthesis• Subgroup analysis and investigation of

heterogeneity• Assessment of biases• Sensitivity analysis• Meta-analysis

Assess the risk of bias in each study

1. Sequence generation (randomization)2. Allocation concealment3. Blinding of participants, personnel and

outcomes4. Incomplete outcome data (attrition and

exclusions)5. Selective outcome reporting6. Other (including topic-specific, design-

specific)

Risk of bias assessment in Cochrane reviews

The questions

• Was the allocation sequence adequately generated? • Was allocation adequately concealed? • Was knowledge of the allocated intervention

adequately prevented during the study? • Were incomplete outcome data adequately

addressed? • Are reports of the study free of suggestion of

selective outcome reporting?

• Was the study apparently free of other problems that

could put it at a high risk of bias?

Interpretation of data

• What does it all mean?• Tables and figures help to present included studies

and their findings in a systematic and clear format. • Forest plots • A GRADE “Summary of findings” table provides key

information concerning the quality of evidence, the magnitude of effect of the interventions examined, and the sum of available data on all important outcomes for a given comparison.

• The Abstract of a Cochrane review should be targeted primarily at healthcare decision makers (including clinicians, informed consumers and policy makers)

Cochrane Collaborative Review Group on HIV Infection and AIDS

• Established in 1997• Editorial base at University of California, San Francisco (UCSF)• Satellite editorial base at South African Cochrane Centre (SACC)

Audiences for our reviews• California Department of Health Services• Centers for Disease Control and Prevention (CDC), Global

AIDS Program• South Africa Department of Health• South African Medical Research Council• United Kingdom Medical Research Council• United States Council on Foreign Relations and Milbank

Memorial Fund• United States Department of State – President’s Emergency

Plan for AIDS Relief (PEPFAR)• United States Institute of Medicine• United States Office of Minority Health• World Health Organization

Maximising the dissemination of evidence-based analysis of HIV/AIDS

prevention, treatment and care interventions

• Systematic reviews and evidence assessments on topics requested by funders and policy makers, using Cochrane (and now GRADE) methods

• Most recent guideline development projects with WHO:– Adult and adolescent ART – Comprehensive prevention and care interventions for

HIV+ adult– Comprehensive prevention and care interventions for

HIV+ child– PMTCT