1

Data Extraction

Handbook, Chapter 7

2

Data Collection Form

Importance Linked directly to review question and

planned assessment Historical record of decisions through the

review process Data repository, from where the analysis will

emerge Vary across reviews, but fundamental

components

3

Components of Data Form

Paper vs electronic forms → Chapter 7.5.2 for considerations

Consider how much information to collect (too much vs too little)

Careful thought and planning

Logical to entry into RevMan, especially for electronic forms

4

Components of Data Form (continued)

Items to include:– Review title, author name, who collecting data– Review unique ID, Study ID (RevMan), unique

report ID (multiple reports)– Date (for multiple chronologic versions)– Notes section (up front) – use for RevMan– Verification of study eligibility Table of excluded studies and reasons required– Study characteristics, information to assess bias,

results– Accurate coding: instructions and decision rules,

use of ‘Not reported’ and ‘Unclear’

5

Details for Protocol

Data categories to be collected– Again, the study is the unit of interest

Who and how many for collection– 2 people, independent– Content and non-content expert?

Piloting, training, existence of coding instructions for the data form– Piloting for information and instructions– Kappa not routinely done, but can and only for the

most important data

6

Details for Protocol (continued)

How data extracted for multiple reports of the same study– How to collate: CONSORT flow diagrams may

help How disagreements handled

– Process (consensus → arbitrator → study authors → report disagreement in review)

Blinding to aspects of study reports not generally recommended

7

Data Form: Process

Think – what are the needs? Design – draft form Pilot – sample of papers, compare completed

forms Refine – modify form, instructions Extract – further revisions may be needed

once data extraction underway, this is okay

Consider retraining or recoding with passage of time, also for updating

8

Characteristics of Included Studies

Extracted data:– Methods– Participants– Interventions– Comparisons– Outcomes– Information for risk of bias (later)– Notes

9

Characteristics of Included Studies

10

Characteristics of Included Studies

11

Characteristics of Included Studies

12

Table of Excluded Studies

13

Studies Awaiting Classification

14

Table of Ongoing Studies

15

Data Types

Dichotomous Continuous Ordinal Counts and rates Time-to-event

16

Dichotomous (binary) Outcomes

The outcome is one of two possibilities, cannot be both e.g., pregnant vs. not

Intervention

Event No event

a b

c d

Event No event

Control

a+b = nI

c+d = nC

Data required: ‘event’ and ‘no event’ for

each group

17

Dichotomous outcomes (continued)

May experience difficulties with:– Poor reporting– Numbers may need to be derived from

percentage data provided in the report (which denominator to use, compatibility with more than one numerator)

Sometimes ordered categorical data (ordinal) are treated as dichotomous data

18

Continuous outcomes

Outcomes that can take any value in a specific range eg, weight, length of stay

Sometimes data from ordered categories (ordinal) are treated as continuous

Check if data can be treated as continuous (Consult CRG statistician)

Effect measures: mean difference (difference in means) or standardized mean difference (factors in standard deviation)

19

Ordinal outcomes

Categories with a natural order Can range in number of categories Measurement scales: important to know if

validated Discussion on how to analyze: Section 9.2.4 Analyze as dichotomous? Continuous? As

is? Consult CRG statistician Extract data in all forms in which reported

20

Counts of events

Events that can happen more than once to a

given individual Eg, MI, adverse event Common vs. rare events Different methods exist for analysis Consult CRG statistician Extract data in the form they are reported in

21

Time-to-event outcomes

Analysis of whether the event occurred and when ‘Survival data’ in statistics E.g., survival, disease recurrence For each individual:

– ‘no event’ period– at the end of that period, whether event occurred

or is just the end of observation (censored) Hazard ratio the most appropriate effect measure Methods of meta-analysis Section 9.4.9; consult

CRG statistician

22

Planning Your Analysis

Handbook, Chapter 9

23

Planning Your Analysis

Specify comparisons First and most important step! Back to PICO – should relate clearly and directly

24

What Comparisons are Important?

Pair-wise comparisons– Glucosamine versus placebo– Glucosamine versus no therapy– Glucosamine versus NSAIDs– Acupuncture plus vitamin B12 injections versus

B12 injections alone

Specify the main comparisons in the protocol If need to modify in light of the data (eg,new

comparison)…document!

25

Characteristics of Intervention and Control

Are the interventions or controls all the same?

Different types of drugs used (eg inhaled steroids? NSAIDs?)

Different dosages/duration of therapy/preparation

If not the same, are they similar enough to be combined?

26

Population Description

Separate or combine? E.g. Mild versus severe rheumatoid arthritis Age issues? Defining separation points?

27

Outcomes

Determine what to combine – shouldn’t be too diverse

Start with the outcomes that are considered most important (specified in protocol)

Eg, mortality, pain, function

May be same variable, but could be categorical in one study and continuous in another

Eg, pain – continuous: 0-10 mmVAS, 1-5 Likertdichotomous: ‘moderate’, ‘severe’

RevMan: outcomes are entered after the comparisons have been set up

28

Hierarchy of Data and Analysis Section

• Results of studies• Tabular• Fixed format• Forest plots automatically generated

29

Sample Table

Hyperlink to Forest plotHyperlink to Forest plot

30

Link out to Forest Plot

31

Make a Plan

Sketch it out on paper Decide what you want to do before you start

entering into RevMan Document changes between the protocol and

conducting the review

32

Critical Elements

Input from more than one person, including “expert”

Transparency; state post-hoc decisions Clear, consistent

33

Data extraction exercise