Upload
mesfin-mulugeta
View
46
Download
4
Tags:
Embed Size (px)
Citation preview
EVALUATION OF EVIDENCE
Shikur MohammedShikur Mohammed
(BSc, MPH)
2/2/2015 1By Shikur Mohammed (BSc, MPH)
Evaluation of Evidence
� Evaluation is a process of determining the
usefulness, reliability and validity of something
against explicit predetermined standards
2/2/2015 2By Shikur Mohammed (BSc, MPH)
Common Problems in observed findings
� Inadequacy of the observed sample (The role ofchance)
� Inappropriate selection of study subjects & unfair
data collection methods (Bias)data collection methods (Bias)
� Comparing unequal (Confounding)
2/2/2015 3By Shikur Mohammed (BSc, MPH)
1. The role of chance
� The larger the sample on which the estimate is
based, the less variability and the more reliable the
inference
� This is done by performing an appropriate test of
statistical significancestatistical significance
� A measure that is often reported from all tests of
statistical significance is the P-value
2/2/2015 4By Shikur Mohammed (BSc, MPH)
The role of chance cont…
� P < 0.05 - statistically significant
� P > 0.05 – no statistically significant association (
chance can not be excluded as a likely explanation)chance can not be excluded as a likely explanation)
2/2/2015 5By Shikur Mohammed (BSc, MPH)
The role of chance cont…
� It is always advisable to report the actual P-value
rather than merely that the results did or did not
achieve statistical significanceachieve statistical significance
� confidence interval (CI) is far more informative
measure than P-value to evaluate the role of chance
2/2/2015 6By Shikur Mohammed (BSc, MPH)
Bias
� It is any systematic error in the design, conduct or
analysis of a study
� Types of biasTypes of bias
1. Selection bias,
2. Information (Observation) bias
2/2/2015 7By Shikur Mohammed (BSc, MPH)
1. Selection bias
� It is a bias introduced while selecting the study
participants
� It is a particular problem in case control and� It is a particular problem in case control and
retrospective cohort studies. E.g. If the way in
which participants are selected into the study is
different for cases and controls
2/2/2015 8By Shikur Mohammed (BSc, MPH)
Examples of selection bias
a) Diagnostic bias: Diagnostic approach related to
knowing exposure status
For example
� women who take oral contraceptives (OCs) may be
screened more often for breast cancer than women who
do not take OCs because of the suspected link between
oral contraceptive and breast cancer
2/2/2015 9By Shikur Mohammed (BSc, MPH)
selection bias…
b). Volunteer bias/ Compliance bias: People who
accept to participate in a study or people who refuse
to participate are often quite different from the
general population
c). Non-response bias: This is due to differences in thec). Non-response bias: This is due to differences in the
characteristics between the responders and non-
responders to the study
2/2/2015 10By Shikur Mohammed (BSc, MPH)
selection bias cont…
d). Loss to follow up: Difference in completeness of
follow-up between comparison groups
e). Berkson’s bias: Studies carried out exclusively in
hospital settings are subject to selection biashospital settings are subject to selection bias
attributable to the fact that risks of hospitalization
can combine in patients who have more than one
condition
2/2/2015 11By Shikur Mohammed (BSc, MPH)
Ways of minimizing selection bias
• Population-based studies are preferable
• Avoid the inclusions as study subjects of people
who have volunteered on their own
In case-control study, it is useful to select several• In case-control study, it is useful to select several
different control groups
• keep losses to follow-up to minimum
2/2/2015 12By Shikur Mohammed (BSc, MPH)
2. Information/Observation bias
� Refers to bias which arises during the data collection
process
� It occur because of mistakes in categorizing study� It occur because of mistakes in categorizing study
subjects with respect to their exposure or disease
status
2/2/2015 13By Shikur Mohammed (BSc, MPH)
Examples of information bias
a) Investigator bias/ Interviewer bias/ Observer bias:
Occurs when investigators collect information
differently in different comparison groups
b) Response bias/Recall bias: Occurs as a result of
difficulty to recall prior exposuresdifficulty to recall prior exposures
c) Social desirability bias: Occurs because subjects are
systematically more likely to provide a socially
acceptable response
2/2/2015 14By Shikur Mohammed (BSc, MPH)
Examples information bias….
d). Placebo effect: tendency for individuals to report favorable
response to any therapy regardless of the physiologic efficacy
of what they received
e). Hawthorn effect: Refers to the change in the dependent
variable which may be due to the process of measurement or
observation itself
2/2/2015 15By Shikur Mohammed (BSc, MPH)
Ways of minimizing information bias
� Blinding: the study subjects doesn't know to which
group they are assigned
� using placebo: Use of placebo minimizes the bias in the
ascertainment of both subjective disease outcomes and
side effects. It facilitates that both groups in the study
gain equal attention
� Using standard procedures, instruments, questionnaires,
interviewing techniques
2/2/2015 16By Shikur Mohammed (BSc, MPH)
Confounding
� Confounding variable is a variable that can cause or
prevent the outcome of interest, is not an
intermediate variable, and is associated with the
factor under investigation
2/2/2015 17By Shikur Mohammed (BSc, MPH)
Confounding …
� Confounding variable must fulfill each of the following criteria:1. the variable must be associated with the exposure and,
independent of that exposure, be a risk factor for thediseasedisease
2. The distribution (frequency) of the confounding variableshould vary between the groups that are compared
3. Confounder must not be an intermediate link in a causalpathway between exposure and outcome
2/2/2015 18By Shikur Mohammed (BSc, MPH)
Confounding …….Example
� An observed association between drinking alcohol
and increased risk of lung cancer (LC) could be due
to the effect of cigarette smoking, since alcohol
drinking is associated with smoking and,drinking is associated with smoking and,
independent of alcohol consumption, smoking is a
risk factor for LC
2/2/2015 19By Shikur Mohammed (BSc, MPH)
Control for Confounding VariablesControl for Confounding VariablesControl for Confounding VariablesControl for Confounding Variables
� In the design:• Randomization• Restriction• Matching
� During analysis:� During analysis:• Standardization• Stratification• Multivariate analysis
2/2/2015 20By Shikur Mohammed (BSc, MPH)
Establishing causal relationships
� The following formal criteria are widely used to evaluate the
likelihood that an association is causal
1. Strength of the association
2. Dose-response relationship
3. Consistency of the relationship3. Consistency of the relationship
4. Temporal relationship
5. Specificity of the association
6. Biological plausibility (coherence with existing
information)
7. Prevention
2/2/2015 21By Shikur Mohammed (BSc, MPH)
Conclusion about causation
� The above criteria are the ones most frequently employed in
trying to establish causation
� None provides in itself a perfect means of providing
causation, and each has its limitationscausation, and each has its limitations
� However, when they are considered together, the weight of
the evidence may allow a tentative conclusion to be reached
2/2/2015 22By Shikur Mohammed (BSc, MPH)