8/26/2004 Ben Dwamena: 3rd North American Stata Users Group Meeting
1
METAGRAPHITI BY STATA
Ben A. Dwamena, MDDepartment of Radiology, University of Michigan Medical School Nuclear Medicine Service, VA Ann Arbor Health Care System
Ann Arbor, Michigan
8/26/2004 Ben Dwamena: 3rd North American Stata Users Group Meeting
2
METAGRAPHITI
Statistical Graphics For Interpretation, Exploration And Presentation Of Meta-analysis Data
8/26/2004 Ben Dwamena: 3rd North American Stata Users Group Meeting
3
METAGRAPHITI BY STATA
VISUOGRAPHIC FRAMEWORK FOR USING STATA FOR:
Testing and correcting for publication bias
Investigating heterogeneity
Summary of Data and Sensitivity Analyses
8/26/2004 Ben Dwamena: 3rd North American Stata Users Group Meeting
4
METAGRAPHITIAvoid potential misrepresentation by faulty distributional and other statistical assumptions.
Facilitates greater interaction between the researcher and the data by highlighting interesting and unusual aspects of the quantitative data.
8/26/2004 Ben Dwamena: 3rd North American Stata Users Group Meeting
5
METAGRAPHITI
User-friendlier summaries of large, complicated quantitative data sets.Preliminary exploration before definite data synthesis.Effective emphasis of important features rather than details of data.
8/26/2004 Ben Dwamena: 3rd North American Stata Users Group Meeting
6
CONTINGENCY TABLE FOR EXTRACTION OF DATA
8/26/2004 Ben Dwamena: 3rd North American Stata Users Group Meeting
7
DIAGNOSTIC VS. TREATMENT TRIAL
True Positives =Experimental Group With the Monitored Outcome Present (a). False Positives = Control Group With the Outcome Present (b). False Negatives=Experimental Group With the Outcome Absent (c). True Negatives= Control Group With the Outcome Absent (d).
8/26/2004 Ben Dwamena: 3rd North American Stata Users Group Meeting
8
DIAGNOSTIC VS. TREATMENT TRIAL
The Expression for the Odds Ratio (OR) =(a x d)/(b x
c).
Relative risk in experimental group {[a/(a + c)]/[b/(b+
d)]} =Likelihood Ratio for a Positive Test.
Relative Risk in Control Group = Likelihood Ratio for
a Negative Test.
8/26/2004 Ben Dwamena: 3rd North American Stata Users Group Meeting
9
EXPLORING PUBLICATION BIAS
Published studies do not represent all studies on a specific topic. Trend towards publishing statistically
significant (p < 0.05) or clinically relevant results. Publication bias assessed by examining
asymmetry of funnel plots of estimates of odds ratios vs. precision.
8/26/2004 Ben Dwamena: 3rd North American Stata Users Group Meeting
10
EXPLORING PUBLICATION BIAS
Funnel plot
Begg’s rank correlation plot
Egger’s regression plot
Harbord’s modified radial plot
8/26/2004 Ben Dwamena: 3rd North American Stata Users Group Meeting
11
FUNNEL PLOT
A funnel diagram (a.k.a. funnel plot, funnel
graph, bias plot):
Special type of scatter plot with an
estimate of sample size on one axis vs.
effect-size estimate on the other axis
8/26/2004 Ben Dwamena: 3rd North American Stata Users Group Meeting
12
FUNNEL PLOT
Based on statistical principle that sampling error decreases as sample size increases
Used to search for publication bias and to test whether all studies come from a single population
8/26/2004 Ben Dwamena: 3rd North American Stata Users Group Meeting
13
FUNNEL PLOTS
8/26/2004 Ben Dwamena: 3rd North American Stata Users Group Meeting
14
FUNNEL PLOTS
STATA SYNTAX/COMMAND• metafunnel ldor seldor, xlab(0(2)8) xtitle (Log
odds ratio) ytitle(Standard error of log OR) saving(zfunnel, replace)
• metafunnel ldor seldor, xlab(0(2)8) xtitle(Log odds ratio) ytitle (Standard error of log OR) egger saving (eggerfunnel, replace)
8/26/2004 Ben Dwamena: 3rd North American Stata Users Group Meeting
15
8/26/2004 Ben Dwamena: 3rd North American Stata Users Group Meeting
16
FUNNEL PLOT0
.51
1.5
2S
tand
ard
erro
r of
log
OR
0 2 4 6 8Log odds rat io
Funnel plot with pseudo 95% confidence limits
8/26/2004 Ben Dwamena: 3rd North American Stata Users Group Meeting
17
FUNNEL PLOT0
.51
1.5
2S
tand
ard
erro
r of
log
OR
0 2 4 6 8Log odds ratio
Funnel plot with pseudo 95% confidence limits
8/26/2004 Ben Dwamena: 3rd North American Stata Users Group Meeting
18
BEGG’S BIAS TEST
An adjusted rank correlation method toassess the correlation between effect estimates and their variances.Deviation of Spearman's rho from zero=estimate of funnel plot asymmetry. Positive values=a trend towards higher levels of effect sizes in studies with smaller sample sizes
8/26/2004 Ben Dwamena: 3rd North American Stata Users Group Meeting
19
BEGG’S BIAS TEST
STATA SYNTAX/COMMAND
metabias LogOR seLogOR, graph(b) saving(beggplot, replace)
8/26/2004 Ben Dwamena: 3rd North American Stata Users Group Meeting
20
8/26/2004 Ben Dwamena: 3rd North American Stata Users Group Meeting
21
BEGG’S BIAS PLOTBegg's funnel plot with pseudo 95% confidence limits
logOR
s.e. of: logOR0 .5 1 1.5 2
0
5
10
8/26/2004 Ben Dwamena: 3rd North American Stata Users Group Meeting
22
BEGG’S BIAS TEST
adj. Kendall's Score (P-Q) = 26Std. Dev. of Score = 40.32 Number of Studies = 24
z = 0.64Pr > |z| = 0.519
z = 0.62 (continuity corrected)Pr > |z| = 0.535 (continuity corrected)
8/26/2004 Ben Dwamena: 3rd North American Stata Users Group Meeting
23
EGGER’S REGRESSION METHOD
Assesses potential association b/n effect size and precision. Regression equation: SND = A + B x SE(d)-1. SND=standard normal deviate (effect, d divided by its standard error SE(d)); A =intercept and B=slope. .
8/26/2004 Ben Dwamena: 3rd North American Stata Users Group Meeting
24
EGGER’S REGRESSION METHOD
The intercept value (A) = estimate of asymmetry of funnel plot
Positive values (A > 0) indicate higher levels of effect size in studies with smaller sample sizes.
8/26/2004 Ben Dwamena: 3rd North American Stata Users Group Meeting
25
EGGER’S REGRESSION METHOD
The intercept value (A) = estimate of asymmetry of funnel plot
Positive values (A > 0) indicate higher levels of effect size in studies with smaller sample sizes.
8/26/2004 Ben Dwamena: 3rd North American Stata Users Group Meeting
26
EGGER’S PLOT
8/26/2004 Ben Dwamena: 3rd North American Stata Users Group Meeting
27
EGGER’S METHOD
STATA SYNTAX/COMMAND
metabias logOR selogOR, graph(e) saving(eggerplot, replace)
8/26/2004 Ben Dwamena: 3rd North American Stata Users Group Meeting
28
8/26/2004 Ben Dwamena: 3rd North American Stata Users Group Meeting
29
EGGER’S PLOT
Egger's publication bias plot
standardized effect
precision0 1 2 3 4
0
2
4
6
8
8/26/2004 Ben Dwamena: 3rd North American Stata Users Group Meeting
30
EGGER’S METHOD
-------------------------------------------------------------Std_Eff | Coef. P>|t| [95% CI]
-------------+-----------------------------------------------
slope | 1.737492 0.001 .8528166 2.622168
bias | 1.796411 0.002 .7487423 2.84408
-------------------------------------------------------------
8/26/2004 Ben Dwamena: 3rd North American Stata Users Group Meeting
31
HARBORD'S MODIFIED BIAS TEST
Test for funnel-plot asymmetry Regresses Z/sqrt(V) vs. sqrt (V), where Z is the efficient score and V is Fisher's information (the variance of Z under the null hypothesis).
Modified Galbraith plot of Z/sqrt(V) vs. sqrt(V) with the fitted regression line and a confidence interval around the intercept.
8/26/2004 Ben Dwamena: 3rd North American Stata Users Group Meeting
32
HARBORD'S MODIFIED BIAS TEST
STATA SYNTAX/COMMAND
metamodbias tp fn fp tn, graph z(Z) v(V) mlabel(index) saving(HarbordPlot, replace)
8/26/2004 Ben Dwamena: 3rd North American Stata Users Group Meeting
33
HARBORD'S MODIFIED BIAS TEST
8
18
213
137
15
64
92211
10
12
24
17
1
2
1416 19
5
23
20
05
10
15
Z/sqrt(V)
0 1 2 3 4sqrt(V)
Study regression line 90% CI for intercept
8/26/2004 Ben Dwamena: 3rd North American Stata Users Group Meeting
34
HARBORD'S MODIFIED BIAS TEST
-----------------------------------------------------------------------------
ZoversqrtV | Coef. Std. Err. P>|t| [90% Conf. Interval]
--+--------------------------------------------------------------------------
sqrtV| 2.406756 .3464027 0.000 1.811933 3.00158
bias| .9965934 .6383554 0.133 -.0995549 2.092742-----------------------------------------------------------------------------
8/26/2004 Ben Dwamena: 3rd North American Stata Users Group Meeting
35
TRIM AND FILL• A rank-based data augmentation
technique used to estimate the number of missing studies and to produce an adjusted estimate of test accuracy by imputing suspected missing studies. Both random and fixed effect models may be used to assess the impact of model choice on publication bias.
8/26/2004 Ben Dwamena: 3rd North American Stata Users Group Meeting
36
TRIM AND FILL
STATA SYNTAX/COMMAND• metatrim LogOR seLogOR,
eform funnel print graph id(author)saving(tweedieplot, replace)
8/26/2004 Ben Dwamena: 3rd North American Stata Users Group Meeting
37
8/26/2004 Ben Dwamena: 3rd North American Stata Users Group Meeting
38
TRIM AND FILL
Filled funnel plot with pseudo 95% confidence limits
th
eta,
fille
d
s.e. of: theta, filled0 .5 1 1.5 2
-2
0
2
4
6
8/26/2004 Ben Dwamena: 3rd North American Stata Users Group Meeting
39
INVESTIGATING HETEROGENEITY
Heterogeneity means that there is between-study variation.
Potential sources of heterogeneity: 1. Study population
2. Study design
3. Statistical methods,
4. Covariates adjusted for (if relevant).
8/26/2004 Ben Dwamena: 3rd North American Stata Users Group Meeting
40
GALBRAITH PLOT
Standardized effect vs. reciprocal of the standard error.
Small studies/less precise results appear on the left side and the largest trials on the right end .
8/26/2004 Ben Dwamena: 3rd North American Stata Users Group Meeting
41
GALBRAITH PLOT
A regression line , through the origin, represents the overall log-odds ratio.
Lines +/- 2 above regression line =95 per cent boundaries of the overall log-odds ratio.
The majority of within area of +/- 2 in the absence of heterogeneity.
8/26/2004 Ben Dwamena: 3rd North American Stata Users Group Meeting
42
GALBRAITH PLOT
STATA SYNTAX/COMMAND
• galbr LogOR seLogOR, id(index) yline(0) saving(gallplot, replace)
8/26/2004 Ben Dwamena: 3rd North American Stata Users Group Meeting
43
8/26/2004 Ben Dwamena: 3rd North American Stata Users Group Meeting
44
GALBRAITH PLOTb/se(b)
1/se(b)
b/se(b) Fitted values
0 3.80729
-2-2
0
2
13.1045
.
7 8131511
18321
1222
1419
17
69
2
24
1
4
10
23
16
5
20
8/26/2004 Ben Dwamena: 3rd North American Stata Users Group Meeting
45
L’ABBE PLOT
This plots the event rate in the experimental (intervention) group against the event rate in the control group
An aid to exploring the heterogeneity of effect estimates within a meta-analysis.
8/26/2004 Ben Dwamena: 3rd North American Stata Users Group Meeting
46
L’ABBE PLOT
STATA SYNTAX/COMMAND
labbe tp fn fp tn, s(O) xlab(0,0.25,0.50,0.75,1) ylab(0,0.25,0.50,0.75,1) l1("TPR) b2("FPR") saving(flabbeplot, replace)
8/26/2004 Ben Dwamena: 3rd North American Stata Users Group Meeting
47
8/26/2004 Ben Dwamena: 3rd North American Stata Users Group Meeting
48
L’ABBE PLOTTPR
FPR0 .25 .5 .75 1
0
.25
.5
.75
1
8/26/2004 Ben Dwamena: 3rd North American Stata Users Group Meeting
49
DATA SUMMARYSTATA 8 SYNTAX
twoway (rcap dorlo dorhi Study, horizontal blpattern(dash))(scatter Study dor, ms(O)msize(medium) mcolor(black))(scatter DOR_with_CIs eb_dor, yaxis(2) msymbol(i) msize(large) mcolor(black))(scatteri 26 83, msymbol(diamond) msize(large)), ylabel(1(1)25 26 "OVERALL", valuelabels angle(horizontal)) xlabel(0 10 100 1000 10000) xscale(log) ylabel(1(1)25 26 "Pooled Estimate", valuelabels angle(horizontal) axis(2)) legend(off) xtitle(Odds Ratio) xline(83, lstyle(foreground)) saving(OddsForest, replace)
8/26/2004 Ben Dwamena: 3rd North American Stata Users Group Meeting
50
DATA SUMMARY
• metan tp fn fp tn, or fixed nowt sortby(year) label(namevar=author, yearvar=year) t1(Summary DOR, Fixed Effects) b2(Diagnostic Odds Ratio) saving(SDORFE, replace) force xlabel(0,1,10,100,1000)
• metan tp fn fp tn, or random nowt sortby(year) label(namevar=author, yearvar=year) t1(Summary DOR, Random Effects) b2(Diagnostic Odds Ratio) saving(SDORRE, replace) force xlabel(0,1,10,100,1000)
8/26/2004 Ben Dwamena: 3rd North American Stata Users Group Meeting
51
STATA 8 FOREST PLOT
1 0 6 .3 3 ( 6 .4 0 - 1 7 6 5 .7 51 0 7 .2 3 ( 4 0 . 3 1 - 2 8 5 .2 81 0 7 .6 7 ( 6 .5 7 - 1 7 6 3 .9 91 2 .3 3 ( 2 .0 4 - 7 4 .4 1 )1 7 .5 0 ( 2 .6 5 - 1 1 5 .6 6 )1 7 5 .0 0 ( 2 8 . 2 2 - 1 0 8 5 .21 8 9 .0 0 ( 1 1 . 3 6 - 3 1 4 3 .31 9 1 .6 7 ( 1 2 . 0 2 - 3 0 5 5 .92 1 .0 0 ( 1 .4 3 - 3 0 8 .2 1 )2 4 0 .8 8 ( 6 8 . 7 5 - 8 4 3 .9 62 5 .0 0 ( 1 .7 2 - 3 6 4 .1 1 )2 6 2 .6 6 ( 2 4 . 3 9 - 2 8 2 9 .12 6 6 .0 0 ( 3 3 . 4 6 - 2 1 1 4 .73 2 .8 2 ( 5 .4 3 - 1 9 8 .3 7 )3 3 .0 0 ( 8 .8 3 - 1 2 3 .2 6 )4 .3 3 ( 1 .0 5 - 1 7 .9 0 )4 2 9 .0 0 ( 1 4 . 6 5 - 1 2 5 6 2 .4 5 .0 0 ( 3 .7 5 - 5 3 9 .3 8 )4 6 .9 3 ( 1 7 .4 8 - 1 2 5 .9 6 )5 6 .0 8 ( 4 .6 2 - 6 8 0 .3 3 )6 .1 0 ( 3 .9 5 - 9 .4 2 )6 .8 6 ( 1 .8 1 - 2 6 .0 1 )7 .0 9 ( 1 .0 6 - 4 7 .4 2 )9 .0 0 ( 0 .5 8 - 1 4 0 .7 1 )9 8 .8 0 ( 1 5 .2 4 - 6 4 0 .5 0 )P o o le d E s t im a t e
DO
R_w
ith_C
Is
A d le r 1 9 9 7A v r il 1 9 9 6
B a s s a 1 9 9 6D a n fo rth 2 0 0 2
G re c o 2 0 0 1G u lle r 2 0 0 2
H u b n e r 2 0 0 0In o u e 2 0 0 4
L in 2 0 0 2N a k a m o to (a ) 2 0 0 2N a k a m o to (b ) 2 0 0 2
N o h 1 9 9 8O h ta 2 0 0 0
P a lm e d o 1 9 9 7R o s to m 1 9 9 9
S c h e id h a u e r 1 9 9 6S c h ir r m e is te r 2 0 0 1
S m ith 1 9 9 8T s e 1 9 9 2
U te c h 1 9 9 6W a h l 2 0 0 4Y a n g 2 0 0 1
Y u ta n i 2 0 0 1Z o r n o z a 2 0 0 4
v a n H o e v e n 2 0 0 2O V E R A L L
0 1 0 1 0 0 1 0 0 0 1 0 0 0 0O d d s R a t io
8/26/2004 Ben Dwamena: 3rd North American Stata Users Group Meeting
52
FIXED EFFECTS META-ANALYSIS
Assumes homogeneity of effects across the studies being combined the true effect size has a common true value for all studies.In the summary estimate only the variance of each study is taken into account.
8/26/2004 Ben Dwamena: 3rd North American Stata Users Group Meeting
53
FIXED EFFECTS FOREST PLOT
Summary DOR, Fixed Effects
Odds ratio.1 1 100 1000 10000
Study
Odds ratio (95% CI)
36.27 (4.27,308.02) Adler (1997)
98.80 (10.66,916.11) Avril (1996)
21.00 (0.86,515.50) Bassa (1996)
4.33 (0.80,23.49) Danforth (2002)
107.23 (33.42,344.07) Greco (2001)
12.00 (1.23,117.41) Guller (2002)
429.00 (7.67,23982.81) Hubner (2000)
189.00 (6.63,5384.60) Lin (2002)
17.50 (1.84,166.04) Nakamoto (2002)
6.86 (1.40,33.57) Nakamoto (2002)
208.33 (7.72,5621.57) Noh (1998)
60.23 (3.09,1174.51) Ohta (2000)
106.33 (3.74,3023.90) Palmedo (1997)
289.00 (15.83,5276.04) Rostom (1999)
107.67 (3.85,3013.13) Scheidhauer (1996)
46.93 (14.47,152.17) Schirrmeister (2001)
266.00 (22.50,3145.19) Smith (1998)
9.00 (0.34,238.21) Tse (1992)
262.66 (15.47,4459.70) Utech (1996)
6.10 (3.64,10.24) Wahl (2004)
25.00 (1.03,608.09) Yang (2001)
45.00 (2.33,867.81) Yutani (2001)
240.88 (54.08,1072.94) Zornoza (2004)
12.33 (1.45,104.97) van Hoeven (2002)
19.85 (14.16,27.82) Overall (95% CI)
8/26/2004 Ben Dwamena: 3rd North American Stata Users Group Meeting
54
RANDOM EFFECTS META-ANALYSIS
Heterogeneity is incorporated into the pooled estimate by including a between study component of variance. Assumes sample of studies included in the
analysis is drawn from a population of studies. Each sample of studies has a true effect
size.
8/26/2004 Ben Dwamena: 3rd North American Stata Users Group Meeting
55
RANDOM EFFECTS FOREST PLOT
Summary DOR, Random Effects
Odds ratio.1 1 100 1000 10000
Study
Odds ratio (95% CI)
36.27 (4.27,308.02) Adler (1997)
98.80 (10.66,916.11) Avril (1996)
21.00 (0.86,515.50) Bassa (1996)
4.33 (0.80,23.49) Danforth (2002)
107.23 (33.42,344.07) Greco (2001)
12.00 (1.23,117.41) Guller (2002)
429.00 (7.67,23982.81) Hubner (2000)
189.00 (6.63,5384.60) Lin (2002)
17.50 (1.84,166.04) Nakamoto (2002)
6.86 (1.40,33.57) Nakamoto (2002)
208.33 (7.72,5621.57) Noh (1998)
60.23 (3.09,1174.51) Ohta (2000)
106.33 (3.74,3023.90) Palmedo (1997)
289.00 (15.83,5276.04) Rostom (1999)
107.67 (3.85,3013.13) Scheidhauer (1996)
46.93 (14.47,152.17) Schirrmeister (2001)
266.00 (22.50,3145.19) Smith (1998)
9.00 (0.34,238.21) Tse (1992)
262.66 (15.47,4459.70) Utech (1996)
6.10 (3.64,10.24) Wahl (2004)
25.00 (1.03,608.09) Yang (2001)
45.00 (2.33,867.81) Yutani (2001)
240.88 (54.08,1072.94) Zornoza (2004)
12.33 (1.45,104.97) van Hoeven (2002)
42.54 (20.88,86.68) Overall (95% CI)
8/26/2004 Ben Dwamena: 3rd North American Stata Users Group Meeting
56
CUMULATIVE META-ANALYSIS
studies are sequentially pooled by adding each time one new study according to an ordered variable. For instance, the year of publication; then, a pooling analysis will be done every time a new article appears.
8/26/2004 Ben Dwamena: 3rd North American Stata Users Group Meeting
57
CUMULATIVE META-ANALYSIS
metacum LogOR seLogOR, eform id(author) effect(f) graph cline saving(year_fcummplot, replace)
8/26/2004 Ben Dwamena: 3rd North American Stata Users Group Meeting
58
8/26/2004 Ben Dwamena: 3rd North American Stata Users Group Meeting
59
8/26/2004 Ben Dwamena: 3rd North American Stata Users Group Meeting
60
CUMMULATIVE META-ANALYSIS
L og O R7.6 73 8 7 2 39 82 .8
W a hl 20 04
Y an g 20 01
Y ut a ni 2 001
Z orn oza 2 004
I no ue 20 04
Na kam o t o(b) 2 002
Na kam o t o(a) 2 002
van Hoe v en 2 00 2
A dl er 19 97
Ro s t om 19 99
A vri l 199 6
G ul ler 20 02
S ch irrm e is t er 20 01
S m i th 1 99 8
Ute c h 19 96
G rec o 2 00 1
O hta 20 00
Hu bn er 20 00
B as s a 19 96
T se 19 92
No h 19 98
S ch eid ha uer 1 996
P al m ed o 19 97
Li n 20 02
Da nf ort h 2 002
8/26/2004 Ben Dwamena: 3rd North American Stata Users Group Meeting
61
INFLUENCE ANALYSIS
studies are pooled according influence of a trial on overall effect defined as the difference between the effect estimated with and without the trial
8/26/2004 Ben Dwamena: 3rd North American Stata Users Group Meeting
62
INFLUENCE ANALYSIS
metaninf tp fn fp tn, id(author) saving(influplot, replace) save(infcoeff, replace)
8/26/2004 Ben Dwamena: 3rd North American Stata Users Group Meeting
63
8/26/2004 Ben Dwamena: 3rd North American Stata Users Group Meeting
64
8/26/2004 Ben Dwamena: 3rd North American Stata Users Group Meeting
65
INFLUENCE ANALYSIS
4.75 6.53 5.41 7.88 10.30
1 2 3 4 5 6 7 8 9
10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25
Lower CI Limit Est im ate Upper CI Limit Meta-analysis estimates , given named study is omitted
8/26/2004 Ben Dwamena: 3rd North American Stata Users Group Meeting
66
8/26/2004 Ben Dwamena: 3rd North American Stata Users Group Meeting
67
ROC PLOT
A scatter plot true positive fraction (sensitivity) vs. false positive fraction (1-specificity)
aids in visualization of range of results from primary studies
8/26/2004 Ben Dwamena: 3rd North American Stata Users Group Meeting
68
ROC PLOT
twoway (scatter TPF FPF, sort ) (lfit uTPR FPF, sort range(0 1) clcolor(black) clpat(dash) clwidth(vthin) connect(direct)) (lfit sTPR FPF, sort range(0 1) clcolor(black) clpat(dot) clwidth(vthin) connect(direct)), ytitle(Sensitivity) ylabel(0(.1)1, grid) xtitle(1-Specificity) xlabel(0(.1)1, grid) title(ROC Plot of SENSITIVITY vs. 1-SPECIFICITY, size(medium)) legend(pos(3) col(1) lab(1 "Observed Data") lab(2 "Uninformative Test") lab(3 "Symmetry Line")) saving(ROCplot, replace) plotregion(margin(zero))
8/26/2004 Ben Dwamena: 3rd North American Stata Users Group Meeting
69
ROC PLOT0
.1.2
.3.4
.5.6
.7.8
.91
Sen
sitiv
ity
0 .1 .2 .3 .4 .5 .6 .7 .8 .9 11 -S p e c if ic ity
O bs erv e d D a taU ni n fo rm a t iv e T e s tS y m m e tr y L in e
R O C P lo t o f S E N S IT IV ITY v s . 1 -S P E C IFIC IT Y
8/26/2004 Ben Dwamena: 3rd North American Stata Users Group Meeting
70
LINEAR REGRESSION MODELS
ORDINARY LEAST SQUARES METHOD:Studies are weighted equallyWEIGHTED LEAST SQUARES METHOD:Weighted by the inverse variance weights of the
odds ratio, or simply the sample sizeROBUST-RESISTANT METHOD:
Minimizes the influence of outliers
8/26/2004 Ben Dwamena: 3rd North American Stata Users Group Meeting
71
REGRESSION ANALYSIS
Logit transformations of the TP rate (sensitivity) and FP rate (1 - specificity).
D=ln(DOR) =logit(TPR) – logit(FPR)Differences in logit transformations, D, regressed on sums
of logit transformations, S.
S=logit(TPR)+logit(FPR)Logit(TPR)=natural log odds of a TP result and logit(FPR) =natural log of the odds of a FP test result.
8/26/2004 Ben Dwamena: 3rd North American Stata Users Group Meeting
72
ACCURACY-THRESHOLD
STATA SYNTAX/COMMAND• twoway (scatter D S, sort msymbol(circle)) (lfit
tfitted S, clcolor(black) clpat(solid) clwidth(thin) connect(direct))(lfit wfitted S, clcolor(black) clpat(dash) clwidth(thin) connect(direct)), ytitle(Discriminatory Power/D) xtitle(Diagnostic Threshold/S) title(REGRESSION PLOT) legend(lab(1 "Observed Data")lab(2 "EWLSR")lab(3 "VWLSR"))saving(regplot, replace) xline(0) yscale(noline)
8/26/2004 Ben Dwamena: 3rd North American Stata Users Group Meeting
73
ACCURACY-THRESHOLD1
23
45
6D
iscr
imin
ator
y P
ower
/D
-4 -2 0 2 4Diagnostic Threshold/S
Observed Data EWLSRVWLSR
REGRESSION PLOT
8/26/2004 Ben Dwamena: 3rd North American Stata Users Group Meeting
74
SUMMARY ROC CURVE
Back transformation of logistic regression to conventional axes of sensitivity [TPR] vs. (1 – specificity) [FPR]) with the equation TPR = 1/{1 + exp[- a/(1 - b )]} [(1 -FPR)/(FPR)](1 + b )/(1 - b ). Slope (b) and intercept (a) are obtained from the linear regression analyses
8/26/2004 Ben Dwamena: 3rd North American Stata Users Group Meeting
75
SUMMARY ROC CURVES
• STATA SYNTAX/COMMAND• twoway (scatter TPF FPF, sort msymbol(circle) msize(medium)
mcolor(black))(fpfit tTPR FPF, clpat(dash)clwidth(medium) connect(direct ))(fpfit wTPR FPF, clpat(solid)clwidth(medium) connect(direct ))(lfit uTPR FPF, sort range(0 1) clcolor(black) clpat(dash) clwidth(thin) connect(direct)) (lfit sTPR FPF, sort range(0 1) clcolor(black) clpat(dot) clwidth(medium) connect(direct)), ytitle(Sensitivity/TPF) yscale(range(0 1)) ylabel( 0(.2)1,grid ) xtitle(1-Specificity/FPF) xscale(range(0 1)) xlabel(0(.2)1, grid) legend(lab(1 "Observed Data")lab(2 "EWLSR")lab(3 "VWLSR")lab(4 "RRLSR")lab(5 "Uninformative Test") lab(6 "Symmetry Line") pos(3) col(1)) title(SUMMARY ROC CURVES) graphregion(margin(zero)) saving(aSROCplot, replace)
8/26/2004 Ben Dwamena: 3rd North American Stata Users Group Meeting
76
SUMMARY ROC CURVES0
.2.4
.6.8
1S
ensi
tivity
/TP
F
0 .2 .4 .6 .8 11-Specificity/FPF
Observed DataEWLSRVWLSRRRLSRUninformative Test
SUMMARY ROC CURVES