Vol. 2, NoI' 1, January 1995 G E N E R A L I Z I N G T H E R O C A N A L Y S I S

ples is conducted, representativeness of our samples cannot be assumed. Thus, although we have identified some problems, there is no way to determine the rela- tive frequency with which these problems will occur.

A C K N O W L E D G M E N T

This work is supported in part by grant numbers CA58283 and CA60259 from the National Cancer Institute.

REFERENCES

1. Swets JA, Pickett RM. Evaluation of diagnostic systems: methods from signal detection theo~ New York: Academic Press, 1982:46-93.

2. Turner DA. An intuitive approach to receiver operating characteristic curve analysis. J Nucl Med 1978; 19:213-220.

3. Hanley JA, McNeil BJ. A method of comparing the areas under receiver operating characteristic curves derived from the same cases. Radiology 1983; 148:839-843.

4. Metz CE. Some practical issues of experimental design and data analysis in radiological ROC studies. Invest Radio11989; 24:234-245.

5. Chakraborty DP. Maximum likelihood analysis of free-response receiver operating characteristic analyses. Med Phys 1989;16:561-568.

6. Rockette HE, Gur D, Cooperstein LA, et al. Effect of two rating formats in multi-disease ROC study of chest images. Invest Radio11999;25:225-229.

7. Agresti A. A model for agreement between ratings on an ordinal scale. Biometrics 1988;44:539-548.

8. Wieand S, Gail MH, James BR, James KL. A family of nonparametfic sta- tistics for comparing diagnostic markers with paired or unpaired data. Biometrika 1989;76:585-592.

9. Lams PM, Cocklin ME Spatial resolution requirements for digital chest radiographs: An ROC study of observer performance in selected cases. Radiology 1986;158:11-19.

10. MacMahon, Vyborny C J, Metz CE, et al. Digital radiography of subtle pul- monary abnormalities: An ROC study of the effect of pixel size on observer performance. Radiology1986;158:21-26.

11. Slasky BS, Gur D, Good WF, et al. ROC analysis of chest image interpre- tation on conventional radiographs, laser-printed radiographs, and high- resolution workstation. Radiology 1990; 174:775-780.

12. Cox GG, Cook LT, McMillan JH, et al. Chest radiography: comparison of high-resolution digital displays with conventional and digital film. Radiology 1990; 176:771-776.

13. Dorfman DD, Berbaum KS, Metz CE. Receiver operating characteristic rating analysis: Generalization to the population of readers and patients with the jackknife method. Invest Radio11992;27:723-731.

14. Metz CE, Shen J-H, Herman BA. Newmethods for estimating a binormal ROC curve from continuously-distributed test results. Paper presented at the 1990 Joint Statistical Meetings of the American Statistical Society and the Biometric Society; August 1990; Anaheim, CA.

15. Tukey JW. Bias and confidence in not-quite large samples. Ann Math Statist 1958;29:614. Abstract.

16. Efron B, Tibshirani RJ. An Introduction to the Bootstrap. Chapman & Hall: London, 1993:145.

17. Arvesen JN. Jackknifing U-statistics. Ann Math Statist1969;40:2076-2100. 18. Arvesen JN, Schmitz TH. Robust procedures for variance component

problems using the jackknife. Biometrics 1970;26:677-686. 19. McClish DK. Comparing the areas under more than two independent

ROC curves. Med Decis Making 1987;7:149-155.

A n n o u n c e m e n t

The UCLA General Radiology Review Course will be held March 5-10, 1995, at the Guest Quarters Suite Hotel in Santa Monica, CA. Forty hours of Category 1 credit will be awarded (pending). The cost is $420. For more informa- tion, contact Darry Bailey, UCLA Medical Center, Department of Radiological Sciences, 10833 Le Conte Ave., B2-170 CHS, Los Angeles, CA 90024-1721; (310) 825-8763; fax (310) 206-5725.

69