1. NonparametricStatistical InferenceFourth Edition, Revisedand ExpandedJean Dickinson GibbonsSubhabrata ChakrabortiThe University of AlabamaTuscaloosa, Alabama, U.S.A.M A R C E LMARCEL DEKKER, INC. NEW YORK BASELDE K K E R

2. Library of Congress Cataloging-in-Publication DataA catalog record for this book is available from the Library of Congress.ISBN: 0-8247-4052-1This book is printed on acid-free paper.HeadquartersMarcel Dekker, Inc.270 Madison Avenue, New York, NY 10016tel: 212-696-9000; fax: 212-685-4540Eastern Hemisphere DistributionMarcel Dekker AGHutgasse 4, Postfach 812, CH-4001 Basel, Switzerlandtel: 41-61-260-6300; fax: 41-61-260-6333World Wide Webhttp:==www.dekker.comThe publisher offers discounts on this book when ordered in bulk quantities. Formore information, write to Special Sales=Professional Marketing at the headquartersaddress above.Copyright # 2003 by Marcel Dekker, Inc. All Rights Reserved.Neither this book nor any part may be reproduced or transmitted in any form or byany means, electronic or mechanical, including photocopying, microfilming, andrecording, or by any information storage and retrieval system, without permission inwriting from the publisher.Current printing (last digit):10 9 8 7 6 5 4 3 2 1PRINTED IN THE UNITED STATES OF AMERICA 3. STATISTICS: Textbooks and MonographsD. B. OwenFounding Editor, 1972-1991Associate EditorsStatistical Computing/Nonparametric StatisticsProfessor William R. SchucanySouthern Methodist UniversityProbabilityProfessor Marcel F. NeutsUniversity of ArizonaMultivariate AnalysisProfessor Anant M KshirsagarUniversity of MichiganQuality Control/ReliabilityProfessor Edward G. SchillingRochester Institute of TechnologyEditorial BoardApplied ProbabilityDr. Paul R. GarveyThe MITRE CorporationEconomic StatisticsProfessor David E A. GilesUniversity of VictoriaExperimental DesignsMr Thomas B. BarkerRochester Institute of TechnologyMultivariate AnalysisProfessor Subir GhoshUniversity of California-RiversideStatistical DistributionsProfessor N. BalaknshnanMcMaster UniversityStatistical Process ImprovementProfessor G. Geoffrey VmingVirginia Polytechnic InstituteStochastic ProcessesProfessor V. LakshrmkanthamFlorida Institute of TechnologySurvey SamplingProfessor Lynne StokesSouthern Methodist UniversityTime SeriesSastry G. PantulaNorth Carolina State University 4. 1 The Generalized Jackknife Statistic, H. L. Gray and W. R Schucany2. Multivariate Analysis, Anant M. Kshirsagar3 Statistics and Society, Walter T. Federer4. Multivanate Analysis. A Selected and Abstracted Bibliography, 1957-1972, Kocher-lakotaSubrahmamam and Kathleen Subrahmamam5 Design of Expenments: A Realistic Approach, Virgil L. Anderson and Robert A.McLean6. Statistical and Mathematical Aspects of Pollution Problems, John W Pratt7. Introduction to Probability and Statistics (in two parts), Part I: Probability; Part II:Statistics, Narayan C. Giri8. Statistical Theory of the Analysis of Experimental Designs, J Ogawa9 Statistical Techniques in Simulation (in two parts), Jack P. C. Kleijnen10. Data Quality Control and Editing, Joseph I Naus11 Cost of Living Index Numbers: Practice, Precision, and Theory, Kali S Banerjee12 Weighing Designs: For Chemistry, Medicine, Economics, Operations Research,Statistics, Kali S. Banerjee13. The Search for Oil: Some Statistical Methods and Techniques, edited by D B. Owen14. Sample Size Choice: Charts for Expenments with Linear Models, Robert E Odeh andMartin Fox15. Statistical Methods for Engineers and Scientists, Robert M. Bethea, Benjamin SDuran, and Thomas L Boullion16 Statistical Quality Control Methods, living W Burr17. On the History of Statistics and Probability, edited by D. B. Owen18 Econometrics, Peter Schmidt19. Sufficient Statistics' Selected Contributions, VasantS. Huzurbazar (edited by Anant MKshirsagar)20. Handbook of Statistical Distributions, Jagdish K. Pate/, C H Kapadia, and D B Owen21. Case Studies in Sample Design, A. C Rosander22. Pocket Book of Statistical Tables, compiled by R. E Odeh, D B. Owen, Z. W.Bimbaum, and L Fisher23. The Information in Contingency Tables, D V Gokhale and Solomon Kullback24. Statistical Analysis of Reliability and Life-Testing Models: Theory and Methods, Lee J.Bain25. Elementary Statistical Quality Control, Irving W Burr26. An Introduction to Probability and Statistics Using BASIC, Richard A. Groeneveld27. Basic Applied Statistics, B. L Raktoe andJ J Hubert28 A Primer in Probability, Kathleen Subrahmamam29. Random Processes: A First Look, R. Syski30. Regression Methods: A Tool for Data Analysis, Rudolf J. Freund and Paul D. Minton31. Randomization Tests, Eugene S. Edgington32 Tables for Normal Tolerance Limits, Sampling Plans and Screening, Robert E. OdehandD B Owen33. Statistical Computing, William J Kennedy, Jr., and James E. Gentle34. Regression Analysis and Its Application: A Data-Onented Approach, Richard F. Gunstand Robert L Mason35. Scientific Strategies to Save Your Life, / D. J. Brass36. Statistics in the Pharmaceutical Industry, edited by C. Ralph Buncher and Jia-YeongTsay37. Sampling from a Finite Population, J. Hajek38. Statistical Modeling Techniques, S. S. Shapiro and A J. Gross39. Statistical Theory and Inference in Research, T A. Bancroft and C -P. Han40. Handbook of the Normal Distribution, Jagdish K. Pate/ and Campbell B Read41 Recent Advances in Regression Methods, Hrishikesh D. Vinod and Aman Ullah42 Acceptance Sampling in Quality Control, Edward G. Schilling43. The Randomized Clinical Trial and Therapeutic Decisions, edited by Niels Tygstrup,John M Lachin, and Enk Juhl 5. 44. Regression Analysis of Survival Data in Cancer Chemotherapy, Walter H Carter, Jr,Galen L Wampler, and Donald M Stablein45. A Course in Linear Models, Anant M Kshirsagar46. Clinical Trials- Issues and Approaches, edited by Stanley H Shapiro and Thomas HLouis47. Statistical Analysis of DNA Sequence Data, edited by B S Weir48. Nonlinear Regression Modeling: A Unified Practical Approach, David A Ratkowsky49. Attribute Sampling Plans, Tables of Tests and Confidence Limits for Proportions, Rob-ertE OdehandD B Owen50. Experimental Design, Statistical Models, and Genetic Statistics, edited by KlausHinkelmann51. Statistical Methods for Cancer Studies, edited by Richard G Cornell52. Practical Statistical Sampling for Auditors, Arthur J. Wilbum53 Statistical Methods for Cancer Studies, edited by Edward J Wegman and James GSmith54 Self-Organizing Methods in Modeling GMDH Type Algorithms, edited by Stanley J.Farlow55 Applied Factonal and Fractional Designs, Robert A McLean and Virgil L Anderson56 Design of Experiments Ranking and Selection, edited by Thomas J Santner and AjitC Tamhane57 Statistical Methods for Engineers and Scientists- Second Edition, Revised and Ex-panded,Robert M. Bethea, Benjamin S Duran, and Thomas L Bouillon58 Ensemble Modeling. Inference from Small-Scale Properties to Large-Scale Systems,Alan E Gelfand and Crayton C Walker59 Computer Modeling for Business and Industry, Bruce L Bowerman and Richard TO'Connell60 Bayesian Analysis of Linear Models, Lyle D Broemeling61. Methodological Issues for Health Care Surveys, Brenda Cox and Steven Cohen62 Applied Regression Analysis and Expenmental Design, Richard J Brook and GregoryC Arnold63 Statpal: A Statistical Package for MicrocomputersPC-DOS Version for the IBM PCand Compatibles, Bruce J Chalmer and David G Whitmore64. Statpal: A Statistical Package for MicrocomputersApple Version for the II, II+, andHe, David G Whitmore and Bruce J. Chalmer65. Nonparametnc Statistical Inference. Second Edition, Revised and Expanded, JeanDickinson Gibbons66 Design and Analysis of Experiments, Roger G Petersen67. Statistical Methods for Pharmaceutical Research Planning, Sten W Bergman andJohn C Gittins68. Goodness-of-Fit Techniques, edited by Ralph B D'Agostino and Michael A. Stephens69. Statistical Methods in Discnmination Litigation, ecMed by D H Kaye and MikelAickin70. Truncated and Censored Samples from Normal Populations, Helmut Schneider71. Robust Inference, M L Tiku, W Y Tan, and N Balakrishnan72. Statistical Image Processing and Graphics, edited by Edward J, Wegman and DouglasJ DePnest73. Assignment Methods in Combmatonal Data Analysis, Lawrence J Hubert74. Econometrics and Structural Change, Lyle D Broemeling and Hiroki Tsurumi75. Multivanate Interpretation of Clinical Laboratory Data, Adelin Albert and Eugene KHam's76. Statistical Tools for Simulation Practitioners, Jack P C Kleijnen77. Randomization Tests' Second Editon, Eugene S Edgington78 A Folio of Distributions A Collection of Theoretical Quantile-Quantile Plots, Edward BFowlkes79. Applied Categorical Data Analysis, Daniel H Freeman, Jr80. Seemingly Unrelated Regression Equations Models: Estimation and Inference, Viren-draK Snvastava and David E A Giles 6. 81. Response Surfaces: Designs and Analyses, Andre I. Khun and John A. Cornell82. Nonlinear Parameter Estimation: An Integrated System in BASIC, John C. Nash andMary Walker-Smith83. Cancer Modeling, edited by James R. Thompson and Barry W. Brown84. Mixture Models: Inference and Applications to Clustering, Geoffrey J. McLachlan andKaye E. Basford85. Randomized Response. Theory and Techniques, Anjit Chaudhuri and Rahul Mukerjee86 Biopharmaceutical Statistics for Drug Development, edited by Karl E. Peace87. Parts per Million Values for Estimating Quality Levels, Robert E Odeh and D B. Owen88. Lognormal Distnbutions: Theory and Applications, ecWed by Edwin L Crow and KunioShimizu89. Properties of Estimators for the Gamma Distribution, K O. Bowman and L R. Shenton90. Spline Smoothing and Nonparametnc Regression, Randall L Eubank91. Linear Least Squares Computations, R W Farebrother92. Exploring Statistics, Damaraju Raghavarao93. Applied Time Series Analysis for Business and Economic Forecasting, Sufi M Nazem94 Bayesian Analysis of Time Series and Dynamic Models, edited by James C. Spall95. The Inverse Gaussian Distribution: Theory, Methodology, and Applications, Raj SChhikara andJ. Leroy Folks96. Parameter Estimation in Reliability and Life Span Models, A. Clifford Cohen and BettyJones Whrtten97. Pooled Cross-Sectional and Time Series Data Analysis, Terry E Die/man98. Random Processes: A First Look, Second Edition, Revised and Expanded, R. Syski99. Generalized Poisson Distributions: Properties and Applications, P C. Consul100. Nonlinear Lp-Norm Estimation, Rene Gonin and Arthur H Money101. Model Discrimination for Nonlinear Regression Models, Dale S. Borowiak102. Applied Regression Analysis in Econometrics, Howard E. Doran103. Continued Fractions in Statistical Applications, K. O. Bowman andL R. Shenton104 Statistical Methodology in the Pharmaceutical Sciences, Donald A. Berry105. Expenmental Design in Biotechnology, Perry D. Haaland106. Statistical Issues in Drug Research and Development, edited by Karl E Peace107. Handbook of Nonlinear Regression Models, David A. Ratkowsky108. Robust Regression: Analysis and Applications, edited by Kenneth D Lawrence andJeffrey L Arthur109. Statistical Design and Analysis of Industrial Experiments, edited by Subir Ghosh110. (7-Statistics: Theory and Practice, A J Lee111. A Primer in Probability: Second Edition, Revised and Expanded, Kathleen Subrah-maniam112. Data Quality Control: Theory and Pragmatics, edited by GunarE Uepins and V. R R.Uppuluri113. Engmeenng Quality by Design: Interpreting the Taguchi Approach, Thomas B Barker114 Survivorship Analysis for Clinical Studies, Eugene K. Hams and Adelin Albert115. Statistical Analysis of Reliability and Life-Testing Models: Second Edition, Lee J. Bamand Max Engelhardt116. Stochastic Models of Carcinogenesis, Wai-Yuan Tan117. Statistics and Society Data Collection and Interpretation, Second Edition, Revised andExpanded, Walter T. Federer118. Handbook of Sequential Analysis, B K. Gfiosn and P. K. Sen119 Truncated and Censored Samples: Theory and Applications, A. Clifford Cohen120. Survey Sampling Pnnciples, E. K. Foreman121. Applied Engineering Statistics, Robert M. Bethea and R. Russell Rhinehart122. Sample Size Choice: Charts for Experiments with Linear Models: Second Edition,Robert Odeh and Martin Fox123. Handbook of the Logistic Distnbution, edited by N Balaknshnan124. Fundamentals of Biostatistical Inference, Chap T. Le125. Correspondence Analysis Handbook, J.-P Benzecn 7. 126. Quadratic Forms in Random Variables: Theory and Applications, A. M Mathai andSerge B Provost127 Confidence Intervals on Vanance Components, Richard K. Burdick and Franklin AGraybill128 Biopharmaceutical Sequential Statistical Applications, edited by Karl E Peace129. Item Response Theory Parameter Estimation Techniques, Frank B. Baker130. Survey Sampling Theory and Methods, Arijrt Chaudhun and Horst Stenger131. Nonparametnc Statistical Inference Third Edition, Revised and Expanded, Jean Dick-insonGibbons and Subhabrata Chakraborti132 Bivanate Discrete Distribution, Subrahmaniam Kochertakota and Kathleen Kocher-lakota133. Design and Analysis of Bioavailability and Bioequivalence Studies, Shein-Chung Chowand Jen-pei Liu134. Multiple Compansons, Selection, and Applications in Biometry, edited by Fred MHoppe135. Cross-Over Expenments: Design, Analysis, and Application, David A Ratkowsky,Marc A Evans, and J. Richard Alldredge136 Introduction to Probability and Statistics- Second Edition, Revised and Expanded,Narayan C Gin137. Applied Analysis of Vanance in Behavioral Science, edited by Lynne K Edwards138 Drug Safety Assessment in Clinical Trials, edited by Gene S Gilbert139. Design of Expenments A No-Name Approach, Thomas J Lorenzen and Virgil L An-derson140 Statistics in the Pharmaceutical Industry. Second Edition, Revised and Expanded,edited by C Ralph Buncher and Jia-Yeong Tsay141 Advanced Linear Models Theory and Applications, Song-Gui Wang and Shein-ChungChow142. Multistage Selection and Ranking Procedures. Second-Order Asymptotics, Nitis Muk-hopadhyayand Tumulesh K S Solanky143. Statistical Design and Analysis in Pharmaceutical Science Validation, Process Con-trols,and Stability, Shein-Chung Chow and Jen-pei Liu144 Statistical Methods for Engineers and Scientists Third Edition, Revised and Expanded,Robert M Bethea, Benjamin S Duran, and Thomas L Bouillon145 Growth Curves, Anant M Kshirsagar and William Boyce Smith146 Statistical Bases of Reference Values in Laboratory Medicine, Eugene K. Harris andJames C Boyd147 Randomization Tests- Third Edition, Revised and Expanded, Eugene S Edgington148 Practical Sampling Techniques Second Edition, Revised and Expanded, Ran/an K.Som149 Multivanate Statistical Analysis, Narayan C Gin150 Handbook of the Normal Distribution Second Edition, Revised and Expanded, JagdishK Patel and Campbell B Read151 Bayesian Biostatistics, edited by Donald A Berry and Dalene K Stangl152 Response Surfaces: Designs and Analyses, Second Edition, Revised and Expanded,Andre I Khuri and John A Cornell153 Statistics of Quality, edited by Subir Ghosh, William R Schucany, and William B. Smith154. Linear and Nonlinear Models for the Analysis of Repeated Measurements, Edward FVonesh and Vemon M Chinchilli155 Handbook of Applied Economic Statistics, Aman Ullah and David E A Giles156 Improving Efficiency by Shnnkage The James-Stein and Ridge Regression Estima-tors,Marvin H J Gruber157 Nonparametnc Regression and Spline Smoothing Second Edition, Randall L Eu-bank158 Asymptotics, Nonparametncs, and Time Senes, edited by Subir Ghosh159 Multivanate Analysis, Design of Experiments, and Survey Sampling, edited by SubirGhosh 8. 160 Statistical Process Monitoring and Control, edited by Sung H Park and G GeoffreyVining161 Statistics for the 21st Century Methodologies for Applications of the Future, editedby C. R Rao and GaborJ Szekely162 Probability and Statistical Inference, Nitis Mukhopadhyay163 Handbook of Stochastic Analysis and Applications, edited by D Kannan and V. Lak-shmtkantham164. Testing for Normality, Henry C Thode, Jr.165 Handbook of Applied Econometncs and Statistical Inference, edited by Aman Ullah,Alan T K. Wan, andAnoop Chaturvedi166 Visualizing Statistical Models and Concepts, R W Farebrother167. Financial and Actuarial Statistics An Introduction, Dale S Borowiak168 Nonparametnc Statistical Inference Fourth Edition, Revised and Expanded, JeanDickinson Gibbons and Subhabrata Chakraborti169. Computer-Aided Econometncs, edited by David E. A GilesAdditional Volumes in PreparationThe EM Algorithm and Related Statistical Models, edited by Michiko Watanabe andKazunori YamaguchiMultivanate Statistical Analysis, Narayan C Giri 9. To the memory of my parents,John and Alice,And to my husband, John S. FieldenJ.D.G.To my parents,Himangshu and Pratima,And to my wife Anuradha, and son, Siddhartha NeilS.C. 10. Preface to the Fourth EditionThis book was first published in 1971 and last revised in 1992. Duringthe span of over 30 years, it seems fair to say that the book has made ameaningful contribution to the teaching and learning of nonpara-metricstatistics. We have been gratified by the interest and thecomments from our readers, reviewers, and users. These commentsand our own experiences have resulted in many corrections,improvements, and additions.We have two main goals in this revision: We want to bring thematerial covered in this book into the 21st century, and we want tomake the material more user friendly.With respect to the first goal, we have added new materialsconcerning the quantiles, the calculation of exact power and simulatedpower, sample size determination, other goodness-of-fit tests, andmultiple comparisons. These additions will be discussed in more detaillater. We have added and modified examples and included exactv 11. vi PREFACE TO THE FOURTH EDITIONsolutions done by hand and modern computer solutions using MINI-TAB,*SAS, STATXACT, and SPSS. We have removed most of thecomputer solutions to previous examples using BMDP, SPSSX, Ex-ecustat,or IMSL, because they seem redundant and take up too muchvaluable space. We have added a number of new references but havemade no attempt to make the references comprehensive on somecurrent minor refinements of the procedures covered. Given the sheervolume of the literature, preparing a comprehensive list of referenceson the subject of nonparametric statistics would truly be a challengingtask. We apologize to the authors whose contributions could not beincluded in this edition.With respect to our second goal, we have completely revised anumber of sections and reorganized some of the materials, more fullyintegrated the applications with the theory, given tabular guides forapplications of tests and confidence intervals, both exact and approx-imate,placed more emphasis on reporting results using P values,added some new problems, added many new figures and titled allfigures and tables, supplied answers to almost all the problems, in-creasedthe number of numerical examples with solutions, and writtenconcise but detailed summaries for each chapter.We think the problemanswers should be a major plus, something many readers have re-questedover the years. We have also tried to correct errors and in-accuraciesfrom previous editions.In Chapter 1, we have added Chebyshevs inequality, the CentralLimit Theorem, and computer simulations, and expanded the listing ofprobability functions, including the multinomial distribution and therelation between the beta and gamma functions. Chapter 2 has beencompletely reorganized, starting with the quantile function and theempirical distribution function (edf), in an attempt to motivate thereader to see the importance of order statistics. The relation betweenrank and the edf is explained. The tests and confidence intervals forquantiles have been moved to Chapter 5 so that they are discussedalong with other one-sample and paired-sample procedures, namely,the sign test and signed rank test for the median. New discussions ofexact power, simulated power, and sample size determination, andthe discussion of rank tests in Chapter 5 of the previous editionare also included here. Chapter 4, on goodness-of-fit tests, has beenexpanded to include Lillieforss test for the exponential distribution,* MINITAB is a trademark of Minitab Inc. in the United States and other countries andis used herein with permission of the owner (on the Web at www.minitab.com). 12. PREFACE TO THE FOURTH EDITION viicomputation of normal probability plots, and visual analysis of good-nessof fit using P-P and Q-Q plots.The new Chapter 6, on the general two-sample problem, definesstochastically larger and gives numerical examples with exact andcomputer solutions for all tests. We include sample size determinationfor the Mann-Whitney-Wilcoxon test. Chapters 7 and 8 are the previous-editionChapters 8 and 9 on linear rank tests for the location and scaleproblems, respectively, with numerical examples for all procedures. Themethod of positive variables to obtain a confidence interval estimate ofthe ratio of scale parameters when nothing is known about location hasbeen added to Chapter 8, along with a much needed summary.Chapters 10 and 12, on tests for k samples, now include multiplecomparisons procedures. The materials on nonparametric correlationin Chapter 11 have been expanded to include the interpretation ofKendalls tau as a coefficient of disarray, the Students t approximationto the distribution of Spearmans rank correlation coefficient, and thedefinitions of Kendalls tau a, tau b and the Goodman-Kruskal coeffi-cient.Chapter 14, a new chapter, discusses nonparametric methods foranalyzing count data.We cover analysis of contingency tables, tests forequality of proportions, Fishers exact test, McNemars test, and anadaptation of Wilcoxons rank-sum test for tables with orderedcategories.Bergmann, Ludbrook, and Spooren (2000) warn of possiblemeaningful differences in the outcomes of P values from different sta-tisticalpackages. These differences can be due to the use of exact versusasymptotic distributions, use or nonuse of a continuity correction, or useor nonuse of a correction for ties. The output seldomgives such details ofcalculations, and even the Help facility and the manuals do not alwaysgive a clear description or documentation of the methods used to carryout the computations. Because this warning is quite valid, we tried toexplain to the best of our ability any differences between our hand cal-culationsand the package results for each of our examples.As we said at the beginning, it has been most gratifying to re-ceivevery positive remarks, comments, and helpful suggestions onearlier editions of this book and we sincerely thank many readersand colleagues who have taken the time. We would like to thankMinitab, Cytel, and Statsoft for providing complimentary copiesof their software. The popularity of nonparametric statisticsmust depend, to some extent, on the availability of inexpensive anduser-friendly software. Portions of MINITAB Statistical Softwareinput and output in this book are reprinted with permission ofMinitab Inc. 13. viii PREFACE TO THE FOURTH EDITIONMany people have helped, directly and indirectly, to bring aproject of this magnitude to a successful conclusion.We are thankful tothe University of Alabama and to the Department of InformationSystems, Statistics and Management Science for providing an en-vironmentconducive to creative work and for making some resourcesavailable. In particular, Heather Davis has provided valuable assis-tancewith typing. We are indebted to Clifton D. Sutton of GeorgeMason University for pointing out errors in the first printing of thethird edition. These have all been corrected. We are grateful to JosephStubenrauch, Production Editor at Marcel Dekker for giving us ex-cellenteditorial assistance. We also thank the reviewers of the thirdedition for their helpful comments and suggestions. These includeJones (1993), Prvan (1993), and Ziegel (1993). Ziegels review inTechnometrics stated, This is the book for all statisticians and stu-dentsin statistics who need to learn nonparametric statistics . . .. Iam grateful that the author decided that one more edition could al-readyimprove a fine package. We sincerely hope that Mr. Ziegel andothers will agree that this fine package has been improved in scope,readability, and usability.Jean Dickinson GibbonsSubhabrata Chakraborti 14. Preface to the Third EditionThe third edition of this book includes a large amount of additions andchanges. The additions provide a broader coverage of the nonpara-metrictheory and methods, along with the tables required to applythem in practice. The primary change in presentation is an integrationof the discussion of theory, applications, and numerical examples ofapplications. Thus the book has been returned to its original fourteenchapters with illustrations of practical applications following thetheory at the appropriate place within each chapter. In addition, manyof the hand-calculated solutions to these examples are verified andillustrated further by showing the solutions found by using one ormore of the frequently used computer packages. When the packagesolutions are not equivalent, which happens frequently because mostof the packages use approximate sampling distributions, the reasonsare discussed briefly. Two new packages have recently been developedexclusively for nonparametric methodsNONPAR: NonparametricStatistics Package and STATXACT: A Statistical Package for Exactix 15. x PREFACE TO THE THIRD EDITIONNonparametric Inference. The latter package claims to compute exactP values. We have not used them but still regard them as a welcomeaddition.Additional new material is found in the problem sets at the end ofeach chapter. Some of the new theoretical problems request verifica-tionof results published in journals about inference procedures notcovered specifically in the text. Other new problems refer to the newmaterial included in this edition. Further, many new applied problemshave been added.The new topics that are covered extensively are as follows. InChapter 2 we give more convenient expressions for the moments oforder statistics in terms of the quantile function, introduce the em-piricaldistribution function, and discuss both one-sample and two-samplecoverages so that problems can be given relating to exceedanceand precedence statistics. The rank von Neumann test for randomnessis included in Chapter 3 along with applications of runs tests in ana-lysesof time series data. In Chapter 4 on goodness-of-fit tests, Lillie-forsstest for a normal distribution with unspecified mean andvariance has been added.Chapter 7 now includes discussion of the control median test asanother procedure appropriate for the general two-sample problem.The extension of the control median test to k mutually independentsamples is given in Chapter 11. Other new materials in Chapter 11 arenonparametric tests for ordered alternatives appropriate for databased on k53 mutually independent random samples. The testsproposed by Jonckheere and Terpstra are covered in detail. The pro-blemsrelating to comparisons of treatments with a control or an un-knownstandard are also included here.Chapter 13, on measures of association in multiple classifica-tions,has an additional section on the Page test for ordered alter-nativesin k-related samples, illustration of the calculation of Kendallstau for count data in ordered contingency tables, and calculation ofKendalls coefficient of partial correlation. Chapter 14 now includescalculations of asymptotic relative efficiency of more tests and alsoagainst more parent distributions.For most tests covered, the corrections for ties are derived anddiscussions of relative performance are expanded. New tables includedin the Appendix are the distributions of the Lillieforss test fornormality, Kendalls partial tau, Pages test for ordered alternatives inthe two-way layout, the Jonckheere-Terpstra test for orderedalternatives in the one-way layout, and the rank von Neumann test forrandomness. 16. PREFACE TO THE THIRD EDITION xiThis edition also includes a large number of additional refer-ences.However, the list of references is not by any means purported tobe complete because the literature on nonparametric inference pro-ceduresis vast. Therefore, we apologize to those authors whose con-tributionswere not included in our list of references.As always in a new edition, we have attempted to correct pre-viouserrors and inaccuracies and restate more clearly the text andproblems retained from previous editions. We have also tried to takeinto account the valuable suggestions for improvement made by usersof previous editions and reviewers of the second edition, namely,Moore (1986), Randles (1986), Sukhatme (1987), and Ziegel (1988).As with any project of this magnitude, we are indebted to manypersons for help. In particular, we would like to thank Pat Coons andConnie Harrison for typing and Nancy Kao for help in the bibliographysearch and computer solutions to examples. Finally, we are indebted tothe University of Alabama, particularly the College of Commerce andBusiness Administration, for partial support during the writing of thisversion.Jean Dickinson GibbonsSubhabrata Chakraborti 17. Preface to the Second EditionA large number of books on nonparametric statistics have appearedsince this book was published in 1971. The majority of them areoriented toward applications of nonparametric methods and do notattempt to explain the theory behind the techniques; they are essen-tiallyusers manuals, called cookbooks by some. Such books serve auseful purpose in the literature because non-parametric methods havesuch a broad scope of application and have achieved widespreadrecognition as a valuable technique for analyzing data, particularlydata which consist of ranks or relative preferences and=or are smallsamples from unknown distributions. These books are generally usedby nonstatisticians, that is, persons in subject-matter fields. The morerecent books that are oriented toward theory are Lehmann (1975),Randles and Wolfe (1979), and Pratt and Gibbons (1981).A statistician needs to know about both the theory and methods ofnonparametric statistical inference. However, most graduate programsxiii 18. xiv PREFACE TO THE SECOND EDITIONin statistics can afford to offer either a theory course or a methodscourse, but not both. The first edition of this book was frequently usedfor the theory course; consequently, the students were forced to learnapplications on their own time.This second edition not only presents the theory with correctionsfrom the first edition, it also offers substantial practice in problemsolving. Chapter 15 of this edition includes examples of applications ofthose techniques for which the theory has been presented in Chapters1 to 14. Many applied problems are given in this new chapter; theseproblems involve real research situations from all areas of social, be-havioral,and life sciences, business, engineering, and so on. The Ap-pendixof Tables at the end of this new edition gives those tables ofexact sampling distributions that are necessary for the reader to un-derstandthe examples given and to be able to work out the appliedproblems. To make it easy for the instructor to cover applications assoon as the relevant theory has been presented, the sections ofChapter 15 follow the order of presentation of theory. For example,after Chapter 3 on tests based on runs is completed, the next assign-mentcan be Section 15.3 on applications of tests based on runs and theaccompanying problems at the end of that section. At the end ofthe Chapter 15 there are a large number of review problems arrangedin random order as to type of applications so that the reader can obtainpractice in selecting the appropriate nonparametric technique to usein a given situation.While the first edition of this book received considerable acclaim,several reviewers felt that applied numerical examples and expandedproblem sets would greatly enhance its usefulness as a textbook. Thissecond edition incorporates these and other recommendations. Theauthor wishes to acknowledge her indebtedness to the following re-viewersfor helping to make this revised and expanded edition moreaccurate and useful for students and researchers: Dudewicz andGeller (1972), Johnson (1973), Klotz (1972), and Noether (1972).In addition to these persons, many users of the first edition havewritten or told me over the years about their likes and=or dislikesregarding the book and these have all been gratefully received andconsidered for incorporation in this edition. I would also like to expressmy gratitude to Donald B. Owen for suggesting and encouraging thiskind of revision, and to the Board of Visitors of the University ofAlabama for partial support of this project.Jean Dickinson Gibbons 19. Preface to the First EditionDuring the last few years many institutions offering graduate pro-gramsin statistics have experienced a demand for a course devotedexclusively to a survey of nonparametric techniques and their justifi-cations.This demand has arisen both from their own majors and frommajors in social science or other quantitatively oriented fields such aspsychology, sociology, or economics. Although the basic statisticscourses often include a brief description of some of the better-knownand simpler nonparametric methods, usually the treatment is neces-sarilyperfunctory and perhaps even misleading. Discussion of only afew techniques in a highly condensed fashion may leave the impres-sionthat nonparametric statistics consists of a bundle of trickswhich are simply applied by following a list of instructions dreamed upby some genie as a panacea for all sorts of vague and ill-defined problems.One of the deterrents to meeting this demand has been the lackof a suitable textbook in nonparametric techniques. Our experience atxv 20. xvi PREFACE TO THE FIRST EDITIONthe University of Pennsylvania has indicated that an appropriate textwould provide a theoretical but readable survey. Only a moderateamount of pure mathematical sophistication should be required sothat the course would be comprehensible to a wide variety of graduatestudents and perhaps even some advanced undergraduates. Thecourse should be available to anyone who has completed at least therather traditional one-year sequence in probability and statistical in-ferenceat the level of Parzen, Mood and Graybill, Hogg and Craig, etc.The time allotment should be a full semester, or perhaps two seme-stersif outside reading in journal publications is desirable.The texts presently available which are devoted exclusively tononparametric statistics are few in number and seem to be pre-dominantlyeither of the handbook style, with few or no justifications,or of the highly rigorous mathematical style. The present book is anattempt to bridge the gap between these extremes. It assumes thereader is well acquainted with statistical inference for the traditionalparametric estimation and hypothesis-testing procedures, basic prob-abilitytheory, and random-sampling distributions. The survey is notintended to be exhaustive, as the field is so extensive. The purpose ofthe book is to provide a compendium of some of the better-knownnonparametric techniques for each problem situation. Those deriva-tions,proofs, and mathematical details which are relatively easilygrasped or which illustrate typical procedures in general nonpara-metricstatistics are included. More advanced results are simply statedwith references. For example, some of the asymptotic distributiontheory for order statistics is derived since the methods are equallyapplicable to other statistical problems. However, the Glivenko Can-tellitheorem is given without proof since the mathematics may be tooadvanced. Generally those proofs given are not mathematically rig-orous,ignoring details such as existence of derivatives or regularityconditions. At the end of each chapter, some problems are includedwhich are generally of a theoretical nature but on the same level as therelated text material they supplement.The organization of the material is primarily according to thetype of statistical information collected and the type of questions to beanswered by the inference procedures or according to the general typeof mathematical derivation. For each statistic, the null distributiontheory is derived, or when this would be too tedious, the procedure onecould follow is outlined, or when this would be overly theoretical, theresults are stated without proof. Generally the other relevant math-ematicaldetails necessary for nonparametric inference are also in-cluded.The purpose is to acquaint the reader with the mathematical 21. PREFACE TO THE FIRST EDITION xviilogic on which a test is based, those test properties which are essentialfor understanding the procedures, and the basic tools necessary forcomprehending the extensive literature published in the statisticsjournals. The book is not intended to be a users manual for the ap-plicationof nonparametric techniques. As a result, almost no numer-icalexamples or problems are provided to illustrate applications orelicit applied motivation. With the approach, reproduction of an ex-tensiveset of tables is not required.The reader may already be acquainted with many of the non-parametricmethods. If not, the foundations obtained from this bookshould enable anyone to turn to a users handbook and quickly graspthe application. Once armed with the theoretical background, the userof nonparametric methods is much less likely to apply tests indis-criminatelyor view the field as a collection of simple prescriptions. Theonly insurance against misapplication is a thorough understanding.Although some of the strengths and weaknesses of the tests coveredare alluded to, no definitive judgments are attempted regarding therelative merits of comparable tests. For each topic covered, some re-ferencesare given which provide further information about the testsor are specifically related to the approach used in this book. Thesereferences are necessarily incomplete, as the literature is vast. Theinterested reader may consult Savages Bibliography (1962).I wish to acknowledge the helpful comments of the reviewers andthe assistance provided unknowingly by the authors of other textbooksin the area of nonparametric statistics, particularly Gottfried E.Noether and James V. Bradley, for the approach to presentationof several topics, and Maurice G. Kendall, for much of the materialon measures of association. The products of their endeavors greatlyfacilitated this project. It is a pleasure also to acknowledge myindebtedness to Herbert A. David, both as friend and mentor. Histraining and encouragement helped make this book a reality. Parti-culargratitude is also due to the Lecture Note Fund of the WhartonSchool, for typing assistance, and the Department of Statistics andOperations Research at the University of Pennsylvania for providingthe opportunity and time to finish this manuscript. Finally, I thank myhusband for his enduring patience during the entire writing stage.Jean Dickinson Gibbons 22. ContentsPreface to the Fourth Edition vPreface to the Third Edition ixPreface to the Second Edition xiiiPreface to the First Edition xv1 Introduction and Fundamentals 11.1 Introduction 11.2 Fundamental Statistical Concepts 92 Order Statistics, Quantiles, and Coverages 322.1 Introduction 322.2 The Quantile Function 332.3 The Empirical Distribution Function 372.4 Statistical Properties of Order Statistics 40xix 23. xx CONTENTS2.5 Probability-Integral Transformation (PIT) 422.6 Joint Distribution of Order Statistics 442.7 Distributions of the Median and Range 502.8 Exact Moments of Order Statistics 532.9 Large-Sample Approximations to the Moments ofOrder Statistics 572.10 Asymptotic Distribution of Order Statistics 602.11 Tolerance Limits for Distributions and Coverages 642.12 Summary 69Problems 693 Tests of Randomness 763.1 Introduction 763.2 Tests Based on the Total Number of Runs 783.3 Tests Based on the Length of the Longest Run 873.4 Runs Up and Down 903.5 A Test Based on Ranks 973.6 Summary 98Problems 994 Tests of Goodness of Fit 1034.1 Introduction 1034.2 The Chi-Square Goodness-of-Fit Test 1044.3 The Kolmogorov-Smirnov One-Sample Statistic 1114.4 Applications of the Kolmogorov-SmirnovOne-Sample Statistics 1204.5 Lillieforss Test for Normality 1304.6 Lillieforss Test for the Exponential Distribution 1334.7 Visual Analysis of Goodness of Fit 1434.8 Summary 147Problems 1505 One-Sample and Paired-Sample Procedures 1565.1 Introduction 1565.2 Confidence Interval for a Population Quantile 1575.3 Hypothesis Testing for a Population Quantile 1635.4 The Sign Test and Confidence Interval forthe Median 1685.5 Rank-Order Statistics 1895.6 Treatment of Ties in Rank Tests 194 24. CONTENTS xxi5.7 The Wilcoxon Signed-Rank Test andConfidence Interval 1965.8 Summary 222Problems 2246 The General Two-Sample Problem 2316.1 Introduction 2316.2 The Wald-Wolfowitz Runs Test 2356.3 The Kolmogorov-Smirnov Two-Sample Test 2396.4 The Median Test 2476.5 The Control Median Test 2626.6 The Mann-Whitney U Test 2686.7 Summary 279Problems 2807 Linear Rank Statistics and the GeneralTwo-Sample Problem 2837.1 Introduction 2837.2 Definition of Linear Rank Statistics 2847.3 Distribution Properties of Linear Rank Statistics 2857.4 Usefulness in Inference 294Problems 2958 Linear Rank Tests for the Location Problem 2968.1 Introduction 2968.2 The Wilcoxon Rank-Sum Test 2988.3 Other Location Tests 3078.4 Summary 314Problems 3159 Linear Rank Tests for the Scale Problem 3199.1 Introduction 3199.2 The Mood Test 3239.3 The Freund-Ansari-Bradley-David-Barton Tests 3259.4 The Siegel-Tukey Test 3299.5 The Klotz Normal-Scores Test 3319.6 The Percentile Modified Rank Tests for Scale 3329.7 The Sukhatme Test 3339.8 Confidence-Interval Procedures 3379.9 Other Tests for the Scale Problem 338 25. xxii CONTENTS9.10 Applications 3419.11 Summary 348Problems 35010 Tests of the Equality of k Independent Samples 35310.1 Introduction 35310.2 Extension of the Median Test 35510.3 Extension of the Control Median Test 36010.4 The Kruskal-Wallis One-Way ANOVA Test andMultiple Comparisons 36310.5 Other Rank-Test Statistics 37310.6 Tests Against Ordered Alternatives 37610.7 Comparisons with a Control 38310.8 The Chi-Square Test for k Proportions 39010.9 Summary 392Problems 39311 Measures of Association for Bivariate Samples 39911.1 Introduction: Definition of Measures ofAssociation in a Bivariate Population 39911.2 Kendalls Tau Coefficient 40411.3 Spearmans Coefficient of Rank Correlation 42211.4 The Relations Between R and T; E(R), t, and r 43211.5 Another Measure of Association 43711.6 Applications 43811.7 Summary 443Problems 44512 Measures of Association in Multiple Classifications 45012.1 Introduction 45012.2 Friedmans Two-Way Analysis of Variance byRanks in a kn Table and Multiple Comparisons 45312.3 Pages Test for Ordered Alternatives 46312.4 The Coefficient of Concordance for k Sets ofRankings of n Objects 46612.5 The Coefficient of Concordance for k Sets ofIncomplete Rankings 47612.6 Kendalls Tau Coefficient for Partial Correlation 48312.7 Summary 486Problems 487 26. CONTENTS xxiii13 Asymptotic Relative Efficiency 49413.1 Introduction 49413.2 Theoretical Bases for Calculating the ARE 49813.3 Examples of the Calculation of Efficacy and ARE 50313.4 Summary 518Problems 51814 Analysis of Count Data 52014.1 Introduction 52014.2 Contingency Tables 52114.3 Some Special Results for k2 Contingency Tables 52914.4 Fishers Exact Test 53214.5 McNemars Test 53714.6 Analysis of Multinomial Data 543Problems 548Appendix of Tables 552Table A Normal Distribution 554Table B Chi-Square Distribution 555Table C Cumulative Binomial Distribution 556Table D Total Number of Runs Distribution 568Table E Runs Up and Down Distribution 573Table F Kolmogorov-Smirnov One-Sample Statistic 576Table G Binomial Distribution for y0.5 577Table H Probabilities for the WilcoxonSigned-Rank Statistic 578Table I Kolmogorov-Smirnov Two-Sample Statistic 581Table J Probabilities for the Wilcoxon Rank-Sum Statistic 584Table K Kruskal-Wallis Test Statistic 592Table L Kendalls Tau Statistic 593Table M Spearmans Coefficient of Rank Correlation 595Table N Friedmans Analysis-of-Variance Statistic andKendalls Coefficient of Concordance 598Table O Lillieforss Test for Normal DistributionCritical Values 599Table P Significance Points of TXY: Z (for KendallsPartial Rank-Correlation Coefficient) 600Table Q Pages L Statistic 601Table R Critical Values and Associated Probabilities forthe Jonckheere-Terpstra Test 602 27. xxiv CONTENTSTable S Rank von Neumann Statistic 607Table T Lillieforss Test for Exponential DistributionCritical Values 610Answers to Selected Problems 611References 617Index 635 28. ! 29. # 30. # 31. $ 32. % 33. # 34. ' 35. # 36. # 37. ++// # 38. # 39. # 40. # 41. = 42. # 43. = # 44. # 45. $ 46. # 47. ++ // 48. / 49. = 50. $ = 51. ? # 52. ++ 53. // 54. # 55. = 56. = 57. $ 58. # 59. # 60. # 61. # 62. ? 63. # 64. # 65. # # 66. @ # 67. # $ # 68. # 69. # 70. # 71. Y 72. = 73. # 74. # 75. = 76. # 77. # 78. # 79. ? # 80. [ # 81. # 82. # 83. @ 84. = 85. # 86. # 87. = 88. # # 89. # 90. $ 91. # 92. % 93. # 94. ] 95. / $ 96. = 97. = 98. # 99. % # 100. ++ // 101. # 102. = 103. = 104. [ ## 105. % 106. # 107. # ? 108. # 109. ^ 110. =# 111. 6 CHAPTER 1of the importance of speed, simplicity and cost factors, and the non-existenceof a fixed and universally acceptable criterion of good per-formance.Box and Anderson (1955) state that to fulfill the needs ofthe experimenter, statistical criteria should (1) be sensitive to changein the specific factors tested, (2) be insensitive to changes, of a mag-nitudelikely to occur in practice, in extraneous factors. These prop-erties,usually called power and robustness, respectively, are generallyagreed upon as the primary requirements of good performance in hy-pothesistesting. Parametric tests are often derived in such a way thatthe first requirement is satisfied for an assumed specific probabilitydistribution, e.g., using the likelihood-ratio technique of test con-struction.However, since such tests are, strictly speaking, not evenvalid unless the assumptions are met, robustness is of great concern inparametric statistics. On the other hand, nonparametric tests are in-herentlyrobust because their construction requires only very generalassumptions. One would expect some sacrifice in power to result. It istherefore natural to look at robustness as a performance criterion forparametric tests and power for nonparametric tests. How then do wecompare analogous tests of the two types?Power calculations for any test require knowledge of the prob-abilitydistribution of the test statistic under the alternative, but thealternatives in nonparametric problems are often extremely general.When the requisite assumptions are met, many of the classical para-metrictests are known to be most powerful. In those cases wherecomparison studies have been made, however, nonparametric tests arefrequently almost as powerful, especially for small samples, andtherefore may be considered more desirable whenever there is anydoubt about assumptions. No generalizations can be made for mod-erate-sized samples. The criterion of asymptotic relative efficiency istheoretically relevant only for very large samples. When the classicaltests are known to be robust, comparisons may also be desirable fordistributions which deviate somewhat from the exact parametric as-sumptions.However, with inexact assumptions, calculation of power ofclassical tests is often difficult except by Monte Carlo techniques, andstudies of power here have been less extensive. Either type of test maybe more reliable, depending on the particular tests compared and typeor degree of deviations assumed. The difficulty with all these com-parisonsis that they can be made only for specific nonnull distributionassumptions, which are closely related to the conditions under whichthe parametric test is exactly valid and optimal.Perhaps the chief advantage of nonparametric tests lies in theirvery generality, and an assessment of their performance under 112. INTRODUCTION AND FUNDAMENTALS 7conditions unrestricted by, and different from, the intrinsic postulatesin classical tests seems more expedient. A comparison under morenonparametric conditions would seem especially desirable for two ormore nonparametric tests which are designed for the same generalhypothesis testing situation. Unlike the body of classical techniques,nonparametric techniques frequently offer a selection from inter-changeablemethods. With such a choice, some judgments of relativemerit would be particularly useful. Power comparisons have beenmade, predominantly among the many tests designed to detect loca-tiondifferences, but again we must add that even with comparisonsof nonparametric tests, power can be determined only with fairlyspecific distribution assumptions. The relative merits of the differenttests depend on the conditions imposed. Comprehensive conclusionsare thus still impossible for blanket comparisons of very generaltests.In conclusion, the extreme generality of nonparametric techni-quesand their wide scope of usefulness, while definite advantages inapplication, are factors which discourage objective criteria, particu-larlypower, as assessments of performance, relative either to eachother or to parametric techniques. The comparison studies so fre-quentlypublished in the literature are certainly interesting, in-formative,and valuable, but they do not provide the sought-forcomprehensive answers under more nonparametric conditions. Per-hapswe can even say that specific comparisons are really contrary tothe spirit of nonparametric methods. No definitive rules of choice willbe provided in this book. The interested reader will find many perti-nentarticles in all the statistics journals. This book is a compendiumof many of the large number of nonparametric techniques which havebeen proposed for various inference situations.Before embarking on a systematic treatment of new concepts,some basic notation and definitions must be agreed upon and thegroundwork prepared for development. Therefore, the remainder ofthis chapter will be devoted to an explanation of the notation adoptedhere and an abbreviated review of some of those definitions and termsfrom classical inference which are also relevant to the special world ofnonparametric inference. A few new concepts and terms will also beintroduced which are uniquely useful in nonparametric theory. Thegeneral theory of order statistics will be the subject of Chapter 2, sincethey play a fundamental role in many nonparametric techniques.Quantiles, coverages, and tolerance limits are also introduced here.Starting with Chapter 3, the important nonparametric techniques willbe discussed in turn, organized according to the type of inference 113. 8 CHAPTER 1problem (hypothesis to be tested) in the case of hypotheses not invol-vingstatements about parameters, or the type of sampling situation(one sample, two independent samples, etc.) in the case of distribution-freetechniques, or whichever seems more pertinent. Chapters 3 and 4will treat tests of randomness and goodness-of-fit tests, respectively,both nonparametric hypotheses which have no counterpart in classicalstatistics. Chapter 5 covers distribution-free tests of hypotheses andconfidence interval estimates of the value of a population quantile inthe case of one sample or paired samples. These procedures are basedon order statistics, signs, and signed ranks.When the relevant quantileis the median, these procedures relate to the value of a locationparameter and are analogies to the one-sample (paired-sample) testsfor the population mean (mean difference) in classical statistics. Rank-orderstatistics are also introduced here, and we investigate the re-lationshipbetween ranks and variate values. Chapter 6 introduces thetwo-sample problem and covers some distribution-free tests for thehypothesis of identical distributions against general alternatives.Chapter 7 is an introduction to a particular form of nonparametric teststatistic, called a linear rank statistic, which is especially useful fortesting a hypothesis that two independent samples are drawn fromidentical populations. Those linear rank statistics which are particu-larlysensitive to differences only in location and only in scale are thesubjects of Chapters 8 and 9, respectively. Chapter 10 extends thissituation to the hypothesis that k independent samples are drawnfrom identical populations. Chapters 11 and 12 are concerned withmeasures of association and tests of independence in bivariate andmultivariate sample situations, respectively. For almost all tests thediscussion will center on logical justification, null distribution andmoments of the test statistic, asymptotic distribution, and other re-levantdistribution properties. Whenever possible, related methods ofinterval estimation of parameters are also included. During the courseof discussion, only the briefest attention will be paid to relative meritsof comparable tests. Chapter 13 presents some theorems relating tocalculation of asymptotic relative efficiency, a possible criterion forevaluating large sample performance of nonparametric tests relativeto each other or to parametric tests when certain assumptions are met.These techniques are then used to evaluate the efficiency of some ofthe tests covered earlier. Chapter 14 covers some special tests based oncount data.Numerical examples of applications of the most commonly usednonparametric test and estimation procedures are included after theexplanation of the theory. These illustrations of the techniques will 114. $ / 115. $ 116. ^ 117. = = 118. # $ 119. !! ! 120. ` 121. = $ 122. #! 123. !$ 124. =$ 125. $ = = 126. = 127. ? 128. # 129. # 130. {| |}|} 131. }~ | 132. ~~ 133. %$ 134. ! 135. #= 136. 137. ~|$ 138. ~ ~ 139. ~ | =++ // 140. = 141. 142. = 143. # 144. | } 145. $ 146. |} 147. # 148. $ 149. |} 150. | } 151. |} 152. | } 153. |$ 154. # 155. $ 156. | 157. } 158. }}}}}|} 159. | 160. | 161. # 162. = [ 163. ~~! 164. #! 165. # ! |} 166. | } 167. 12 CHAPTER 1EXni1aiXi!Xni1aiEXivarXni1aiXi!Xni1a2ivarXi 2XX14ij4naiaj covXi;XjcovXni1aiXi;Xni1biXi!Xni1aibi varXi XX14ij4naibj ajbicovXi;XjPROBABILITY FUNCTIONSSome special probability functions are shown in Table 2.1, along withthe corresponding mean, variance, and moment-generating function.Both discrete and continuous distributions are included; for a discretedistribution the probability function means the pmf, whereas for acontinuous distribution the probability function stands for the corres-pondingpdf. The term standard normal will designate the particularmember of the normal family where m 0 and s 1. The symbols fx and Fx will be reserved for the standard normal density andcumulative distribution functions, respectively.Three other important distributions are:Students tn: fXx n1=21 x2=nn1=2Bn=2; 1=2n0Snedecors Fn1;n2:n1=2fXx n1n2xn1=21 1 n1x=n2n1n2=2Bn1=2; n2=2x0; n1; n20Fishers zn1; n2:fXx 2n1=2n1n2en1x 1 n1e2x=n2n1n2=2Bn1=2; n2=2x1; n1; n20The gamma and beta distributions shown in Table 2.1 each contains aspecial constant, denoted by Ga and Ba; b respectively. The gammafunction, denoted by Ga, is defined asGa Z10xa1ex dx for a0 168. Table 2.1 Some special probability functionsName Probability function fXx mgf E(X) var(X)DiscretedistributionsBernoulli px1p1x pet 1 p p p1px 0; 104p41Binomial 169. nxpx1pnx pet 1pn np np1px 0; 1; . . . ; n04p41MultinomialN!x1!x2! 170. xk!k p1et1 171. pk1etk1 pkN EXi Npi VarXi Npi1pi CovXi;Xj Npipjpx11 px22 172. pxkxi 0; 1; . . . ;N;Pxi N,04pi41;Ppi 1Hypergeometric NNpNpxn xnp np1p NnNxN1x 0; 1; . . . ; n 04p41INTRODUCTION AND FUNDAMENTALS 13 173. Table 2.1 (Continued)Name Probability function fXx mgf E(X) var(X)Geometric 1px1ppet11pet1p1pp2x 1; 2; . . .04p41Uniform on 1, 2, . . ., N1Nx 1; 2; . . . ;NXNx1etxNN 12N2112ContinuousdistributionsUniform on a; b1baaxbetbetatbab a2ba212Normale1=2s2xm2ffiffiffiffiffiffiffiffiffiffiffip2ps2 etmt2s2=2 m s21x; m1;s014 CHAPTER 1 174. Gammaxa1ex=bbaGa 1btabt1ab ab20x; a; b1Exponential gamma with a 1 1bt1 b b2bt1Chi-square(n) gamma with a n=2; b 2 12t12t1n 2nWeibull abxb1eaxb at=b; G1 t=b a1=b; G1 1=b a2=bG1 2=b 0a; b; x1 G21 1=bBetaxa11xb1Ba; baa baba b2a b 1 0x1; a; b0 Laplace(double exponential)ejxyj=f2f1x; y1;f0y 2fLogistic11 exy=fyf2p231x; y1;f0*The mgf is omitted here because the expression is too complicated.INTRODUCTION AND FUNDAMENTALS 15 175. , | || 176. |} 177. = |} |} |}} $ [ |}} 178. |~||| ~ ~ 179. { 180. =| % | 181. ~ |} [ } || 182. ~ 183. }|}|~!#! 184. ! 185. # !!' * * 186. +#! |} 187. | } 188. # ! || } 189. }| } 190. | } 191. !! ! 192. !| !} ! 193. #!!| } 194. = ||| } 195. }}| } 196. | } 197. 9$ 198. 199. #| } 200. |} 201. | } 202. = 203. ! $ 204. =$ $ 205. |} 206. |} 207. | } 208. | #| } 209. || } 210. }| } 211. | } 212. | } 213. ! 214. 215. |} 216. | } 217. | } 218. | } 219. || } 220. #| } 221. #| } 222. | 223. | } 224. |#| } 225. | | } 226. # | } 227. | } 228. |!! 229. % | 230. | 231. ||*#-!*/! 0 - 232. } ~ % 233. ' 234. 18 CHAPTER 1PjXmj5k4s2k2Note that the finite variance assumption guarantees the existence ofthe mean m.The following result, called the Central Limit Theorem (CLT), isone of the most famous in statistics. We state it for the simplest i.i.d.situation.CENTRAL LIMIT THEOREMLet X1;X2; . . . ;Xn be a random sample from a population with mean mand variance s20 and let XXn be the sample mean. Then for n!1,the random variableffiffiffipnXXnm=s has a limiting distribution that isnormal with mean 0 and variance 1.For a proof of this result, typically done via the moment gen-eratingfunction, the reader is referred to any standard graduatelevel book on mathematical statistics. In some of the non-i.i.d. si-tuationsthere are other types of CLTs available. For example, if theXs are independent but not identically distributed, there is a CLTgenerally attributed to Liapounov. We will not pursue these anyfurther.POINT AND INTERVAL ESTIMATIONA point estimate of a parameter is any single function of randomvariables whose observed value is used to estimate the true value. Let^yyn uX1;X2; . . . ;Xn be a point estimate of a parameter y. Somedesirable properties of ^yyn are defined as follows for all y.1. Unbiasedness: E^yyn y for all y.yn x1; x2; . . . ; xnj^yyn does not depend on y, or,2. Sufficiency: fX1; X2;...;Xnj^yequivalently,fX1;X2;...;Xn x1; x2; . . . ; xn; y g^yyn; yHx1; x2; . . . ; xnwhere Hx1; x2; . . . ; xn does not depend on y.3. Consistency (also called stochastic convergence and convergence inprobability):limn!1Pj^yynyje 0 for every e0 235. INTRODUCTION AND FUNDAMENTALS 19a. If ^yyn is an unbiased estimate of y and limn!1 var^yyn 0,then ^yyn is a consistent estimate of y, by Chebyshevsinequality.b. ^yyn is a consistent estimate of y if the limiting distribution of ^yynis a degenerate distribution with probability 1 at y.yny24E^y4. Minimum mean squared error: E^yyny2, for anyother estimate ^yyn.yn4var^y5. Minimum variance unbiased: var^yyn for any otherestimate ^yyn where both ^yyn and ^yyn are unbiased.An interval estimate of a parameter y with confidence coefficientn1ffia, or a 1001a percent confidence interval for y, is a randominterval whose end points U and V are functions of observable randomvariables (usually sample data) such that the probability statementPUyV 1a is satisfied. The probability PUyV shouldbe interpreted as PUy PVy since the confidence limits Uand V are random variables (depending on the random sample) and yis a fixed quantity. In many cases this probability can be expressed interms of a pivotal statistic and the limits can be obtained via tabulatedpercentiles of standard probability distributions such as the standardnormal or the chi-square. A pivotal statistic is a function of a statisticand the parameter of interest such that the distribution of the pivotalstatistic is free from the parameter p ffiffi(and is often known or at leastderivable). For example, t Xm=S is a pivotal statistic forsetting up a confidence interval for the mean m of a normal populationwith an unknown standard deviation. The random variable t follows aStudents tn1 distribution and is thus free from any unknownparameter. All standard books on mathematical statistics cover thetopic of confidence interval estimation.A useful technique for finding point estimates for parameterswhich appear as unspecified constants (or as functions of such con-stants)in a family of probability functions, say fX:; y, is the method ofmaximum likelihood. The likelihood function of a random sample ofsize n from the population fX:; y is the joint probability function of thesample variables regarded as a function of y, orLx1; x2; . . . ; xn; y Yni1fXxi; yA maximum-likelihood estimate (MLE) of y is a value ^yy such that forall y,Lx1; x2; . . . ; xn;^yy5Lx1; x2; . . . ; xn; y 236. %[ 237. Y! 238. $ 239. Y! Y! 240. *-*!! !'$ 241. $ 242. ? 243. *~ 244. *| *$ 245. $ 246. *~ 247. *~ 248. $ 249. $ 250. + 251. *~ 252. + = 253. / 254. *~ / 255. + 256. /+ /$ 257. $ 258. *~ 259. /*~ *| 260. /+ / + 261. 0 2 / 262. 4 263. # 264. [ 265. INTRODUCTION AND FUNDAMENTALS 21nonparametric hypothesis testing, some confusion might arise if thesedistinctions were adhered to here. So the symbol a will be used todenote either the size of the test or the significance level or theprobability of a type I error, prefaced by the adjective exact wheneversupy 2o ay a.The power of a test is the probability that the test statistic willlead to a rejection of H0, denoted by Pwy PT 2 R. Power is ofinterest mainly as the probability of a correct decision, and so thepower is typically calculated when H0 if false, or H1 is true, and thenPwy PT 2 Rj y 2 Oo 1by. The power depends on thefollowing four variables:1. The degree of falseness of H0, that is, the amount of discrepancybetween the assertions stated in H0 and H12. The size of the test a3. The number of observable random variables involved in the teststatistic, generally the sample size4. The critical region or rejection region RThe power function of a test is the power when all but one of thesevariables are held constant, usually item 1. For example, we can studythe power of a particular test as a function of the parameter y, for agiven sample size and a. Typically, the power function is displayed as aplot or a graph of the values of the parameter y on the X axis againstthe corresponding power values of the test on the Y axis. To calculatethe power of a test, we need the distribution of the test statistic underthe alternative hypothesis. Sometimes such a result is either un-availableor is much too complicated to be derived analytically; thencomputer simulations can be used to estimate the power of a test. Toillustrate, suppose we would like to estimate the power of a test for themean m of a population with H0: m 10. We can generate on thecomputer a random sample from the normal distribution with mean 10(and say variance equal to 1) and apply the test at a specified level a. Ifthe null hypothesis is rejected, we call it a success. Now we repeat thisprocess of generating a same size sample from the normal distributionwith mean 10 and variance 1, say 1000 times. At the end of these 1000simulations we find the proportion of successes, i.e., the proportion oftimes when the test rejects the null hypothesis. This proportion is anempirical estimate of the nominal size of a test which was set a priori.To estimate power over the alternative, for example, we can repeat thesame process but with samples from a normal distribution with, say,mean 10.5 and variance 1. The proportion of successes from these 266. |~ 267. # /# 268. |~~~ 269. ||~~~$ 270. # $ 271. $ ++// 272. $ 273. # 274. $ 275. 8| } 276. [ 277. # 278. *~ *| 279. /+ / ~| / / 88 280. 88 281. = 282. = 283. *~ 284. + 285. / *~ $ 286. / / 287. / 288. INTRODUCTION AND FUNDAMENTALS 23logarithm is one of the most commonly used g(.) functions. Thelikelihood-ratio test is always a function of sufficient statistics, andthe principle often produces a uniformly most powerful test whensuch exists. A particularly useful property of T for constructingtests based on large samples is that, subject to certain regularityconditions, the probability distribution of 2 lnT approaches thechi-square distribution with k1k2 degrees of freedom as n!1,where k1 and k2 are, respectively, the dimensions of the spaces Oand o; k2k1.All these concepts should be familiar to the reader, since they arean integral part of any standard introductory probability and in-ferencecourse. We now turn to a few concepts which are especiallyimportant in nonparametric inference.P VALUEAn alternative approach to hypothesis testing is provided by com-putinga quantity called the P value, sometimes called a probabilityvalue or the associated probability or the significance probability.A P value is defined as the probability, when the null hypothesis H0 istrue, of obtaining a sample result as extreme as, or more extreme than(in the direction of the alternative), the observed sample result. Thisprobability can be computed for the observed value of the test statisticor some function of it like the sample estimate of the parameter in thenull hypothesis. For example, suppose we are testing H0: m 50 ver-susH1: m50 and we observe the sample result for X is 52. TheP value is computed as PX552 j m 50. The appropriate directionhere is values of X that are greater than or equal to 52, since thealternative is m greater than 50. It is frequently convenient to simplyreport the P value and go no further. If a P value is small, this isinterpreted as meaning that our sample produced a result that israther rare under the assumption of the null hypothesis. Since thesample result is a fact, it must be that the null hypothesis statement isinconsistent with the sample outcome. In other words, we shouldreject the null hypothesis. On the other hand, if a P value is large, thesample result is consistent with the null hypothesis and the nullhypothesis is not rejected.If we want to use the P value to reach a decision about whetherH0 should be rejected, we have to select a value for a. If the P value isless than or equal to a, the decision is to reject H0; otherwise, thedecision is not to reject H0. The P value is therefore the smallestlevel of significance for which the null hypothesis would be rejected. 289. = | /| ~~|} }/} /| ~~ 290. | } 291. |Y $= = 292. # 293. [ 294. ~~| ~~= 295. =# 296. # 297. ? 298. # 299. # 300. # 301. # = 302. / 303. *~ 304. 9 ## *| = /|/ 305. [ 306. [# | |!!-$ 307. | 308. $ 309. | 310. 311. ++// 312. = 313. = 314. = $ 315. = 316. = 317. 318. $ 319. $ 320. / 321. ~~ 322. / 323. /~ 324. /+ / 325. [# 326. ~ ~ 327. /+ / ~ 328. / |}} 329. = 330. , 331. -$ 332. # 333. 4 334. # 335. = 336. 4 337. : 338. : : 339. # [ 340. # 341. : 342. *~ *| 343. 4 344. $! : 345. : : 346. *|*~ 347. $! # 348. *| 349. ] 350. | 351. *| 352. *~ 353. | | $! 354. # 355. $ 356. 9 357. $! 358. = = 359. # 360. ! # 361. $! 362. Y 363. $! 364. | 365. ]# 366. ## $ 367. [ 368. || = 369. = 370. 4 *~ ~ 371. # 372. ~ 373. = 374. $ 375. ~ 376. ~ |; !!% 377. = 378. ++#// 379. # 380. = 381. = 382. % 383. ~| 384. 385. / 386. / 387. = 388. 28 CHAPTER 1number of jump points in the cdf of the test statistic. These exactprobabilities will be called exact a values, or natural significancelevels. The region can then be chosen such that either (1) the exact ais the largest number which does not exceed the nominal a or (2) theexact a is the number closest to the nominal a. Although most sta-tisticiansseem to prefer the first approach, as it is more consistentwith classical test procedures for a composite H0, this has not beenuniversally agreed upon. As a result, two sets of tables of criticalvalues of a test statistic may not be identical for the same nominal a;this can lead to confusion in reading tables. The entries in each tablein the Appendix of this book are constructed using the first approachfor all critical values.Disregarding that problem now, suppose we wish to compare theperformance, as measured by power, of two different discrete teststatistics. Their natural significance levels are unlikely to be the same,so identical nominal a values do not ensure identical exact prob-abilitiesof a type I error. Power is certainly affected by exact a, andpower comparisons of tests may be quite misleading without identicalexact a values. A method of equalizing exact a values is provided byrandomized test procedures.A randomized decision rule is one which prescribes rejection ofH0 always for a certain range of values of the test statistic, rejectionsometimes for another nonoverlapping range, and acceptance other-wise.A typical rejection region of exact size as a might be writtenT 2 R with probability 1 if T5t2, and with probability p if t14Tt2,where t1t2 and 0p1 are chosen such thatPT5t2jH0 pPt14Tt2jH0 aSome random device could be used to make the decision in practice,like drawing one card at random from 100, of which 100p are labeledreject. Such decision rules may seem an artificial device and areprobably seldom employed by experimenters, but the technique isuseful in discussions of theoretical properties of tests. The power ofsuch a randomized test against an alternative H1 isPwy PT5t2jH1 pPt14Tt2jH1A simple example will suffice to explain the procedure. A randomsample of size 5 is drawn from the Bernoulli population. We wish totest H0 : y 0:5 versus H1 : y0:5 at significance level 0.05. The teststatistic is X, the number of successes in the sample, which has the 389. INTRODUCTION AND FUNDAMENTALS 29binomial distribution with parameter y and n 5. A reasonablerejection region would be large values of X, and thus the six exactsignificance levels obtainable without using a randomized test fromTable C of the Appendix are:c 5 4 3 2 1 0PX5cjy 0:5 1=32 6=32 16=32 26=32 31=32 1A nonrandomized test procedure of nominal size 0.05 but exact sizea 1=32 0:03125has rejection regionX 2 R for X 5The randomized test with exact a 0:05 is found with t1 4 andt2 5 as follows:PX55jy 0:5 pP44X5 1=32 pPX 4 0:05so,1=32 5p=32 0:05 and p 0:12Thus the rejection region is X 2 R with probability 1 if X 5 and withprobability 0.12 if X 4. Using Table C, the power of this randomizedtest when H1: y 0:6 isPw0:6 PX 5jy 0:6 0:12 PX 4jy 0:6 0:0778 0:120:2592 0:3110CONTINUITY CORRECTIONThe exact null distribution of most test statistics used in nonpara-metricinference is discrete. Tables of rejection regions or cumulativedistributions are often available for small sample sizes only. However,in many cases some simple approximation to these null distributions isaccurate enough for practical applications with moderate-sized sam-ples.When these asymptotic distributions are continuous (likethe normal or chi square), the approximation may be improved by 390. 30 CHAPTER 1introduction a correction for continuity. This is accomplished byregarding the value of the discrete test statistic as the midpoint of aninterval. For example, if the domain of a test statistic T is only integervalues, the observed value is considered to be t0:5. If the decisionrule is to reject for T5ta=2 or T4t0and the large-samplea=2 approximation to the distribution of TETjH0 is the standard normalsTjH0under H0, the rejection region with continuity correction incorporatedis determined by solving the equationsta=2 0:5ETjH0sTjH0 za=2 andt0a=2 0:5ETjH0sTjH0 za=2where za=2 satisfies Fza=2 1a=2. Thus the continuity-corrected,two-sided, approximately size a rejection region isT5ETjH0 0:5 za=2sTjH0 orT4ETjH00:5za=2sTjH0One-sided rejection regions or critical ratios employing continuitycorrections are found similarly. For example, in a one-sided test withrejection region T5ta, for a nominal size a, the approximation to therejection region with a continuity correction is determined by solvingfor ta inta0:5ETjH0sTjH0 zaand thus the continuity corrected, one-sided upper-tailed, approxi-matelysize a rejection region isT5ETjH0 0:5 zasTjH0Similarly, the continuity corrected, one-sided lower-tailed, approxi-matelysize a rejection region isT4ETjH00:5zasTjH0The P value for a one-sided test based on a statistic whose nulldistribution is discrete is often approximated by a continuous dis-tribution,typically the normal, for large sample sizes. Like the rejec-tionregions above, this approximation to the P value can usually beimproved by incorporating a correction for continuity. For example,if the alternative is in the upper tail, and the observed value of aninteger-valued test statistic T is t0, the exact P value PT5t0jH0 is 391. INTRODUCTION AND FUNDAMENTALS 31approximated by PT5t00:5jH0. In the Bernoulli case withn 20; H0: y 0:5 versus H1: y0:5, suppose we observe X 13successes. The normal approximation to the P value with a continuitycorrection isPX513jH0 PX12:5 PX10 ffiffiffi5p 12:510 ffiffiffip5 PZ1:12 1F1:12 0:1314This approximation is very close to the exact P value of 0.1316 fromTable C. The approximate P value without the continuity correction is0.0901, and thus the continuity correction greatly improves the P valueapproximation. In general, let t0 be the observed value of the teststatistic T whose null distribution can be approximated by the normaldistribution. When the alternative is in the upper tail, the approx-imateP value with a continuity correction is given by 1Ft0ETjH00:5sTjH0In the lower tail, the continuity corrected approximate P value is givenby Ft0ETjH0 0:5sTjH0When the alternative is two-sided, the continuity corrected approx-imateP value can be obtained using these two expressions andapplying the recommendations given earlier under P value. 392. 2Order Statistics, Quantiles,and Coverages2.1 INTRODUCTIONLet X1;X2; . . . ;Xn denote a random sample from a population withcontinuous cdf FX. First let FX be continuous, so that the probability iszero that any two or more of these random variables have equalmagnitudes. In this situation there exists a unique ordered arrange-mentwithin the sample. Suppose that X1 denotes the smallest of theset X1;X2; . . . ;Xn; X2 denotes the second smallest; . . . and Xndenotes the largest. ThenX1X2 393. Xndenotes the original random sample after arrangement in increasingorder of magnitude, and these are collectively termed the order sta-tisticsof the random sample X1;X2; . . . ;Xn. The rth smallest, 14r4n,of the ordered Xs, Xr, is called the rth-order statistic. Some familiar32 394. ORDER STATISTICS, QUANTILES, AND COVERAGES 33applications of order statistics, which are obvious on reflection, are asfollows:1. Xn, the maximum (largest) value in the sample, is of interest inthe study of floods and other extreme meteorological phenomena.2. X1, the minimum (smallest) value, is useful for phenomenawhere, for example, the strength of a chain depends on theweakest link.3. The sample median, defined as Xn1=2 for n odd and any numberbetween Xn=2 and Xn=21 for n even, is a measure of location andan estimate of the population central tendency.4. The sample midrange, defined as X1 Xn=2, is also a measureof central tendency.5. The sample range XnX1 is a measure of dispersion.6. In some experiments, the sampling process ceases after collectingr of the observations. For example, in life-testing electric lightbulbs, one may start with a group of n bulbs but stop takingobservations after the rth bulb burns out. Then information isavailable only on the first r ordered lifetimes X1X2 395. Xr, where r4n. This type of data is often referred to as cen-soreddata.7. Order statistics are used to study outliers or extreme observations,e.g., when so-called dirty data are suspected.The study of order statistics in this chapter will be limited totheir mathematical and statistical properties, including joint andmarginal probability distributions, exact moments, asymptotic mo-ments,and asymptotic marginal distributions. Two general uses oforder statistics in distribution-free inference will be discussed later inChapter 5, namely, interval estimation and hypothesis testing of po-pulationpercentiles. The topic of tolerance limits for distributions,including both one-sample and two-sample coverages, is discussedlater in this chapter. But first, we must define another property ofprobability functions called the quantile function.2.2 THE QUANTILE FUNCTIONWe have already talked about using the mean, the variance, and othermoments to describe a probability distribution. In some situations wemay be more interested in the percentiles of a distribution, like thefiftieth percentile (the median). For example, if X representsthe breaking strength of an item, we might be interested in knowing 396. 34 CHAPTER 2the median strength, or the strength that is survived by 60 percent ofthe items, i.e., the fortieth percentile point. Or we may want to knowwhat percentage of the items will survive a pressure of say 3 lb. Forquestions like these, we need information about the quantiles of adistribution.A quantile of a continuous cdf FX of a random variable X is a realnumber that divides the area under the pdf into two parts of specificamounts. Only the area to the left of the number need be specifiedsince the entire area is equal to one. The pth quantile (or the 100pthpercentile) of FX is that value of X, say Xp, such that 100p percent ofthe values of X in the population are less than or equal to Xp, for anypositive fraction p0p1. In other words, Xp is a parameter of thepopulation that satisfies PX4Xp p, or, in terms of the cdfFXXp p. If the cdf of X is strictly increasing, the pth quantile is theunique solution to the equation Xp F1X p QX p, say. We callQX p; 0p1, the inverse of the cdf, the quantile function (qf ) ofthe random variable X.Consider, for example, a random variable from the exponentialdistribution with b 2. Then Table 2.1 in Chapter 1 indicates that thecdf isFXx 0 x01ex=2 x50(Since 1eXp=2 p for x0, the inverse is Xp 2 ln1p for0p1, and hence the quantile function is QXp 2 ln1p.The cdf and the quantile function for this exponential distribution areshown in Figures 2.1 and 2.2, respectively.Suppose the distribution of the breaking strength random vari-ableX is this exponential with b 2. The reader can verify that thefiftieth percentile QX0:5 is 1.3863, and the fortieth percentile QX0:4 is 1.0217. The proportion that exceeds a breaking strength of 3 poundsis 0.2231.In general, we define the pth quantile QX p as the smallest Xvalue at which the cdf is at least equal to p, orQXp F1X p inf x: FXx5p 0p1This definition gives a unique value for the quantile QX p even whenFX is flat around the specified value p, whereas the previous definitionwould not give a unique inverse of FX at p.Some popular quantiles of a distribution are known as thequartiles. The first quartile is the 0.25th quantile, the second quartile 397. ORDER STATISTICS, QUANTILES, AND COVERAGES 35Fig. 2.1 The exponential cdf with b 2.is the 0.50th quantile (the median), and the third quartile is the 0.75thquantile. These are also referred to as the 25th, the 50th, and the 75thpercentiles, respectively. Extreme quantiles (such as for p 0:95, 0.99,or 0.995) of a distribution are important as critical values for some teststatistics; calculating these is important in many applications.The cdf and the qf provide similar information regarding thedistribution; however, there are situations where one is more naturalthan the other. Note that formulas for the moments of X can also beexpressed in terms of the quantile function. For example,EX Z 10QX p dp and EX2 Z 10Q2X p dp 2:1so that s2 R 10 Q2X p dpR 10 QX p dp2. 398. 36 CHAPTER 2Fig. 2.2 The exponential quantile function with b 2.The following result is useful when working with the qf. LetfXp F0Xp be the pdf of X.Theorem 2.1 Assuming that the necessary derivatives all exist, thefirst and the second derivatives of the quantile function QX p areQ0Xp 1fXQXpand Q00 Xp f0XQXpffXQXpg3The proof of this result is straightforward and is left for thereader.It is clear that given some knowledge regarding the distri-butionof a random variable, one can try to use that information,perhaps along with some data, to aid in studying properties of sucha distribution. For example, if we know that the distribution of X isexponential but we are not sure of its mean, typically a simple randomsample is taken and the population mean is estimated by the samplemean XX. This estimate can then be used to estimate properties of thedistribution. For instance, the probability PX43:2 can be estimated 399. ORDER STATISTICS, QUANTILES, AND COVERAGES 37by 1e3:2=XX, which is the estimated cdf of X at 3.2. This, of course, isthe approach of classical parametric analysis. In the field of non-parametricanalysis, we do not assume that the distribution is ex-ponential(or anything else for that matter). The natural question thenis how do we estimate the underlying cdf ? This is where the sampledistribution function (sdf ) or the empirical cumulative distributionfunction (ecdf ) or the empirical distribution function (edf ) plays acrucial role.2.3 THE EMPIRICAL DISTRIBUTION FUNCTIONFor a random sample from the distribution FX, the empirical dis-tributionfunction or edf, denoted by Snx, is simply the proportion ofsample values less than or equal to the specified value x, that is,Snx number of sample values 4xnIn the above example, Sn3:2 can be used as a point estimate ofPX43:2. The edf is most conveniently defined in terms of the orderstatistics of a sample, defined in Section 2.1. Suppose that the nsample observations are distinct and arranged in increasing order sothat X1 is the smallest, X2 is the second smallest, . . . , and Xn is thelargest. A formal definition of the edf Snx isSnx 0 ifxX1i=n if Xi14xXi; i 1; 2; . . . ; n1 ifx5Xn8:3:1Suppose that a random sample of size n 5 is given by 9.4, 11.2,11.4, 12, and 12.6. The edf of this sample is shown in Figure 3.1.Clearly, Snx is a step (or a jump) function, with jumps occuring at the(distinct) ordered sample values, where the height of each jump isequal to the reciprocal of the sample size, namely 1=5 or 0.2.When more than one observation has the same value, we saythese observations are tied. In this case the edf is still a step functionbut it jumps only at the distinct ordered sample values X j and theheight of the jump is equal to k=n, where k is the number of datavalues tied at X j.We now discuss some of the statistical properties of the edf Snx.Let Tnx nSnx, so that Tnx represents the total number ofsample values that are less than or equal to the specified value x. 400. 38 CHAPTER 2Fig. 3.1 An empirical distribution function for n 5.Theorem 3.1 For any fixed real value x, the random variable Tnx hasa binomial distribution with parameters n and FXx.Proof For any fixed real constant x and i1, 2, . . . , n, define theindicator random variabledix IXi4x 1 ifXi4x0 ifXixThe random variables d1x; d2x; . . . ; dnx are independent andidentically distributed, each with the Bernoulli distribution withparameter Py, where y Pdix 1 PXi4x FXx. Now, sinceTnx ni1 dix is the sum of n independent and identicallydistributed Bernoulli random variables, it can be easily shown thatTnx has a binomial distribution with parameters n and y FXx.From Theorem 3.1, and using properties of the binomial dis-tribution,we get the following results. The proofs are left for thereader.Corollary 3.1.1 The mean and the variance of Snx are(a) ESnx FXx 401. ORDER STATISTICS, QUANTILES, AND COVERAGES 39(b) VarSnx FXx1FXx=nPart (a) of the corollary shows that Snx, the proportion ofsample values less than or equal to the specified value x, is an un-biasedestimator of FXx. Part (b) shows that the variance of Snx tends to zero as n tends to infinity. Thus, using Chebyshevs inequality,we can show that Snx is a consistent estimator of FXx.Corollary 3.1.2 For any fixed real value x; Snx is a consistent esti-matorof FXx, or, in other words, Snx converges to FXx inprobability.Corollary 3.1.3 ETnxTn y nFXxFXy, for xy.The convergence in Corollary 3.1.2 is for each value of x in-dividually,whereas sometimes we are interested in all values of x,collectively. A probability statement can be made simultaneously forall x, as a result of the following important theorem. To this end, wehave the following classical result [see Fisz (1963), for example, for aproof].Theorem 3.2 (Glivenko-Cantelli Theorem) Snx converges uniformly toFXx with probability 1, that is,P limn!1sup1x1jSnxFXxj 0 1Another useful property of the edf is its asymptotic normality,given in the following theorem.Theorem 3.3 As n!1, the limiting probability distribution of thestandardized Snx is standard normal, orlimn!1P( )ffiffiffinpffiffiffiffiSffiffiffiffinffiffiffiffixffiffiffiffiffiffiffiffiffiffiFffiffiffiXffiffiffiffiffixffiffiffiffiffiffiFXx1FXxp 4t FtProof Using Theorem 3.1, Corollary 3.1.1, and the central limittheorem, it follows that the distribution of nffiffiSffiffiffiffinffiffiffiffixffiffiffiffiffiffiffiffiffiffinffiffiffiFffiffiffiXffiffiffiffiffixffiffiffiffiffiffiffiffip ffiffip SnxFXx nFXx1FXx ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiFXx1FXxnffip approaches the standard normal as n!1. 402. 40 CHAPTER 2THE EMPIRICAL QUANTILE FUNCTIONSince the population quantile function is the inverse of the cdf and theedf is an estimate of the cdf, it is natural to estimate the quantilefunction by inverting the edf. This yields the empirical quantilefunction (eqf ) Qnu; 04u1, defined below.Qnu X1 if 0u41X2 if1n u42nX3 if2n u43n. . . . . . . . . . . . . . . . . . . . . . . .Xn ifn1n u418:Thus Qnu inf x : Snx5u. Accordingly, the empirical (or thesample) quantiles are just the ordered values in a sample. For exam-ple,if n 10, the estimate of the 0.30th quantile or the 30th percentileis simply Q100:3 X3; since 2100:34 310. This is consistent withthe usual definition of a quantile or a percentile since 30 percent of thedata values are less than or equal to the third order statistic in asample of size 10. However, note that according to definition, the0.25th quantile or the 25th percentile (or the 1st quartile) is also equalto X3 since 2=100:2543=10.Thus the sample order statistics are point estimates of the corre-spondingpopulation quantiles.For this reason, a study of the properties oforder statistics is as important in nonparametric analysis as the study ofthe properties of the samplemean in the context of a parametric analysis.2.4 STATISTICAL PROPERTIES OF ORDER STATISTICSAs we have outlined, the order statistics have many useful applica-tions.In this section we derive some of their statistical properties.CUMULATIVE DISTRIBUTION FUNCTION (CDF) OF XrTheorem 4.1 For any fixed real tPXr4t XnirPnSnt iXnirniFXti1FXtni 1t 1 4:1 403. ORDER STATISTICS, QUANTILES, AND COVERAGES 41This theorem can be proved in at least two ways. First, Xr4t ifand only if at least r of the Xs are less than or equal to t, and Theorem3.1 gives the exact distribution of the number of Xs less than or equalto t. This result holds even if the underlying distribution is discrete.A second proof, using mathematical statistical results about orderstatistics, is given later.PROBABILITY DENSITY FUNCTION (PDF) OF XrTheorem 4.2 If the underlying cdf FX is continuous with F0Xx fXx;the pdf of the rth-order statistic is given byfXr x n!r1!nr! FXxr11FXxnrfXx1 x 1 4:2This can be proved from Theorem 4.1 by differentiation and somealgebraic manipulations. A more direct derivation is provided later.Theorems 4.1 and 4.2 clearly show that the sample quantiles arenot distribution free. Because of this, although intuitively appealing aspoint estimators of the corresponding population quantiles, thesestatistics are often not convenient to use except in very special si-tuations.However, they frequently provide interesting starting pointsand in fact are the building blocks upon which many distribution-freeprocedures are based. The study of order statistics is thus vital to theunderstanding of distribution-free inference procedures.Some important simplification occur when we assume that thesample comes from the continous uniform population on (0,1). Notethat for this distribution FXt t for 0t1.