Sense and Nonsense

  • View

  • Download

Embed Size (px)

Text of Sense and Nonsense

Controversy, Misuse, and Subtlety &&11 Chamont Wang . POPULAR STATISTICS a ,erla tldltMl by D. B. Owen of StatI8Ilca Southern JlethodIaI Univnalty DtIlla, TGtII Nancy R. Mama 1liomt.ItIunnII University of CtIIifornla at Loa Angelu Loa Angelu, California 1. How to Tell the Liars from the Statisticians, Robert Hooke 2. Educated Guessing: How to Cope In an Uncertain World, Samuel Kotz and Donna F. Stroup 3. The Statistical Exorcist: Dispelling Statistics Anxiety, Myles Hollander and Frank Proschan 4. The Fascination of Statistics, edited by Richard J. Brook, Gregory C. Amold, Thomas H. Hassard, and Robert M. Pringle 5. Misused Statistics: Straight Talk for Twisted Numbers, A. J. Jaffe and Herbert F. Splrer 6. Sense and Nonsense of Statistical Inference: Controversy, Misuse, and Subtlety, Chamont Wang Sense and . Nonsense of Statistical Inference Controversy.lfisuse. and Subtlety Chamont Wang Department of Mathematics and StatIBtIcs Tlflllton State College Tlflllton, New .IeIwy Marcel Dekker, Inc. New York - Basel- Hong Kong Library of Conp-ess Cataloging-in-Publication Data Wang, Cbamont. Sense and nonsense of statistical inference : controversy, misuse, and subtlety I Chamont Wang. p. C1D. -- (Popular statistics ; 6) Includes bibliographical references and index. ISBN 0-8247-8798-6 (aJk. paper) 1. Science--Statistical methods. 2. Science-Philosophy. 3. Mathematical statistics. I. Title. n. Series. Q175.W276 1993 507.2-dc20 92-35539 CIP This book is printed on acid-free paper. Copyright 1993 by Marcel Dekker,lnc. All rights reserved. Neither this book nor any part may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying, microfilming, and recording, or by any information storage and retrieval system, without permission in writing from the publisher. Marcel Dekker, Inc. 270 Madison Avenue, New York, New York 10016 Current printing (last digit) 10 9 8 7 6 S 4 3 2 1 PRINTED IN THE UNITED STATES OF AMERICA .. ' ............. --Vtrt'l'lllda:, 1I3S-lRD Preface This book concerns the misuse of statistics in science. Before and during the preparation of the manuscript, we witnessed a growing interest among the pub-lic in the exposure of misused statistics in a variety of sources: news stories, advertisements, editorials, federal reports, and academic research. This book presents examples from these areas, but its primary focus is the abuse and misconception of statistical inference in scientific journals and statistical litera-ture. Currently there are about 40,000 scientific journals being published regu-larly (The New York Times, June 13, 1988). While it is often claimed that sci-ence is self-correcting, this belief is not quite true when applied to the use of statistical generalization, probabilistic reasoning, and causal inference in scientific investigation. The examples of misused statistics in this book (most of which were produced by prominent scientists or statisticians) show that a large number of scientists, including statisticians, are unaware of, or unwilling to challenge, the chaotic state of statistical practices. The central theme of this book is that statistical inference, if used improp-erly, can do more harm than good to the course of science. Accordingly, the book is written for the following audiences: 1. Graduate students of probability and mathematical statistics who will eventually teach and apply textbook statistics to real-life data. Many v vi Preface scientists and statisticians stumble in their application of seemingly innocuous statistical techniques. We hope that our students will not have to carry these mistakes to the next generation. 2. Teachers of statistics who would like to understand better the limita-tions of statistical inference and would like penetrating examples to enrich their instruction. The examples in this book and the related dis-cussions may well change the way they teach. 3. Empirical scientists who use statistical techniques to explore new fron-tiers or to support their research findings. In certain scientific discip-lines (such as economics, educational research, psychology, biomedical research, and social science), researchers have been using sophisticated statistical methods, encouraged in part by the reward systems in their fields. Such researchers may find the discussions herein to be relevant to their academic endeavors. Two additional audiences for whom this book will be informative are: 1. Applied statisticians who enjoy reading intellectual debates and would like to broaden their views on the foundation of statistical enterprise. 2. Theoretical statisticians who are curious about how their inventions have been abused by applied statisticians, empirical scientists, and other theoretical statisticians. In recent years there is growing impatience on the part of competent statisticians (like Professor David Freedman of the University of California at Berkeley); with misused statistics in scientific and statistical literature. Indeed, all empirical scientists and academic statisticians who plan to conduct their business as usual now run some risk of rmding their publications under attack. Since its early development in the late 19th and early 20th centuries, statis-tics has generated waves of interest among empirical scientists. As a new branch of science, statistics was intended to deal with the uncertainty in the data. But, in many instances, statistical methods may actually create new kinds of uncertainty in scientific investigation. These incidents are, nevertheless, quite normal in any young discipline. This book is thus an attempt to examine the "growing pains" of the statistical profession. For this purpose, the book is organized around the abuse and misconception of the following modes of statistical reasoning: 1. Tests of significance 2. Statistical generalization 3. Statistical causality 4. Subjective inference Chapter 1 is devoted to the ubiquitous statistical tests. Testing of statistical hypotheses, as it appears in the literature, is at the center of the teaching and Preface vii practice of statistical reasoning. However, in the empirical sciences, such tests are often irrelevant, wrong-headed, or both. The examples in this section are chosen from the biomedical sciences, economics, the behavioral sciences, and the statistical literature itself. Chapter 2 discusses conflicting views of randomization, which, according to orthodox statisticians, is the very signature of statistical inference. In practice, randomization has two functions: (i) generalization and (ii) causal inference. Both functions have close connections with induction, a notorious topic in epistemology (the philosophy of science). Part of Chapter 2 is thus devoted to certain aspects of induction and epistemology, and also relates to the discus-sion in Chapter 3, as well as to a variety of topics concerning subjective knowledge and objective experimentation (Chapters 5-8). Chapters 3 and 4 give examples of fallacious practices in statistical causal inference. Special emphasis is placed on regression models and time-series analysis, which are often used as instant formulas to draw causal relationships. In addition, Chapter 4 complements a motto of which all users of statistics should be aware: "No Causation without Manipulation" (Holland, 1987, JASA). Chapters 5-8 are devoted to favorable accounts of observational studies in soft sciences, and to human judgment in nonexperimental settings. Contrary to a popular belief among scientists, sometimes subjective knowledge is more reliable than "objective" experimentation. Collectively, these chapters advo-cate the development of a critical eye and an appreciative mind toward subjec-tive knowledge. For many years, I felt cheated and frustrated by the application of so-called statistical models and by the practice of applying hypothesis testing to nonex-perimental data. I am now less hostile to those statistics, because if used skill-fully they can be intellectually illuminating. This book presents, therefore, not only criticism but also appreciation of uses of statistical inference. To emphasize the constructive use of statistics, I have tried to include some illustrative examples: a modem version of Fisher's puzzle; graphic presenta-tion and time-series analysis of Old Faithful data; Shewhart control chart; descriptive statistics; Bayesian analysis; chi-square test; parameter design in quality control; random inputs to a feedback system; probabilistic algorithms for global optimization; nonlinear modeling; spectral estimation; and quantum probability. These examples are intended to illustrate that statistics is infinitely rich in its subtlety and esoteric beauty. Nevertheless, I believe that the practice of statistical inference has drifted into nevernever land for too long. Therefore, this book concentrates on the misuse rather than on the bright side of statistical inference. A theme running through most of the examples is that the scientific community (and our society viII Preface as a whole) will be better off if we insist on quality statistics. Individuals who share the same belief are urged to bring similar examples to light. A good way to start is to have a critical mind when readin