Commentary

Fraud Is Bad, Studying Fraud Is Hard

Susan S. Ellenberg, PhDFood and Drug Administration, Rockville, Maryland

Recently reported in Science: a German hematologist with a bibliography of 347papers has been shown to have included fraudulent data in 52 of these papers, withanother 42 raising strong suspicions of fraud [1]. It appears this gentleman will bejoining the list of notorious researchers who, motivated presumably by unrestrainedambition, falsified and manipulated data in numerous publications before being discov-ered and exposed. Names such as William Summerlin, John Darsee, Robert Slutskyand Roger Poisson will forever remind us that those who enter the realm of scientificresearch are not always inspired solely by the desire to seek truth [2, 3].

Each new instance of research fraud that makes its way into the public consciousnesseats away at public confidence in the scientific enterprise. This is particularly true forfraud in clinical investigations because the results of clinical research have a moredirect impact on the treatment of patients. For example, much more media attentionwas devoted to Roger Poisson, who falsified data on women with breast cancer tomake them appear eligible for clinical trials, than to any prior uncovering of fraud. Asa consequence, this case had a more explosive impact than others, garnering the atten-tion of the U.S. Congress and ultimately affecting the careers of senior scientists andleading to changes in the management of sponsored research [3].

It is surely in the best interest of science that fraudulent results be detected beforethey can be disseminated. One might hope that the peer review process would identifysuch distortions, but major fraud seems unlikely to be readily detected by the typicallylimited attention given by peer review. Horton, commenting on another recent case ofdata falsification in a breast cancer trial performed in South Africa [4], suggests thatincreasing the burden on the review process to improve the chance of detecting fraudis not warranted [5].

Those in the best position to ferret out data falsification or other manipulation priorto publication of results are probably the scientific collaborators of the individualattempting to perpetrate fraud. These individuals have in-depth knowledge of theinvestigation in progress and an extended time period during which they may becomeaware of the acts of fraud. Data from a review of cases of alleged misconduct inacademic institutions during 1980 to 1987 support this speculation; half of the 26 casesanalyzed were initially detected by coworkers, while only 3 cases were identified byeditorial peer review [6].

The paper by Ranstam et al. in this issue of Controlled Clinical Trials suggests thefrightening scenario that fraud in clinical research may be much more prevalent than

Controlled Clinical Trials 21:498–500 (2000) Elsevier Science Inc. 2000 0197-2456/00/$–see front matter655 Avenue of the Americas, New York, NY 10010 PII S0197-2456(00)00089-1

Commentary 499

one might expect. Ranstam and others developed a questionnaire circulated to allmembers of the International Society for Clinical Biostatistics (ISCB), asking about theirawareness of instances of fraud or attempted fraud in their research environments.Unfortunately, only 37% of the addressees responded to the questionnaire. Even moreunfortunately, 51% of those who did respond indicated that they were personally awareof one or more instances of fraud in a scientific investigation. Even if we assume thatnone of the nonresponders had ever observed fraud, this would still mean that 18%of all ISCB members were aware of scientific colleagues committing fraudulent acts inthe conduct of research. Although it is reasonable to speculate that ISCB membersaware of fraud might have been more likely to respond to the questionnaire, it islikely that the true percentage of members who have been aware of scientific fraud issubstantially higher than 18%.

There are two difficulties with the Ranstam paper. The first of these is the embar-rassingly low response rate to a survey conducted and reported by statisticians. It isunfortunate that the ISCB committee that undertook this project did not make moreof an effort to improve the response rate and thereby improve the accuracy of itsestimates. It is understandable that the committee was very sensitive to the need tomaintain anonymity of respondents, and that this may have discouraged attempts totarget initial nonresponders. Still, the result is far from satisfactory, and others havedone better—a survey of academic deans in 1988 asking about allegations of misconductin their institutions achieved a 75% response rate [6]. Only the glaringly high rate ofpositive responses among those ISCB members who completed and submitted thequestionnaire makes the current report worthy of some attention.

The second problem is that there exists no clear-cut and well-accepted definition offraud. Some might want to limit this term to the most severe acts of deception, thosethat could perhaps lead to criminal prosecution. Others may want to include a muchwider range of infractions. The directions to those completing the questionnaire indicatethat poor methodology and “borderline practices” should not be considered as fraud,but also indicate that ignoring multiple comparisons in a confirmatory trial may beconsidered fraud. The questionnaire itself includes suppression or selective deletionof data as a type of fraud, as well as deceptive design/analysis practices and deceptiveapproaches to reporting, in addition to the more severe types of misconduct relatingto fabrication and falsification of data. Virtually all statisticians involved in clinicalresearch for any length of time have interacted with investigators who wish to ignorenot only multiple comparisons but preplanned analyses, agreed-on procedures forincluding and excluding cases, and most other aspects of good statistical practices.Such interactions probably result more from ignorance of the importance of statisticalconsiderations than from any purposeful misconduct or intent to deceive. Differentrespondents may have had quite different views of what constitutes fraud, and we donot know the specifics of the cases of fraud in the minds of the responders. Thus, whilethe report unquestionably raises concerns, true alarm bells will have to await morerigorous efforts to document the problem.

REFERENCES1. Hagmann M. Panel finds scores of suspect papers in German fraud probe. Science

2000;288:2106–2107.2. Lock S. Research misconduct: a resume of recent events. In Lock S, Wells F, eds,

Fraud and Misconduct in Medical Research. BMJ Publishing Group, London, 1993.

500 S.S. Ellenberg

3. Peto R, Collins R, Sackett D, et al. The trials of Dr. Bernard Fisher: a Europeanperspective on an American episode. Control Clin Trials 1997;18:1–13.

4. Weiss RB, Rifkin RM, Stewart FM, et al. High-dose chemotherapy for high-riskprimary breast cancer: an on-site review of the Bezwoda study. Lancet 2000;355:999–1003.

5. Horton R. After Bezwoda. Lancet 2000;355:942–943.6. Committee on Science, Engineering, and Public Policy (U.S.). Panel on Scientific

Responsibility and the Conduct of Research. Responsible Science: Ensuring the Integ-rity of the Research Process/Panel on Scientific Responsibility and the Conduct ofResearch Committee on Science, Engineering, and Public Policy, National Academy ofSciences, National Academy of Engineering, Institute of Medicine, Vol. 1. Washington,DC: National Academy Press; 1992.