STATISTICS IN MEDICINE, VOL. 13. 1473- I477 (1994)

A LAY PERSON’S PERSPECTIVE ON STARTING AND STOPPING CLINICAL TRIALS

JEAN ROBINSON 56. Loncluli~ Routl. O.+rtl OX2 7EP. L‘.K

This is a rare opportunity to address an international group of clinicians and statisticians. 1 hope you will forgive me if I indicate some consumer concerns about research. You are concerned with how clinical trials should stop. We have not caught up with you yet; we are more concerned about how they start. It is not that we are opposed to research - far from it. When chorionic villus sampling became available I was one of a small group of consumers from different organiztions who got together and supported its introduction only in the context of a randomized trial comparing outcome with amniocentesis.

However, too many research projects come to a premature and unsatisfactory end, not because statistical analyses show that it is unnecessary or unethical to continue, but because they were ill conceived and badly planned in the first place. I t is both unethical and wasteful to use patients as research material in such projects, and a number of my contacts in consumer organizations insisted that I should bring you that message. I believe that ethics committees should monitor how many of the protocols they approve reach satisfactory completion.

Most of us find statistics terrifying and we would rather face a sigmoidoscopy than a Fleming-O’Brien boundary! I only had the courage to enter the territory because I had no idea that what I was doing was called ‘statistics’. In the early 1970’s I was Chair of the Patients Association dealing with hundreds of letters from women about induced births. I realized I had to try to read medical journals to understand what obstetricians were doing. I discovered a randomized trial from Glasgow’ which was designed to show that elective induction of labour could reduce the incidence of mature stillbirth. About 200 normal women were randomly allocated either to elective induction at term, or to have ‘normal’ care - which was a 46 per cent induction rate. So only 54 per cent of controls were not induced. One of the authors had previously studied mature stillbirths in Glasgow, and had reported that the incidence was 3 per 1000. I could not understand how this study could have demonstrated a reduced incidence - especially as high risk women were excluded. I challenged the The Glasgow study is still quoted as showing that elective induction of labour is safe and effective. However it taught me an important lesson: trust your common sense and when in doubt read the references.

The Report of the Royal College of General Practitioners on oral contraceptives4 concluded on the basis of a study of 46,000 women that ‘taking the Pill was a risk a well-informed women would be happy to take’. I had received many letters about Pill problems from women, who were beginning to question their doctor’s statistical advice, which was invariably: ‘Taking the Pill is safer than crossing the road’. After reading the RCGP study I came to the opposite conclusion from the author^,^ and women’s groups up and down the country shared my opinion. Here came the second lesson: experts’ conclusions from statistics may differ markedly from those of patients who see the data themselves.

CCC 0277-671 5/94/141473-05 0 1994 by John Wiley & Sons, Ltd.

1474 J. ROBINSON

The complaints I received about 'victim blaming' which burgeoned after the published association between sexual behaviour and cervix cancer risk led me to a paper on increased mortality in widows6 - which was presumed to be due to their sexual activity after disposal of the spouse. 'Who gets widowed early? I asked myself - which led to the wonderful data provided in the Registrar General's Decennial Supplements. I was able to show that risk was also affected by women's own occupation (textiles) and the occupation of a spouse who worked with dust, chemicals, metal, machine tool oil or meat, and who himself had a raised occupational risk of cancers of the lung, stomach or Here came the third lesson: unrecognized and unstated prejudices or assumptions (in this case about women's sexual behaviour) may decide the research pathways which are followed and the ways in which statistics are interpreted.

We did not seek, but were forced into, more statistical encounters. A small group of us who opposed a long term licence for Depo Provera found it easy to challenge a number of claims for the drug based on published literature, for example, that it improved breast feeding.' A valuable fourth lesson came from that work: comparison of reports from many women about adverse effects with those in the literature showed how published data can pare down, distort and omit information which potential users need to evaluate potential risks and benefits. A narrow, highly selective approach greatly reduces the benefits which could be obtained from the effort and expense involved. More consumer input - and possibly more consumer power - could greatly improve quality. However elegant the statistics, what they are actually measuring may be of little value to us.

The Association for Improvements in the Maternity Services (AIMS) now regularly monitors and criticizes published studies and submitted evidence to the House of Commons Select Committee on Health on the lack of proven effectiveness as well as consumer concerns about adverse effects of many routine childbirth interventions." It has persistently criticized the short-term nature of most obstetric research and demanded the retention of research records, so that longer term outcomes can be followed up. We were delighted with the report on maternity care which the Select Committee issued." It had taken us a long time to learn the fifth lesson: if doctors and researchers will not listen to us, in a service we pay for, eventually elected politicians may. In order to do that we need to understand statistics and quote them in a way others can understand.

Of course we are still learning. It may be useful to quote our latest consumer encounter where we believe a literature-based meta-analysis from the Oxford Perinatal Database was used inappropriately. Our major current concern at AIMS is lack of proof of efficacy and potential long-term risks of antenatal ultrasound screening.12 In a recent randomized study of Doppler ultrasound routine screening of placental perfusion in pregnancy, there was a substantial excess of perinatal deaths of normally formed babies in the exposed group compared with unexposed controls (16 versus 4).13 The authors rather dismissed a possible causal association because a meta-analysis of its use in high risk pregnancies had shown it to be associated with reduced risk.I4 However, the meta-analysis had included four studies only of high risk pregnancies, where the risk/benefit ratio was likely to be different from that of normal pregnancies; in two of the four studies there had been no unexposed ~ o n t r o l s ; ' ~ " ~ in the third less than half the 'treatment' group had actually been exposed;I7 in the fourth study (500 subjects) most women had only one Doppler exposure after 32 weeks (18) compared with the Davies study where participants had at least two, the first at 19-22 weeks. Perinatal mortality was the same in both groups, though there were more low Apgar scores in the exposed group. I have given brief details here to show that we are beginning to realize that if a meta-analysis is quoted, its relevance must be clear, and we have to understand its basis.

LAY PERSON'S PERSPECTIVE ON STARTING AND STOPPING CLINICAL TRIALS 1475

Table I. Induction rates

Untrasound group Controls

London 1982 19.0% 19.6% Alesund 1984 1.9% 7.8 Yo Trondheim 1984 6.5% 7.9 ?'o Glasgow 1984 3 1 .o% 29.0%

So you see our statistical education progresses very slowly - and we had to learn not because we wanted to, but because we could not avoid it. I have already made it clear that consumers are aware of quality issues in research. We have even had the first example of a group of subjects in a cancer research project critizing its methodology and conclusions. l 9 Nowadays neither re- searchers nor statisticians can ignore consumer concerns.

To come to the subject of stopping trials. I was aware from my early Patients Association days that there were ethical problems in stopping as well as starting. At one time I has a batch of complaints from relatives of sufferers from incurable neurological disease who had been involved in a drug trial from which patients felt they benefited. The trial had now stopped - and so had supplies of the experimental drug. The relatives were now left caring for patients who had sunk into deep depression following the withdrawal of an apparently hopeful treatment. No one had prepared them for it.

On early stopping I should perhaps give my own views, though they may not be representative; if I am a subject in a research study, I would prefer my contribution to be used in a way which gives the best chance of producing effective results which will benefit others. Therefore I would prefer the trial not to be stopped until it came up with a definitive answer if that is possible. Many of us are aware of research which altered the ethical territory for future researchers, but did not come up with definitive answers - the classic example being the early non-randomized study by Smithells on use of vitamin supplements for prevention of neural tube defects2'

Are we, however, too strongly influenced by the view that the only way, or the most effective way, rapidly to improve most doctor's care is to carry on until you get a humdinger of a result of overwhelming statistical significance? As consumers we have to give that view serious considera- tion when a simple treatment is available which can markedly reduce mortality from a common disease, and a major and rapid shift in attitude to treatment can save far more lives than those lost by later recruits to the study who die because they got the less effective treatment. Nowadays it is not just a medical conversion which is required of course. This must be followed by management decisions to divert resources from other care, or even government action which will bring in additional resources.

It is not just the power of statistics that influences others. Studies of what affects doctors' decisions show a variety of influences, including personal experiences and peer pressure.21 It is particularly worrying to find that doctors who are most confident are not neessarily the best informed, as shown in a recent study of blood transfusions,22 but more work needs to be done, particularly since sociological findings on the other side of the Atlantic are not necessarily transferable here or to other cultures. As the AIMS study noted, medical choices and the results from research may not easily be changed by rational argument because they are not in essence rationally determined. We quoted Thacker's comparative data on the effects of ultrasound scanning on induction rates at different centres. The most interesting result to us was not the effect of a new procedure, but the hugh and inexplicable geographical differences in rates which existed before and after the research was done23 (see Table I).

1476 J. ROBINSON

If bringing about change when reasonable evidence exists is the real problem, then that is what we have to tackle. Klimt and Canner suggest that clinical trials should have a built-in mechanism so that the impact of their evidence on medical practice can be e~aluated.’~

Of course I am biased in believing consumer pressure could be used more effectively. I know of one recent example where supplying information to consumers via the local press rapidly changed medical care. A nurse tutor in Nortern Ireland, Jane Waterson, found some maternity units were still routinely using injected instead of oral vitamin K, despite the increased leukaemia risk, because paediatricians did not accept the research findings. After her letter was published in the local newspaper there was a rapid change of policy.z5

Even if participating doctors are shielded from the Data Monitoring Committee’s knowledge that there are early but inconclusive indications of benefit in one arm of a trial, the ethical problem that new recruits are entering on a different consent basis remains - it is merely that the information has not been allowed to filter down the chain. Some of us may be able to live more comfortably with that than others.

When I was first on a district ethics committee I feared getting lost in a specialist field that was new to me. I decided that my bottom line would be ‘Could I happily defend this decision if there was a shock/horror story in the tabloids tomorrow?. It was a useful guideline, because that was exactly what happened. A randomized trial of anaesthesia for babies I had approvedz6 was attacked as ’barbaric’ by a Member of Parliament. The researchers were devastated. I felt that I could confidently defend our decision, and the brief I sent to the President of the General Medical Council was subsequently useful to the doctors in their successful libel action.

However hard we try, we are unlikely as consumers to pick up anything but the more obvious and simple statistical queries. It was therefore enormously encouraging to me to learn what a deep ethical interest and concern there is among statisticians. Our greatest protection, it seems, is that what we do not understand, they will have to justify to each other.

REFERENCES 1. Cole, R. A., et al. ‘Elective induction of labour - a randomised prospective trial’, Lancet, 1, 767-770

2. Robinson, J. ‘Elective induction of labour’, Lancet. 1, 1088 (1975). 3. Robinson, J. ‘Elective induction of labour’, Lancet, 1, 1242 (1975). 4. Royal College of General Practitioners, Oral Contraceptives and Health, Pitman Medical, 1974. 5. Grant, E. and Robinson, J. ‘The RCGP Oral Contraception Study’, Lancet, 1, 1206 (1975). 6. Leck, I., et al. ‘Incidence of cervical cancer by marital status’, Journal of Epidemiology and Community

Health, 32, 108- 110 (1978). 7. Robinson, J. ‘Occupational risks of husbands and wives and possible preventive strategies’, in Jordan,

J. A., Sharp, F., Singer, A., (eds.) Pre-clinical Neplasia of the Cervix: Proceedings of the ninth study group of the Royal College of Obstetricians and Gynaecologists (October 1981). RCOG, London, 1982, pp.

(1975).

1 1-27. 8. Robinson, J. ‘Cervix cancer: occupational risks’, Lancet, 11, 1496- 1497 (1983). 9. Submission to public hearing on Depo Provera. Behrer, M., Rakusen, J. and Robinson, J. Womens

Health Information’ Centre 52 Featherstone St, London EClY 8RT, 1983. 10. House of Commons Session 1991-2. Health Committee Second Report. Maternity Services Vol. 11,

Minutes of Evidence, HMSO, London, 1992, pp. 465-510. 11. House of Commons Session 1991-2. Health Committee Second Report. Maternity Services Vol. 1.

Report, HMSO, London, 1992. 12. Beech, B. and Robinson, J. ‘Ultrasound? Unsound’, Aims Journal, 5, No. 1, 3-26, (1993). 13. Davies, J. et al. ‘Randomised controlled trial of Doppler ultrasound screening of placental perfusion

during pergnancy’, Lancet, 2, 1299- 1303 (1992).

LAY PERSONS PERSPECTIVE ON STARTING AND STOPPING CLINICAL TRIALS 1477

14. Neilson, J. P. ‘Doppler ultrasound study of umbilical artery waveforms in high risk pregnancies’, in Chalmers, I. (ed.) Oxford Database of Perinatal Trials (computer file), Version 1.2, disk issue 6, record 3889, Oxford University Press, 1991.

15. Trudinger, B. J., Cook, C. M., Giles, L. W. B., Connelly, A. and Thompson, R. S. L. ‘Umbilical artery flow velocity waveforms in high-risk pregnancy: randomised controlled trial’, Lancet, 1, 188-90 (1987).

16. McParland, P. and Pearce, J. M. P. ‘Review article: Doppler blood flow in pregnancy’, Placenta, 9,

17. Omtzigt, A. W. J. ‘Clinical value of umbilical Doppler velocimetry - a randomised controlled trial’, University of Utrecht, 1990.

18. Newnham, J. P., ODea, M. R. A., Reid, K. P. and Diepeveen, D. A. ‘Doppler flow velocity waveform analysis in high risk pregnancies: a randomised controlled trial’, British Journal of Obstetrics and Gynaecoloyy, 98, 956-963 (1991).

427-450 (1988).

19. Bourke, I. and Goodare, H. ‘Bristol Cancer Help Centre’, Lancet, 338, 1401 (1991). 20. Smithells, R. W. et a / . ‘Apparent prevention of neural tube defects by periconceptional vitamin

21. Weinberg, A. D., Ullian, L., Richards, W. and Cooper, P. ‘Informal advice and information seeking

22. Salem-Schatz, S. et al. ‘Influence of clinical knowledge, organizational context and practice style on

23. Thacker, S. ‘Quality of controlled clinical trials. The case of imaging ultrasound in obstetrics: a review’,

24. Klimt, C. and Canner, P. ‘Terminating a long-term clinical trial’, Clinical Pharmacology and

25. Waterson, J. personal communication. 26. Anand, K. J. et a / . ‘Randomised trial of fentanyl anaesthesia in preterm babies undergoing surgery:

supplementation’, Archives of Diseases in Children, 56, 91 1-918 (1981).

between physicians’, Journal of Medical Education, 56, 174- 180 (1981).

transfusion decision making, Journal of the American Medical Association, 264, (4), 476-483 (1990).

British Journal of Obstetrics and Gynaecology, 92, 437-444 (1985).

Therapeutics, 641 -646 (1979).

effects on the stress response’, Lancet, 1, 62-66 (1987).