British Journal of Surgery 1995, 82, 1200-1203

Observer variation in assessment of quality of life in patients with oesophageal cancer J . M . B L A Z E B Y , M . H . W I L L I A M S * , D . A L D E R S O N and J . R . F A R N D O N

Universip Department of Surgery, Bristol Royal Infirmary and *Department of Social Medicine, University of Bristol, Bristol, UK Correspondence to: Miss J. M. Blazeby. Department of Surgery. Bristol Royal Infirmary, Bristol BS2 8HU: UK

Quality of life (QOL) assessment may improve the Treatment of Cancer Study Group on Quality of Life. One evaluation of treatment for oesophageal cancer but patient doctor, 52 patients and 39 carers completed the instru- compliance using self completion questionnaires is often ment independently. Proxy and patient ratings were poor. The use of a proxy to estimate QOL might improve analysed for agreement using weighted K scores. data collection from patients who are either unable or Agreement was poor or moderate in most QOL scales and unwilling to complete the questionnaire. This study items ( ~ 5 6 0 ) . This study demonstrates that a carer or examined whether a doctor or patients' carers could doctor is not sufficiently accurate to postulate the accurately assess QOL using the QLQ-C30 questionnaire patient's responses to the questionnaire. Quality of life developed by the European Organisation for Research and data, therefore, should come from the patients themselves.

Despite recent advances in preoperative staging procedures, surgical techniques and adjuvant treatments for patients with oesophageal cancer, the prognosis remains guarded. The 5-year survival rate following oesophagectomy is between 3 and 36 per cent and there are significant perioperative morbidity r a t e ~ l - ~ . For patients presenting with advanced disease, the median survival time following palliative intervention is 3-8 month^^-^. Standard outcome measures of survival, morbidity and relief of dysphagia are insufficient to evaluate all treatment options because they fail to take into account the broader effects of both illness and intervention on the patient. These effects have not been strictly defined but, as they include physical, social and psychological aspects of health, they are regarded as health-related quality of life (QOL)"-' I .

Assessment of health-related QOL is becoming more common, particularly in the management of cancer where standard measures of outcome do not adequately reflect treatment aimsl2-Is. There is an increasing number of well validated and reliable instruments available to measure QOL'2,16-18. The majority are questionnaires designed to be completed by the patient in an attempt to avoid potential observer bias, interobserver variation and because it is felt that patients alone can sensibly comment on their physical and psychological ~ t a t u s ' ~ - ~ ~ . Collecting accurate QOL data is not easy and a number of practical difficulties exist especially in elderly patients with advanced diseasez6. This can lead to poor compliance rates; in studies that have evaluated treatment of oesophageal cancer using validated QOL instruments, patient compliance rates vary between 30 and 92 per centZ7-31 . Failure to account for incomplete data may invalidate any conclusions drawn.

In view of the poor prognosis for many of these patients, the use of a proxy to collect QOL information may be necessary to obtain sufficient data to evaluate treatment options. This investigation, therefore, was

Also presented to the joint meeting of the British Association of Surgical Oncology and the British Oncological Association at York, UK in July 1995. Published in abstract form as Br J Cancer 1995; 72: SXXV 37 Paper accepted 9 May 1995

1200

designed to establish whether a doctor or the patient's carer could be a valid proxy for the patient's assessment of QOL.

Patients and methods Patient details Consecutive new patients presenting with oesophageal cancer and patients within 12 months of completion of treatment were studied between July and September 1994. Patients were interviewed at home without their carer present by one surgical registrar trained to administer a generic QOL questionnaire in a standard fashion. Interviews were based on a check-list containing all the items in the QOL questionnaire. The QLQ- C30 does not contain questions about swallowing and therefore an additional dysphagia scale was added. At the end of the interview the patient, a carer when one was available, and the doctor independently and concurrently completed the questionnaire. All patients and carers were aware of the diagnosis, the aims of treatment and the purpose of the study. The questionnaire for the patient's carer referred throughout to a spouse, close relative or partner living in the same house. The questionnaires were checked for omitted answers before completion of the interviews to avoid missing data.

Quality of life and dysphagia assessment The QOL instrument used was the European Organisation for Research and Treatment of Cancer (EORTC) generic measure of QOL, the QLQ-C30'". This has been extensively validated and tested in patients with lung, breast and ovarian canceP2. The EORTC QLQ-C30 questionnaire is composed of multi-item scales and single items as shown in Table 1. Further methodological details underlying the construction of this questionnaire are described elsewhere and scoring algorithms have been produced by the EORTC Study Group on Quality of

. All scale and item scores were linearly transformed LifeI".33-7S to a categorical scale ranging from zero to 100. For the five functional scales and the global QOL scale a higher score represents a better level of function, whereas in the symptom scales and single items a higher score corresponds to a higher level of symptoms. The EORTC QLQ-C30 questionnaire was developed with a modular approach to allow the core questionnaire to be sufficiently general to permit cross-study comparisons, with the addition of disease-specific modules to address particular questions of relevance to patients in a particular study'".

0 1995 Blackwell Science Ltd

Q U A L I T Y O F L I F E A S S E S S M E N T 1201

The dysphagia scale used in this study was designed for self completion. It was taken from the specific module for oesophageal cancer currently under development by the authors in conjunction with the EORTC Study Group on Quality of Life. It contains three questions concerning the ability to eat solid foods, soft foods and liquids. The dysphagia score was also linearly transformed such that final scores ranged from zero to 100, a high score being equivalent to a higher level of dysphagia.

Table 1 Quality of life scales and items in the European Organisation for Research and Treatment of cancer QLQ-C30 questionnaire

No. of items

Function scales Physical Role Cognitive Emotional Social Global QOL

Symptom scales Pain Fatigue Emesis

Symptom items Dyspnoea Trouble sleeping Appetite loss Constipation Diarrhoea Financial impact

2 3 2

QOL, quality of life

Statistical analysis The doctor’s and carer’s ratings were compared with those for the corresponding patient. The closeness of agreement was assessed by calculating the overall proportion of ratings in exact agreement. Cohen’s weighted K score was used to take into account both chance agreement and the level of disagreement

between raters, with differences of one category being less serious than those of two or three categories3“. The weighted K scores range from less than 0 to 1, with 1 indicating perfect agreement, 0 indicating no agreement in excess of chance and negative values indicating worse than chance agreement. Weighted K values greater than 0.80 indicate excellent agreement, between 0.61 and 0.80 good, between 0.41 and 0.60 moderate, between 0.21 and 0.40 fair, and 0.20 or less, poor agreement3’.

Results Seventy-seven patients were treated for oesophageal cancer in the 12-month period preceding the study, of whom 44 were still alive. Of these patients, 40 were able to be interviewed. Four patients were excluded because of confusion or poor eyesight. Twenty-four had undergone an oesophagectomy between 4 and 50 weeks previously, and 16 patients had received palliative treatment between 4 and 20 weeks previously. In addition, 12 new patients presented during the study period, all of whom were interviewed before treatment started. On the day of the interviews of the 52 participants, seven carers were not

Table 2 Characteristics of patients and carers

Patients Carers

Number 52

Sex ratio (M : F) Treatment

Mean (range) age (years) 68 (39- 32 : 20

Before treatment 12 < 6 months after oesophagectomy 10 < 6 months after palliation 16 > 6-12/12 months after oesophagectomy14

Husband or wife - Partner -

Relationship to patient

Sibling - Son or daughter -

39

16 : 23 -84)59 (38-87)

25

1 11

Table 3 Agreement between patient and scores generated by the proxy

Patients and doctor (n = 52) Patients and carers (n = 39)

Pairs with exact agreement Weighted K agreement Weighted K

Pairs with exact

Function scales Physical Role Cognitive Emotional Social Global QOL

Symptom scales Pain Fatigue Emesis

Symptom items Dyspnoea Trouble sleeping Appetite loss Constipation Diarrhoea Financial impact Dysphagia

16 (31) 35 (67) 25 (48) 9 (17) 6 (12) 7 (14)

26 (50) 26 (50) 18 (35)

20 (39) 27 (52)

25 (48) 32 (62j 37 (71) 33 (64) 13 (25)

0.43 0.55 0.35 0.41 0.14 0.33

0.61 0.53 0.33

031 0.26 0.47 0.54 0.25 0.23 0.43

14 (36) 26 (67)

11 (28) 17 (44) 12 (31)

17 (44) 12 (31) 18 (46)

23 (59) 23 (59) 23 (59)

21 (54)

26 (67j 30 (77) 27 (69)

0.53 0.58 0.54 0.40 0.58 0.50

058 0.39 0.54

0.48 0.44 0.56 0.58 0.52 0.39 0.20

Values in parentheses are percentages. QOL, quality of life

0 1995 Blackwell Science Ltd, British Journal ofsurgely 1995,82, 1200-1203

1202 J . M. B L A Z E B Y , M . H . W I L L I A M S , D. A L D E R S O N and J. R. F A R N D O N

available to complete the questionnaire and six patients lived alone. Thirty-nine carers were therefore studied. Table 2 presents the features of the patients and carers.

Agreement between patients’ and doctor’s scores The percentages of function scores with exact agreement between the doctor and patient were low (below 49 per cent) in all but the role function scale (67 per cent) (Table 3). The proportion of pairs with exact agreement in the ten symptom scales and items was higher than for the function scores, with six having agreement in 50 per cent or more pairs and a highest agreement of 71 per cent in the diarrhoea scale. The weighted K values obtained for the function and symptom scales were moderate for seven (0.41-0.60) and fair for seven (0.21-0.40). The exceptions were social function (~=0 .14) , where the percentage of pairs in exact agreement was low (12 per cent), and pain where the weighted K score indicated good agreement ( ~ ~ 0 . 6 1 ) .

Agreement between patients’ and carers’ scores The percentages of pairs with exact agreement for function and symptom ratings were generally higher between patients and carers than for patients and the doctor (Table 3). Exceptions were role function, pain and fatigue. The highest agreement was for diarrhoea (77 per cent). Weighted K values were moderate in 12 of 16 scales, fair in three (emotional function, fatigue and financial impact) and poor in one, dysphagia (K = 0.2).

Discussion In the selection of patients with oesophageal cancer for major surgery or for purely palliative therapies, the influence of treatment options on QOL should be an integral part of management decision making. This was recognized by Stoller et al.38 as early as 1977 but little attention has been paid to this important issue since. None of the published studies on QOL in patients with oesophageal cancer has used the EORTC QLQ-C30 questionnaire, which is one of the best researched, valid and reliable measures of QOL currently available for the study of patients with ~ a n c e r ’ ~ ~ ’ ~ , ~ ~ .

The results of this study indicate that the use of a proxy to estimate the patient’s responses to the QOL questionnaire appropriately is incorrect, since a trained doctor who had spent 12 months undertaking QOL research in patients with oesophageal cancer achieved agreement with only between 12 and 71 per cent of the patients’ responses. Other doctors without this training might be even less accurate. The use of a carer as a proxy for the patient, although producing a closer agreement than that found between patient and doctor, was also insufficiently accurate to properly determine the patient’s answers. The carers were better judges of the functional aspects that determine the patients’ QOL than the doctor. Doctors may be poor at measuring function because they are more accustomed to assessing symptoms than the emotional aspects or social impact of disease. Other studies, using different QOL instruments, have also found that the ability of doctors to rate patients’ QOL accurately is limited. There is no reason to believe that any other group of health professionals would fare differently. Indeed, if the task were delegated to a number of

professionals the problem of interobserver variation would probably only lead to wider d i s c r e p a n ~ i e s ~ ~ ~ ~ ~ ~ ~ ~ . Studies that have incorporated QOL ratings by carers and patients have reported varying levels of agreement. The majority have found similar results to those of this study, that carers tend to under-rate patients’ QOL1s32n,41~42. The assumption that the patients gave the correct answer may not be correct, and an answer agreed between patient, carer and doctor may produce different results. However, the aim of measuring QOL is to record the patient’s perspective, and this study demonstrates that a carer or doctor is not sufficiently accurate to postulate the patient’s responses to the questionnaire.

QOL questionnaires should be used to evaluate new therapeutic strategies for patients with oesophageal cancer. They should be completed by the patient with help from a trained interviewer only to clarify the format of the questionnaire. If the patient’s opinions are not available because of ill health or illiteracy, the use of a trained researcher to estimate a patient’s QOL may be employed but limitations in analysis and conclusions must be acknowledged.

Acknowledgements The authors acknowledge Mr K. Jeyasingham and Mr C. P. Forrester-Wood for allowing their patients to be studied. J.M.B. is a Research Training Fellow supported by the Research and Development Directorate, South and West Regional Health Authority.

References 1 Salama FD, Leong YP. Resection for carcinoma of the

oesophagus. J R Colt Surg Edinb 1989; 3 4 97-100. 2 Muller JM, Erasmi H, Stelzner M, Zieren U, Pichlmaier H.

Surgical therapy of oesophageal carcinoma. Br J Surg 1990;

3 Watson A. A study of the quality and duration of survival following resection, endoscopic intubation and surgical intubation in oesophageal carcinoma. Br J Surg 1982; 69:

4 Watson A. Operable esophageal cancer: current results from the West. World J Surg 1994; 15: 361-6.

5 Oliver SE, Robertson CS, Logan RFA. Oesophageal cancer: a population-based study of survival after treatment. Br J Surg

6 Maunoury V, Brunetaud JM, Cochelard D, Boniface B, Cortot A. Paris JC. Endoscooic oalliation for inoDerable

77: 845-57.

585-8.

1992; 79: 1321-5.

7

8

9

10

11

12

malignant’ dysphagia: long te;m f h o w up. Gut 1!992; 33:

Reed CE, Marsh WH, Carlson LS, Seymore CH, Kratz JM. Prospective, randomised trial of palliative treatment for unresectable cancer of the esophagus. Ann Thorac Surg 1991;

Highley MS, Parnis FX, Trotter GA et a/ . Combination chemotherapy with epirubicin, cisplatin and 5-fluorouracil for the palliation of advanced gastric and oesophageal cancer. Br

Troidl H. Quality of life: definition, conceptualization and implications - a surgeon’s view. Theoretical Surgery 1991; 6:

Fallowfield U. The Quality of Life: The Missing Measurement in Health Care. London: Souvenir Press, 1990. Tchekmedyian NS, Cella DF. Quality of life in current oncology practice and research. Oncology 1990 4 1-232. de Haes JCJM, van Knippenberg FCE, Neijt JP. Measuring psychological and physical distress in cancer patients: structure and application of the Rotterdam Symptom Checklist. Br J Cancer 1990; 62: 1034-8.

1602-7.

51: 552-6.

J S U ~ 1994; 81: 1763-5.

138-42.

0 1995 Blackwell Science Ltd, British Journal ofSutgery 1995,82, 1200-1203

Q U A L I T Y O F L I F E A S S E S S M E N T 1203

13 Selby P. Measurement of quality of life in cancer patients. J Pharm Pharmacol 1993; 45(Suppl 1): 384-6.

14 Bleehen NM, Girling DJ, Machin D, Stephens RJ. A randomised trial of three or six courses of etoposide cyclophosphamide methotrexate and vincristine or six courses of etoposide and ifosfamide in small cell lung cancer (SCLC) 11: quality of life. Br J Cancer 1993; 68: 1157-66.

15 Olschewski M, Schulgen G, Schumacher M, Altman DG. Quality of life assessment in clinical cancer research. Br J Cancer 1994; 70: 1-5.

16 Aaronson NK, Ahmedzai S, Bergman B et al. The European Organisation for Research and Treatment of Cancer QLQ- C30: a quality of life instrument for use in international clinical trials in oncology. J Natl Cancer Znst 1993; 85: 365-76.

17 Schipper H, Clinch J, McMurray A, Levitt M. Measuring the quality of life of cancer patients: the Functional Living Index - Cancer: development and validation. J Clin Oncol

18 Spitzer WO, Dobson AJ, Hall J et al. Measuring the quality of life of cancer patients. A concise QL-index for use by physicians. Journal of Chronic Diseases 1981; 34: 585-97.

19 Hall R, Horrocks JC, Clamp SE, De Dombal FT. Observer variation in assessment of results of surgery for peptic ulceration. BMJ 1976; 1: 814-16.

20 Slevin ML, Plant H, Lynch D, Drinkwater J, Gregory WM. Who should measure quality of life, the doctor or the patient? Br J Cancer 1988; 57: 109-12.

21 Selby PJ, Chapman JAW, Etazadi-Amoli J, Dalley D, Boyd NF. The development of a method for assessing the quality of life of cancer patients. BrJ Cancer 1984; 50: 13-22.

22 Mackillop WJ, Stewart WE, Ginsburg AD, Stewart SS. Cancer patients’ perceptions of their disease and its treatment. Br J Cancer 1988; 58: 355-8.

23 Cook DJ, Guyatt GH, Juniper E et al. Interviewer versus self- administered questionnaires in developing a disease-specific, health-related quality of life instrument for asthma. J Clin Epidemiol 1993; 46: 529-34.

24 Higginson IJ, McCarthy M. Validity of the support team assessment schedule: do staffs’ ratings reflect those made by patients or their families? Palliative Medicine 1993; 7: 219-28.

25 Osoba D. Lessons learned from measuring health-related quality of life in oncology. J Clin Oncol 1994; 12: 608-16.

26 Hopwood P, Stephens RJ, Machin D, MRC Lung Cancer Working Party. Approaches to the analysis of quality of life data: experiences gained from a Medical Research Council Lung Cancer Working Party palliative chemotherapy trial. Quality of Life Research 1994; 3: 339-52.

27 Barr H, Krasner N, Raouf A, Walker RJ. Prospective randomised trial of laser therapy only and laser therapy followed by endoscopic intubation for the palliation of

1984; 2: 472-83.

malignant dysphagia. Gut 1990; 31: 252-8. 28 Loizou LA, Rampton D, Atkinson M, Robertson C, Bown

SG. A prospective assessment of quality of life after endoscopic intubation and laser therapy for malignant dysphagia. Cancer 1992; 7 0 386-91.

29 Barr H, Krasner N. Prospective quality-of-life analysis after palliative photoablation for the treatment of malignant dysphagia. Cancer 1991; 68: 1660-4.

30 Roder JD, Herschbach P, Sellschopp A, Siewert JR. Quality- of-life assessment following oesophagectomy. Theoretical Surgery 1991; 6: 206-10.

31 van Knippenberg FCE, Out JJ, Tilanus HW, Mud HJ, Hop WCJ, Verhage F. Quality of life in patients with resected oesophageal cancer. SOC Sci Med 1992; 35: 139-45.

32 Osoba D, Zee B, Pater J, Warr D, Kaizer L, Latreille J. Psychometric properties and responsiveness of the EORTC quality of life questionnaire (QLQ-C30) in patients with breast, ovarian and lung cancer. Quality of Life Research 1994;

33 Aaronson NK, Bullinger M, Ahmedzai S, A modular approach to quality-of-life assessment in cancer clinical trials. Recent Results Cancer Res 1988; 111: 231-49.

34 Aaronson NK, Ahmedzai S, Bullinger M et al. The EORTC core quality of life questionnaire: interim results of an international field study. In: Osoba D, ed. Effect of Cancer on Quality of Life. Boston, Massachussetts: CRC Press, 1991:

35 Aaronson NK, Beckmann JH. The Quality of Life of Cancer Patients. New York: Raven Press, 1987.

36 Cohen J. Weighted kappa: nominal scale agreement with provision for scaled disagreement or partial credit. Psycho1

37 Altman DG. Practical Statistics for Medical Research. London: Chapman and Hall, 1991.

38 Stoller JL, Samer KJ, Toppin DI, Flores AD. Carcinoma of the esophagus. A new proposal for the evaluation of treatment. Can J Surg 1977; 20: 454-9.

39 Saunders CM, Baum M. Quality of life during treatment for cancer. Br J Hosp Med 1992; 48: 119-23.

40 Ganz PA, Schag CA, Cheng HL. Assessing the quality of life - a study in newly diagnosed breast cancer patients. J Clin Epidemiol 1990; 43: 75-86.

41 Ganz PA, Haskell CM, Figlin RA, La Soto N, Siau J. Estimating the quality of life in a clinical trial of patients with metastatic lung cancer using the Karnofsky Performance Status and the Functional Living Index - Cancer. Cancer

42 Guyatt G, Mitchell A, Irvine EJ et al. A new measure of health status for clinical trials in inflammatory bowel disease. Gastroenterology 1989; 96: 804-10.

3: 353-64.

185-203.

Bull 1968: 70: 213-20.

1988; 61: 849-56.

0 1995 Blackwell Science Ltd, British Journal of Surgely 1995, 82, 1200-1203