© U.S. Cancer Pain Relief Committee, 1999 0885-3924/99/$–see front matterPublished by Elsevier, New York, New York PII S0885-3924(99)00087-1

316 Journal of Pain and Symptom Management Vol. 18 No. 5 November 1999

Original Article

Validation of the Brief Pain Inventory in a Taiwanese Population

Luo-Ping Ger, RN, MPH, Shung-Tai Ho, MD, Wei-Zen Sun, MD, Ming-Shang Wang, PhD, and Charles S. Cleeland, PhD

Department of Medical Education and Research (L-P.G.), Veterans General Hospital-Kaohsiung, Kaohsiung, Taiwan; Department of Anesthesiology (L-P.G., S-T.H.), National Defense Medical Center/Tri-Service General Hospital, Taipei; Department of Anesthesiology (W-Z.S.), National Taiwan University Hospital, Taipei; College of Nursing (M-S.W.), National Yang-Ming University, Taipei, Taiwan, People’s Republic of China; and Pain Research Group (C.S.C.), University of Texas M. D. Anderson Cancer Center, Houston,Texas, USA

Abstract

Assessment of pain in cancer patients is very important to all health care professionals. This paper describes the development of a Taiwanese version of the Brief Pain Inventory (BPI-T) and discusses its psychometric properties in Taiwan. The BPI-T was developed from the original BPI using back-translation and committee review. A total of 534 cytologically or pathologically diagnosed cancer patients in three medical centers in Taiwan were interviewed between July 1992 and October 1997. The intraclass correlation coefficient for the test–retest reliability was 0.79 for the pain severity scale and 0.81 for the pain interference scale. The explained variance for the within-scale factor analyses was larger than 60% in both scales. The coefficient alpha for the internal reliability was 0.81 for the severity scale and 0.89 for the interference scale. Confirmatory factor analysis of the BPI-T clearly identified the same two scales (severity and interference scales) in the 299 adult patients (age between 20–64) with high education (education years

.

9) or patients at an early stage of disease. However, in the 235 nonadult patients with distant metastasis or low education patients with distant metastasis, the “most severe pain” item loaded more to the interference scale than the severity scale. Convergent validity of the pain severity was demonstrated by significant correlations with stage of disease (National Cancer Institute’s Surveillance, Epidemiology, and End Results Program [SEER]), performance status (Eastern Cooperative Oncology Group [ECOG]), and pain interference. In conclusion, interviewer-administered BPI-T was a reliable instrument for cancer pain severity and its interference in Taiwan. Additionally, it was a valid instrument on adult cancer patients with high education or patients at an early stage of disease.

J Pain Symptom Manage 1999;15:316–322.

© U.S. Cancer Pain Relief Committee, 1999.

Key Words

Brief Pain Inventory, cancer pain, pain intensity, pain interference, reliability, validity

Introduction

Assessment of pain in the cancer patient isimportant to all health care professionals be-cause failure to assess pain can lead to its un-dertreatment. In Taiwan, there are three com-monly used pain assessment tools: simple

Address reprint requests to:

Luo-Ping Ger, RN, MPH,Department of Medical Education and Research,Veterans General Hospital-Kaohsiung, 386 Ta-Chung 1st Rd., Kaohsiung, Taiwan, People’s Repub-lic of China.

Accepted for publicaton: December 24, 1998.

Vol. 18 No. 5 November 1999 Validation of a Taiwanese Version Brief Pain Inventory 317

descriptive pain intensity scale, 0–10 numericpain intensity scale, and visual analog scale(VAS). Our unpublished survey of 204 physi-cians in two medical centers in Taiwan showedthat the 0–10 numeric pain intensity scale wasthe most commonly used pain scale. The BriefPain Inventory (BPI) utilizes 0–10 numericpain intensity scales. It provides a simple way tomeasure both the intensity of pain (sensory di-mension) and the interference of pain in thepatient’s life (reactive dimension).

1

First devel-oped in English, the BPI has been widely usedin the United States

2

and has been validated inseveral other languages.

3–7

Studies using trans-lated BPI have been conducted in France,

5

Mexico,

4

Vietnam,

3

the Philippines,

4

Italy,

6

andChina.

7

It has been adopted in several coun-tries for clinical pain assessment, epidemiologi-cal studies, and in studies of the effectivenessof pain treatment.

A Chinese version of the BPI (BPI-C), devel-oped by Wang et al., is a reliable and valid mea-sure of pain and its impact on cancer patients.The BPI-C is suggested for use in studies of theepidemiology of pain, pain treatment out-come, and as a clinical pain tool in a Chinesepopulation.

7

However, the Chinese in Taiwanand most Chinese in other countries use thetraditional character set, not the simplified oneused by Chinese in China. Additionally, thereare minor differences between China and Tai-wan in the use of terminology and expression.In any case, a Taiwanese version of the BPI(BPI-T) was developed in parallel and its psy-chometric properties (reliability and validity)were examined.

Methods

Subjects

This study was restricted to hospitalized can-cer patients who had been cytologically orpathologically diagnosed with malignanciesand had cancer pain. Patients having had sur-gery within 1 month prior to interview were ex-cluded. Patients were recruited from threemedical centers in Taiwan.

From a total of 747 eligible patients, 534(71%) were available for evaluation in thisstudy. Among the 213 excluded patients, 105(14%) were too ill to be interviewed, 74 (10%)had communication barriers (listening, talking,or cognitive failure ), 28 (4%) refused to be in-

terviewed, and the other 6 (1%) had incom-plete data on the questionnaire. The patientswere interviewed at the three hospitals duringdifferent time periods: 147 patients during theperiod from July 1992 to December 1993 and135 patients during the period from November1995 to July 1996 at Tri-Service General Hospi-tal (TSGH), 152 patients during October 1996to May 1997 at Veteran General Hospital-Kaohsiung (VGH-K), and another 100 patientsduring March 1997 to October 1997 at NationalTaiwan University Hospital (NTUH).

Study Instrument

The English version of the BPI was translatedinto Chinese by a bilingual (English- and Chinese-speaking) epidemiologist working in consulta-tion with the Pain Research Group, Departmentof Neurology, University of Wisconsin–Madisionin 1990. Minor modifications were performedto preserve semantic and idiomatic equivalencein traditional Chinese characters for Mandarin-and Taiwanese-speaking people. The Taiwaneseversion of the BPI (BPI-T) was translated backto English by a second bilingual researcherwho had not seen the original English version.The English back-translation of BPI-T and theoriginal BPI were then reviewed and comparedby a committee of three English-speaking-onlypersons.

Procedure

It has been shown that comparable results areobtained from self- and interviewer-adminis-tered BPI versions for patients with adequate ed-ucation.

8

However, the same comparison cannot be made for patients who are illiterate orhave lower education level because only inter-viewer-administered surveys were conducted. Alarge portion, about 45%, of the patients were il-literate or had only an elementary school educa-tion (including some educated in Japanese);

9,10

they cannot completely understand the writtenBPI-T by themselves. For comparability of data,the interviewers interviewed all patients, literateor not.

All the interviewers received intensive train-ing in standardized interview techniques by thestudy supervisor for 1 week. They were in-structed to clarify but not to amplify the ques-tions in response to patients’ inquiries. The in-terviewer read the questions to patients, andasked them to answer or point to the number

318 Ger et al. Vol. 18 No. 5 November 1999

in the pain severity and interference scale. Arandomized subset of the patients (

n

5

40)were reinterviewed by a second interviewer onthe following days to assess test–retest reliabil-ity. Thirty-two were reinterviewed on the nextday, 5 after 2 days, 1 after 3 days, and only 2 af-ter 10 days.

Statistics

Test–retest reliabilities of each of the 2 scalesof BPI-T were evaluated by calculating the intra-class correlation coefficient (ICC) for ordinalmeasures.

11,12

ICC

$

0.70 were considered tosupport acceptable test–retest reliability.

12

Thereliability was also assessed by performing aprincipal axis factor (PAF) solution on the in-dividual scales to determine if the items of theconsidered scale were loaded on a single fac-tor.

13,14

Items that failed to load on the first fac-tor suggested that they did not assess the samedimension of pain scale or interference scale.Results were expressed as the percentage of ex-plained variance by the first factor. The inter-nal consistencies of the appropriate subscaleswere estimated by computing the Cronbach’salpha internal consistency coefficients. Cron-bach’s alpha

$

0.70 was considered as evi-dence of an acceptable internal consistency forthe considered scale.

15

Validity of the BPI-T was studied extensively.Construct validity was assessed by a PAF solu-tion with direct oblimin rotation

13,14

to assessthe 2-scale structure. Factors were retained iftheir eigenvalues (a statistical measure of theirpower to explain variation between patients)exceeded 1.0. The ability of the different fac-tors to represent the data was expressed by thepercentage of explained variance. Loading ofthe items with each factor was examined.When loading exceeded 0.45, the item wasconsidered meaningful for the considered fac-tor. Additionally, to assess convergent validity,Spearman’s correlation coefficients were used.

Results

Of the 534 patients, 341 (64%) were menand 193 (36%) were women. The mean age (

6

standard deviation) was 55.1

6

15.1 years(range 12–80). Forty-five percent of the pa-tients were either illiterate or had only elemen-tary school education, 15% completed juniorhigh school, 23% finished senior high school,

and 17% had college education. Twenty-sevenpercent of patients were retired, 25% reportedthat they were employed full-time or part-time,11% did housework at home, 9% were unem-ployed, 27% took sick leaves from work, and1% were students. The leading sites of malig-nancy were as follows: 18% lung cancer, 11%colon and rectum, 9% liver, 8% stomach, 7%breast cancer, 6% uterus cervix, and 41% oth-ers. Sixty-one percent of patients were in ad-vanced stage of disease.

Committee Reviews of the English Back-Translation of the BPI-T

The comparison between the original BPIand back-translation of BPI-T was carried outby a three-person committee. The conclusionof the review committee was that the meaningof the original BPI was accurately representedin the Taiwanese version, although there weredifferences in some sentence structures andwording. For example, the “most severe pain”item of pain severity scale in the BPI-T, thequestion “What is the level of the most severepain which you experienced in the last week?Please circle the number which best describesyour level of pain.” does not have the same sen-tence structures and wording of the “painworst” item in the BPI as “Please rate your painby circling the one number that best describesyour pain at its worst in the last week.” Addi-tionally, the anchor points for each of the painseverity items are different in wording. In theBPI-T, the anchor points refer to “0” being “nopain” and “10” being “extreme pain”, whereasin the original BPI, “0” is “no pain” and “10” is“pain as bad as you can imagine”. This differ-ence was not considered significant so as to af-fect the intent of the original survey becauseboth refer to the worst condition/pain thateach individual patient has experienced.

The BPI-T asks patients to rate their pain on0-to-10 scales at the time of responding to thequestionnaire (pain now), and also at theitems of “the most severe”, “the mildest”, and“average” in the last week. The anchor points,in each items of pain severity scale were “0”(“no pain”) and “10” (“extreme pain”). Usingthe same types of scales, patients were alsoasked to rate how their pain interferes with 7life domains, including general activity, mood,walking ability, normal work, relations withothers, sleep, and enjoyment of life. The an-

Vol. 18 No. 5 November 1999 Validation of a Taiwanese Version Brief Pain Inventory 319

chor points, in each items of interferencescale, were “0” (“not interfered”) and “10”(“completely interfered”).

Descriptive Results of the BPI-T

The mean scores and standard deviation ineach items of pain severity scale and interfer-ence scale among 534 Taiwanese patients withcancer pain are presented in Table 1. Themean score of most severe pain was the oneover midpoint on the pain severity scale. Pa-tients rated interference with work as the high-est (mean

5

6.15) and relationship with othersas the lowest (mean

5

2.70) on the interfer-ence scale.

Reliability of the BPI-T

The results of test–retest reliability, PAF solu-tion, and internal reliability of 534 subjects arepresented in Table 2. To assess the test–retestreliability of the BPI-T, the ICC for each itemand each scale were calculated separately. Thetest–retest reliability was acceptable for the 2scales. In terms of individual items, 7 itemswere acceptable but 4 items were not, includ-ing least pain, average pain, “pain now”, andwalking ability. The PAF solution performedwithin each scale showed that all the items ineach scale loaded on a single principal factor,which explained 66% of the variance for sever-ity scale and 61% for interference scale. Thecoefficient alpha was 0.81 for the severity scaleand 0.89 for the interference scale. The alpha

values for the scale if an item is deleted werecomparable to the overall alpha values for eachof the 2 scales.

Validity of the BPI-T

Confirmatory factor analysis was primarilyused to examine construct validity. This was ex-amined by a PAF solution with direct obliminrotation. The result of the factor loadings ofthe 11 items on these two factors is shown inTable 3. Two factors emerged with eigenvaluesgreater than unity and accounted for 57.3% ofthe variance in the responses of the 534 sub-jects. The seven pain interference items and“most severe pain” item made up the first fac-

Table 1

Mean Scores and Standard Deviations in Pain Severity Scale and Interference Scale among 534

Taiwanese Patients with Cancer Pain

Variables Mean SD

Pain severityMost severe 6.55 2.81Least 1.39 1.70Average 3.59 2.31Now 2.54 2.40

Pain interferenceGeneral activity 4.43 4.01Mood 5.10 3.66Walking ability 4.87 4.04Normal work 6.15 4.19Relationship 2.70 3.44Sleep 5.34 3.86Enjoyment of life 5.16 4.25

Table 2

Reliability Analysis of the Taiwanese Version BPI

Test–retest reliability (

n

5

40)Factor analysis

(

n

5

534)Internal reliability

(

n

5

534)

Intraclass correlationcoefficient

Variance%

a

Corrected item total correlation

Alpha if item deleted

Cronbach’s alpha

Pain severity 0.7857 66% 0.8143Most severe 0.8002 0.5728 0.8126Least 0.6777 0.6365 0.7823Average 0.6447 0.7353 0.7179Now 0.5483 0.6630 0.7524

Pain interference 0.8090 61% 0.8941General activity 0.7226 0.7235 0.8749Mood 0.7965 0.7241 0.8754Walking ability 0.6899 0.7673 0.8695Normal work 0.7124 0.6974 0.8783Relationship 0.8076 0.5689 0.8920Sleep 0.8068 0.6245 0.8865Enjoyment of life 0.7350 0.7508 0.8715

a

Percentage of variance explained by the first factor in principal axis factor solution. The two scales were represented in only one factor, i.e., eigen-values of the 2nd factor of the principal axis factor solution was always below 1.0.

320 Ger et al. Vol. 18 No. 5 November 1999

tor extracted (accounting for 46.8% of the to-tal variance). The second factor was limited topain severity ratings except the item “most se-vere pain” (accounting for 10.5% of the totalvariance).

For further evaluation, the subjects werestratified into four strata: (1) adult patients(age between 20–64) with high education (ed-ucation years

.

9 yr) and early stage of disease(

n

5

63); (2) adult patients with high educa-tion and distant metastasis (

n

5

90); (3) non-adult (age

,

20 or age

.

64) patients withearly stage of disease or low education patients(education years

#

9 yr) with early stage of dis-ease (

n

5

146); and (4) nonadult patients withdistant metastasis or low education patientswith distant metastasis group (

n

5

235). Theresults of factor analysis of the preceding threestrata were comparable; the item “most severepain” was loaded more on the pain severityscale than the interference scale. Therefore,these subjects were combined into one groupwhich included adult patients with high edu-cation or patients at an early stage of disease(

n

5

299).For this combined group, the first factor

identified the 7 pain interference items (ac-counting for 46.9% of the total variance) and

the second factor identified all the 4 pain se-verity items (accounting for 10.1% of the totalvariance). In the nonadult patients with distantmetastasis or low education patients with dis-tant metastasis group, the results of factor anal-ysis were similar to that of total subjects’. Theitem “most severe pain” loaded more on the in-terference scale than the pain severity scale.

In assessing convergent validity, we foundthat patients with more advanced disease hadmore pain (correlation coefficient

5

0.10,

P

5

0.02), patients with worse Eastern CooperativeOncology Group (ECOG) scores had higherpain (correlation coefficient

5

0.33,

P

,

0.01),and patients with more pain had more interfer-ence of pain in their life (correlation coeffi-cient

5

0.58,

P

,

0.01).

Discussion

The mean value and standard deviation forthe most severe pain in this study populationwere comparable with other BPI studies on can-cer patients. However the standard deviation ofthe worst pain in the Chinese study was consid-erably smaller than this and other studies.

7,16,17

This might be due to the fact that the pain rat-ings were made for the last 24 hours in the Chi-

Table 3

Factor Loadings Using Principal Axis Factoring with Direct Oblimin Rotation

All patients(

n

5

534)

Nonadult

a

patients withdistant metastasis or loweducation

b

patients withdistant metastasis

(

n

= 235)

Adult

c

patients with high education

d

or patients at an early

stage of disease(

n

5

299)

Factor I Factor II Factor I Factor II Factor I Factor II

Eigen values 5.14 1.16 4.97 1.22 5.16 1.11Variance 46.8% 10.5% 45.2% 11.1% 46.9% 10.1%Pain interference

General activity 0.7767

2

0.0045 0.7289 0.0351 0.7937

2

0.0097Mood 0.8011

2

0.0610 0.7528

2

0.0154 0.8065

2

0.0641Walking ability 0.8640

2

0.0901 0.8470

2

0.1161 0.8449

2

0.0375Normal work 0.7328 0.0260 0.7700

2

0.0222 0.6566 0.1028Relationship 0.5758 0.0373 0.5453 0.1174 0.5993

2

0.0234Sleep 0.6254 0.0893 0.7040

2

0.0317 0.5095 0.2305Enjoyment of life 0.8084

2

0.0259 0.7968

2

0.0125 0.7923

2

0.0094Pain severity

Most severe 0.4855 0.3439 0.5447 0.2633 0.3554 0.4765Least

2

0.0368 0.7771

2

0.0670 0.7899

2

0.0114 0.7695Average 0.2449 0.6347 0.2738 0.6253 0.2053 0.6673Now

2

0.0584 0.8713

2

0.0139 0.8401

2

0.1099 0.8876

a

Age below 20 years or age greater than 64 years.

b

Education years equal or below 9 years.

c

Age between 20 and 64 years.

d

Education years greater than 9 years.

Vol. 18 No. 5 November 1999 Validation of a Taiwanese Version Brief Pain Inventory 321

nese study, but pain ratings were made over theprevious week in this and other studies.

For assessing test–retest reliability, a subset ofpatients were reinterviewed. The second inter-views were conducted a short time after the ini-tial interview (the majority on the next day) be-cause of factors such as the intrinsic fluctuatingnature of some cancer pain syndromes, pro-gressive prognosis of cancer, and confoundingeffect of therapeutic intervention. The result oftest–retest reliability showed that the item“most severe pain” had the highest stability and“pain now” was lowest in the scale of pain sever-ity. This result was congruous with the knowl-edge that cancer pain is quite variable.

4

There-fore, test–retest reliability of the “pain now”item was not suggested in the BPI-T. In addi-tion, least pain, average pain, and walking abil-ity items needed further evaluation if we couldcontrol the confounding effect of therapeuticintervention.

In the internal reliability analysis, both coef-ficient alphas of the pain severity scale and thepain interference scale were above 0.80, dem-onstrating the relatively good internal consis-tencies of the scales.

15

Additionally, alpha val-ues were not higher than 0.90, demonstratingthe items without redundancy.

18

Correcteditem total correlations were all well above 0.20.This indicated that there was no need to elimi-nate or rewrite any item from the scale.

18

Thealpha values for the scale did not increase sig-nificantly if an item was deleted. Therefore, wedid not need to discard any items from thesetwo scales.

18

Regarding the result of factor analysis, twocommon factors have been extracted whichconsistently demonstrated in other languageversions of the BPI except the “most severepain” item.

4,6,7,19

The “most severe pain” itemloaded more on the pain interference scalethan the pain severity scale, especially in thenonadult (age

,

20 yr or age

.

64 yr) patientswith distant metastasis or low education (edu-cation

#

9 yr) patients with distant metastasis.This was quite different from all other coun-tries’ results.

4,6,7,19 However, the “most severepain” item loaded more on the factor pertain-ing to the pain severity scale than the pain in-terference scale in the adult patients (age $ 20and age , 65 yr) with high education (. 9 yr)or patients at an early stage of disease. This wascompatible with Wisconsin, Vietnamese, Mexi-

can, and Philippine results of factor analysis.4

Those studies showed that the pain interfer-ence items make up the first factor extracted,with additional contribution from the rating of“worst pain”. The second factor is limited painseverity ratings.

Four possibilities might explain this discrep-ancy in the factor analysis. First, the most se-vere pain was more closely related to pain in-terference in younger patients with distantmetastasis (, 20 yr), older patients (. 64 yr)with distant metastasis, or low education (edu-cation # 9 yr) patients with distant metastasisthan the adult patients with high education orpatients at an early stage of disease. Althoughour patients had similar age and education dis-tribution as patients in China, their results offactor analysis were consistent with other lan-guage versions of the BPI. The reason that thisstudy shows a different analysis result on the“worst pain” item may be because there weremore patients at an advanced stage of disease(61%) in this study than those in China (49%).

Second, the self-administered BPI survey hadthe selection effect of literate subjects, who weremore clearheaded or less ill. Almost all otherlanguage versions of the BPI were self-adminis-tered except all the patients in Vietnam3 andsome patients (too ill or illiterate) in China.7

However, all our patients were questioned by in-terviewers. Therefore, a larger number of lesseducated patients with distant metastasis, about199 patients (37%), were included in this study.These patients might have more cognitive diffi-culties. The reliability results showed that thedata were of quite good quality, and less edu-cated patients with distant metastasis wouldhave different factor structure than other coun-tries’ patients.

The third possibility is that the cross transla-tion was somewhat different in the “worst pain”item. Comparison between the BPI-T and theoriginal BPI indicated that the meaning of“most severe pain” might be somewhat differ-ent from the original “worst pain”. Althoughthe review committee concluded that the con-tent validity of the original BPI was preserved,we could not exclude this potential source thatmight have contributed to the discrepancycompared to other countries.

Fourth, our patients were rating their painclose to the time of initial diagnosis (49%within 90 days). The newly diagnosed patients

322 Ger et al. Vol. 18 No. 5 November 1999

might just perceive that “most severe pain” andpain interference were the impact of their “dis-ease”. Our previous study showed that most ofthe patients (63%) did not know their diagno-sis, especially the old (75%) or less educated(72%) patients.9 It is possible that newly diag-nosed patients have a different pain scalestructure.

In conclusion, the interviewer-administeredTaiwanese version of BPI was a reliable instru-ment for cancer pain severity and its interfer-ence. Additionally, it was a valid instrument foradult patients with high education or patientsat an early stage of disease. The BPI used for il-literate patients or those with little educationneeds further study, especially in developingcountries.

AcknowledgmentsPortions of this research were supported by

Grants DOH85-TD-054, DOH86-TD-081, andVGHKS88-34. We thank Dr. Paul Lee for hisvaluable comments and linguistic review.

References1. Cleeland CS. Measurement of pain by subjective

report. In: Chapman CR, Loeser JD, eds. Advancesin pain research and therapy, vol 12. New York:Raven, 1989:391–403.

2. Cleeland CS, Gonin R, Hatfield AK, et al. Painand its treatment in outpatients with metastatic can-cer. N Engl J Med 1994;330:592–596.

3. Cleeland CS, Ladinsky JL, Serlin RC, et al. Multi-dimensional measurement of cancer pain: compari-sons of US and Vietnamese patients. J Pain SymptomManage 1988;3:23–27.

4. Cleeland CS. Assessment of pain in cancer: mea-surement issues. In: Foley KM, eds. Advances in painresearch and therapy, vol 16. New York: Raven,1990:47–55.

5. Larue F, Carlier AM, Brasseur L, et al. Assessingthe prevalence and severity of cancer pain in France.BMJ 1991;310:1034–1037.

6. Caraceni A, Mendoza TR, Mencaglia E, et al. Avalidation study of an Italian version of the BriefPain Inventory. Pain 1996;65:87–92.

7. Wang XS, Mendoza TR, Gao SZ, et al. The Chi-nese version of the Brief Pain Inventory (BPI-C): itsdevelopment and use in a study of cancer pain. Pain1996;67:407–416.

8. Cleeland CS. Pain assessment in cancer. In: Os-oba D, ed. Effect of cancer on quality of life. BocaRaton, FL: CRC Press, 1991:293–305.

9. Ger LP, Ho ST, Chiang HH, et al. Cancer pa-tients’ knowledge of their diagnosis. J Formos MedAssoc 1996;95:605–611.

10. Ger LP, Ho ST, Wang JJ, et al. The prevalenceand severity of cancer pain: a study of newly diag-nosed cancer patients in Taiwan. J Pain SymptomManage 1998;15:285–293.

11. Deyo RA, Diehr P, Patrick DL. Reproducibilityand responsiveness of health status measures: statis-tics and strategies for evaluation. Control Clin Trials1991;12:142s–158s.

12. Shrout PE, Fleiss JL. Intraclass correlations: usesin assessing rater reliability. Psychol Bull 1979;86:420–428.

13. Gorsuch RL. Factor analysis. Philadelphia: Saun-ders, 1974.

14. Mulaik SA. The foundations of factor analysis.New York: McGraw-Hill, 1972.

15. Nunnally JC. Psychometric theory, 2nd ed. NewYork: McGraw-Hill, 1978.

16. Serlin RC, Mendoza TR, Nakamura Y, et al.When is cancer pain mild, moderate or severe?Grading pain severity by its interference with func-tion. Pain 1995;61:277–284.

17. Cleeland CS, Nakamura Y, Mendoza TR, et al.Dimensions of the impact of cancer pain in a fourcountry sample: new information from multidimen-sional scaling. Pain 1996;67:267–273.

18. Streiner DL, Norman GR. Health measurementscales: a practical guide to their development anduse, 2nd ed. New York: Oxford University Press, 1995.

19. Romero J, Plancarte R, Heidrich G, et al. Firsttime opioid use for the control of cancer in a previ-ously opioid naive population: a comparison of twosamples (abstract). American Pain Society 10th An-nual Scientific Meeting, New Orleans, Nov. 7–10, 1991.