A Rapid Staging System for Predicting Mortality From HIV-Associated Community-Acquired Pneumonia

DOI 10.1378/chest.123.4.1151 2003;123;1151-1160Chest

Deloria-Knoll, Joan S. Chmiel, Laura Phan and Paul R. YarnoldAhsan M. Arozullah, Jorge Parada, Charles L. Bennett, Maria

*Community-Acquired PneumoniaMortality From HIV-Associated A Rapid Staging System for Predicting

http://chestjournal.chestpubs.org/content/123/4/1151.full.html

services can be found online on the World Wide Web at: The online version of this article, along with updated information and

ISSN:0012-3692)http://chestjournal.chestpubs.org/site/misc/reprints.xhtml(

written permission of the copyright holder.this article or PDF may be reproduced or distributed without the priorDundee Road, Northbrook, IL 60062. All rights reserved. No part of Copyright2003by the American College of Chest Physicians, 3300Physicians. It has been published monthly since 1935.

is the official journal of the American College of ChestChest

© 2003 American College of Chest Physicians by guest on July 12, 2011chestjournal.chestpubs.orgDownloaded from


http://chestjournal.chestpubs.org/site/misc/reprints.xhtml

http://chestjournal.chestpubs.org/

A Rapid Staging System for PredictingMortality From HIV-AssociatedCommunity-Acquired Pneumonia*

Ahsan M. Arozullah, MD, MPH; Jorge Parada, MD, MPH;Charles L. Bennett, MD, PhD; Maria Deloria-Knoll, PhD; Joan S. Chmiel, PhD;Laura Phan, MPH; and Paul R. Yarnold, PhD

Study objective: Community-acquired pneumonia (CAP) accounts for an increasing proportion of thepulmonary infections in individuals with HIV infection. During the mid-1990s, hospital mortality ratesfor HIV-associated CAP ranged from 0 to 28%. While hospital differences in case mix may account formortality rate variation, few methods to evaluate illness severity for HIV-associated CAP have beenreported previously. The study objective was to develop a staging system for categorizing mortalityrisk of patients with HIV-associated CAP using information available prior to hospital admission.Design/setting/patients: Retrospective medical records review of 1,415 patients hospitalized withHIV-associated CAP from 1995 to 1997 at 86 hospitals in seven metropolitan areas.Measurements: In-patient mortality rate.Results: Hierarchically optimal classification tree analysis was used to develop a preadmission stagingsystem for predicting inpatient mortality. The overall inpatient mortality rate was 9.1%. Thesignificant predictors of mortality included the presence of neurologic symptoms, respiratory rate> 25 breaths/min, and creatinine > 1.2 mg/dL. The model identified a five-category staging system,with the mortality rate increasing by stage: 2.3% for stage 1, 5.8% for stage 2, 12.9% for stage 3, 22.0%for stage 4, and 40.5% for stage 5. The classification accuracy of the model was 85.2%.Conclusions: Our staging system categorizes inpatient mortality risk for patients with HIV-associatedCAP using three routinely available variables. The staging system may be useful for guiding clinicaldecisions about the intensity of patient care and for case-mix adjustment in future studies addressingvariation in hospital mortality rates. (CHEST 2003; 123:1151–1160)

Key words: community-acquired pneumonia; HIV; hospital mortality

Abbreviations: CAP � community-acquired pneumonia; CTA � classification tree analysis; ICD � International Classifi-cation of Diseases, ninth revision; PCP � Pneumocystis carinii pneumonia

P atients with HIV infection are at increased riskfor pulmonary infections including Pneumocystis

carinii pneumonia (PCP), tuberculosis, and bacterialpneumonia.1,2 For the past several years, the inci-dence of bacterial pneumonia exceeded the inci-dence of PCP among patients with HIV infection.3Since the introduction of protease inhibitors, there

has been a 75% decline in the incidence of AIDS-defining opportunistic infections. While community-acquired pneumonia (CAP) has been responsible fora majority of the pulmonary-related morbidity andmortality in patients with HIV,4–6 the majority ofHIV-associated pneumonia quality-of-care studieshave focused on PCP.7–10

Any assessment that evaluates hospital perfor-mance for HIV-associated pneumonia must take intoaccount differences in patient factors associated with*From the VA Chicago Healthcare System (Dr. Arozullah),

Westside Division, Department of Medicine, University of Illi-nois College of Medicine, Chicago; Hines VA Hospital (Dr.Parada), Hines, and Department of Medicine, Loyola UniversityMedical School, Maywood; VA Chicago Healthcare System (Dr.Bennett), Lakeside Division, Chicago; Department of Medicine(Ms. Phan), Northwestern University Medical School, Chicago;Department of Preventive Medicine (Drs. Deloria-Knoll andChmiel), Northwestern University Medical School, Chicago; andDepartment of Psychology (Dr. Yarnold), University of Illinois atChicago, Chicago, IL.This study was supported in part by a grant from the NationalInstitute of Drug Abuse (5R01 DA10628-02) and the HealthServices Research Division of the Department of Veterans Affairs.

Dr. Arozullah and Dr. Parada are supported by Research CareerDevelopment Awards from the Health Services Research andDevelopment Service of the Department of Veterans Affairs.Manuscript received March 27, 2002; revision accepted October8, 2002.Reproduction of this article is prohibited without written permis-sion from the American College of Chest Physicians (e-mail:[email protected]).Correspondence to: Ahsan M. Arozullah, MD, MPH, VA ChicagoHealthcare System, Westside Division (151WS), 820 S DamenAve, Chicago, IL 60612; e-mail: [email protected]

www.chestjournal.org CHEST / 123 / 4 / APRIL, 2003 1151



mortality. The prior quality-of-care studies that eval-uated HIV-associated PCP or even those that eval-uated non–HIV-associated CAP have incorporateddisease-specific and condition-specific staging sys-tems. Fine and colleagues11,12 developed a pneumo-nia severity index for non–HIV-associated CAP thatwas based on 19 clinical, laboratory, and radio-graphic variables. We previously developed similardisease-specific staging systems for HIV-associatedPCP that was diagnosed in the first and seconddecades of the AIDS epidemic.8–10 These PCPstaging systems were based on clinical factors such asage, history of AIDS, and weight loss, as well aslaboratory factors such as alveolar-arterial oxygengradient and serum hemoglobin and albumin levels.The purpose of this study was to develop and validatea disease-specific and condition-specific severity-of-illness staging system for HIV-associated CAP. Wealso compared our staging system with previousmodels developed for non–HIV-associated CAP andHIV-associated PCP.

Materials and Methods

Sampling of Patients and Hospitals

Medical records were abstracted for patients with HIV infec-tion with confirmed or probable CAP who received medical careat a study institution between January 1, 1995, and December 31,1997. Study institutions included 86 public and private hospitalsin seven metropolitan areas, including New York City, LosAngeles, Miami, Chicago, Seattle, Nashville/Memphis, and Phoe-nix/Tucson. During 1996–1997, New York City had the largestnumber of reported AIDS patients in the United States, with LosAngeles having the second, Miami having the fifth, and Chicagohaving the eighth.13 The institutional review board of eachparticipating hospital approved the study protocol.

We used a sampling methodology, similar to previous studies,that employed two hierarchical levels: hospitals within metropol-itan areas and patients within hospitals.9,10,14 The number ofrandomly selected patient charts reviewed at each hospitalrelative to the charts reviewed in each metropolitan area wasroughly proportional to the square root of the individual hospitalcaseload divided by the metropolitan area caseload. The numberof charts reviewed within each hospital was stratified by patientdischarge status to approximate the actual hospital mortality rate.

Registered nurses experienced in caring for patients with HIVand utilization review methods were trained to perform retro-spective chart reviews of patients identified by medical informa-tion system analysts at study hospitals. All discharges includingInternational Classification of Diseases, ninth revision(ICD-9) codes for bacterial pneumonia (481– 486) and HIV-related disease (042– 044) were screened. Patients were ex-cluded if they had any one of the following criteria: age � 18years, transfer from another acute care hospital inpatient service,cytologically proven PCP in the previous 30 days, culture-provenpulmonary tuberculosis or CAP within the past 30 days, orhospitalization for any other reason within the past 30 days.Patients with the following cancers were also excluded: adeno-carcinoma with unknown primary, bladder, brain, colon, esoph-agus, gastric or stomach, larynx, leukemia, liver, lung, oat cell, or

small cell, oropharynx, ovary, pancreas, sarcoma (other thanKaposi). Patients with a history of these cancers were excluded inorder to minimize the misclassification of abnormal findings onchest radiography.

Data abstracted from medical charts included the following:patient sociodemographics; insurance status; HIV-related andnon–HIV-related coexisting illnesses; cigarette, alcohol, and druguse history; preadmission use of antiretroviral and prophylacticmedications; T-cell count; initial vital signs, arterial blood gas,and laboratory data; treatment medications received; principaland secondary diagnostic and procedure codes; length of stay;discharge status. Neurologic symptoms were considered presentif a physician noted that the patient was either comatose,confused, and/or had symptoms of neurologic change on day 1 or2 of hospital admission. These three categories were combinedbased on chart abstractor observations that the majority ofsymptoms of neurologic change noted were mental statuschanges. Two physicians trained in quality assurance for the studymaintained data quality by reviewing completed medical chartabstraction forms. The physician overreaders categorized � 1%of entries as possibly inaccurate.

Statistical Analysis

Univariate associations between measured attributes and in-hospital mortality status were evaluated using univariate optimaldiscriminant analysis. A multivariate nonlinear model was ob-tained via hierarchically optimal classification tree analysis (CTA)using an algorithm that maximized mean sensitivity.10,15,16 Meansensitivity is the average of the sensitivity for patients classified asdead and for those classified as alive. Parametric analyses wereinappropriate due to violations of assumed distributional features,and traditional nonparametric analyses were inappropriate due tothe many tied data values. Jackknife validity analysis was con-ducted to assess the potential generalizability of the findings. Forcharacteristics with stable effect strength in jackknife validityanalysis, the optimal cut points identified would be expected tocross-generalize at the jackknife estimate if they were used toclassify independent random patient samples. In contrast, ifstatistically significant characteristics were unstable in jackknifevalidity analysis, these characteristics would be expected to bepredictive of mortality, but using model cut points different thanthose identified. For the CTA model, a sequentially rejectiveSidak Bonferroni-type multiple comparisons procedure was usedto ensure an experiment-wise type I error rate of p � 0.05.15

Results

There were 1,415 patients with HIV-associatedCAP who met our eligibility criteria with an in-hospital mortality rate of 9.1%. Hospital-specificmortality rates for hospitals with � 10 eligible pa-tients varied from 0 to 28%. Patient demographicand clinical characteristics are given in Tables 1, 2.The mean age was 39.8 years, with 78% of thepatients being male and 47% being African Ameri-can. Over half of the patients had a prior AIDS-related diagnosis; however, only 43% of patientsreported receiving any antiretroviral therapy prior tohospital admission, and only 15% reported receivingprotease inhibitors. Nearly one third (32%) of thepatients had a history of non-PCP pneumonia, while

1152 Clinical Investigations



25% had a history of PCP. Nearly 21% of thepatients had neurologic symptoms, including 1.6%who were comatose and 8.0% who were confused.

Univariate Analysis

Univariate associations between preadmissioncharacteristics and in-hospital mortality identifiedseveral characteristics that were significantly associ-ated (p � 0.05) with an increased risk of inpatientmortality (Tables 3, 4). These included factors suchas age � 32 years, current or former alcohol use,history of renal disease, and presence of neurologic

Table 2—Clinical Characteristics of Patients WithConfirmed or Suspected CAP

VariablesPatients,

No.SummaryValues*

Coexisting illness 1,415Liver disease 6.7Renal disease 4.0Diabetes mellitus 3.6Congestive heart failure 2.4Cerebrovascular disease 1.5

Factors related to previous HIV treatmentPrior AIDS-related diagnosis 1,415 57.0Prior PCP prophylaxis 1,415 50.7Any prior antiretroviral therapy 1,415 43.3Prior zidovudine use 1,415 24.2Prior M avium complex 1,415 15.9Prior protease inhibitor use 1,415 14.7CD4† lymphocyte count 1,197

� 50 cells/�L 47.050–100 cells/�L 14.3101–200 cells/�L 16.0201–500 cells/�L 18.7�500 cells/�L 3.9

Factors related to respiratory statusHistory of non-PCP pneumonia 1,415 31.9History of PCP 1,415 25.0History of COPD 1,415 13.1History of tuberculosis 1,415 11.5Pleural effusion 1,415 7.8Cigarette smoking 1,230

Current 63.3Never 25.5Former 11.2

Percent oxygen saturation† 1,165 93.7 (6.7)Partial pressure of oxygen,† mm Hg 970 80.3 (38.8)Respiratory rate, breaths/min 1,398 25 (7)

Initial vital signs and laboratory valuesLowest systolic BP, mm Hg 1,406 104.1 (16.8)Lowest diastolic BP, mm Hg 1,396 62.0 (12.4)Highest heart rate, beats/min 1,409 111.5 (20.0)WBC count, 1,000 cells/�L 1,399 7.6 (5.5)Hematocrit, mg percent 1,377 33.1 (7.2)Sodium, mg/dL 1,394 135.9 (5.4)Glucose, mg/dL 1,352 105.1 (41.3)Blood urea nitrogen, mg/dL 1,388 18.8 (19.5)Creatinine, mg/dL 1,392 1.3 (1.8)Albumin, g/dL 1,122 3.1 (0.8)

Wasting 1,415 37.0Factors related to neurologic status

Neurologic symptoms‡ 1,415 20.9Symptoms of neurologic change 1,415 15.6Confused 1,415 8.0Comatose 1,415 1.6

In-hospital mortality rate 1,415 9.1

*Summary values given are mean value (SD) or %, in which case thesummary value given is the No. of patients with the characteristicdivided by No. for that characteristic.

†Fraction of inspired oxygen was known for most patients(n � 1,097). However, percent oxygen saturation and partial pres-sure of oxygen given do not take fraction of inspired oxygen intoaccount.

‡Neurologic symptoms were considered present if a physician notedthat the patient was either comatose, confused, and/or had symp-toms of neurologic change on day 1 or day 2 of the hospitaladmission.

Table 1—Demographics of Patients With Confirmed orSuspected CAP

VariablesPatients,

No.SummaryValues*

Age, yr 1,415 39.8 (8.5)Male gender, % 1,415 78.1Race 1,372

African American 46.7White 34.0Hispanic 16.8Other 2.4

Preadmission residence 1,300Home/apartment 78.3Other living situation 7.6Homeless 6.2Nursing home 3.8Hotel/single resident occupancy 2.5Prison 1.6

Gay/bisexual men 562 56.2Noninjection drug use 922 36.6Injection drug use 895 48.2Alcohol use 1,159

Current 48.1Never 39.4Former 12.5

Insurance 1,415Medicaid 54.6None identified/other 18.2Private 16.7Medicare 7.2Self pay 3.3

Metropolitan area 1,415New York City 26.0Chicago 20.6Los Angeles 15.1Miami 13.2Seattle 11.5Nashville/Memphis 8.0Phoenix/Tucson 5.5

Hospital type 1,415Not-for-profit 51.8County 30.0Church operated 9.8For-profit 8.4

*Summary values given are % or mean (SD). Data presented as % arethe No. of patients with the characteristic divided by the No. for thatcharacteristic.




symptoms. Previous HIV treatment and currentimmune status characteristics that were significantlyassociated with increased mortality included priorAIDS-related diagnosis, prior use of Mycobacteriumavium complex prophylaxis, CD4 lymphocyte count� 50 cells/�L, and WBC count � 5,750 cells/�L.Noninjection drug use (p � 0.03) and prior antiret-roviral therapy (p � 0.008) were significantly associ-ated with decreased mortality. Prior PCP prophylaxiswas not significantly associated with mortality(p � 0.15). All of these characteristics except age,CD4 lymphocyte count, and WBC count were stablein jackknife validity analysis, implying that the cutpoints identified are expected to cross-generalizewith comparable effect strength if used to classifyindependent random patient samples.

Respiratory status variables significantly associatedwith an increased risk of mortality included currentcigarette smoking, oxygen saturation � 90%, andrespiratory rate � 25 breaths/min. Vital signs andlaboratory values significantly associated with in-creased mortality included systolic BP � 95 mm Hg,diastolic BP � 60 mm Hg, heart rate � 112 beats/min, hematocrit � 31 mg/dL, sodium � 132 mg/dL,glucose � 71 mg/dL, BUN � 20.5 mg/dL, creatinine� 1.2 mg/dL, and albumin � 3.1 g/dL. All of thesecharacteristics, except current cigarette use and re-

spiratory rate, were unstable in jackknife validityanalysis. These unstable characteristics are expectedto be predictive of mortality when used to classifyindependent random samples, but the optimal cutpoints may differ from those identified in Tables 3, 4.

CTA Model

A nonlinear multivariate model for predictinginpatient mortality was created using hierarchicallyoptimal CTA (Fig 1). The presence of neurologicsymptoms was selected as the initial (root) attributein the CTA model because it had the highest effectstrength for predicting mortality that was stable injackknife validity analysis. The other significant vari-ables used in the CTA model were respiratory rateand creatinine level. While creatinine level wasunstable in jackknife validity analysis for the entiresample, it provided stable mortality estimates whenused to classify patients in the subset without neu-rologic symptoms and � 25 breaths/min.

Using the CTA model to classify individual pa-tients is straightforward. Consider a patient withoutneurologic symptoms and respiratory rate of 20breaths/min. Starting with the first node, since thepatient had no neurologic symptoms, the left branchis appropriate. At the second node, the left branch is

Table 3—Univariate Associations of Demographics With In-hospital Mortality

Variables

Optimal Discriminant Analysis Training Analysis

Optimal Cut Point†Patients,

No.Mortality Rate,

%Effect

Strength‡ p Value

Age, yr* � 32 280 3.6 13.2 0.02� 32 1,135 10.4

Gender Male 1,105 9.4 3.5 0.39Female 310 7.7

Race White, Hispanic 698 10.3 6.9 0.31African American, other 674 8.0

Preadmission residence* Home/apartment, other living situation, homeless, or prison 1,333 8.4 7.4 0.25Nursing home, hotel/single-resident occupancy, or no data 82 19.5

Gay/bisexual men Yes 316 10.4 15.3 0.06No 246 5.7

Noninjection drug use Yes 337 5.3 12.9 0.03No 464 9.0

Alcohol use Current, former 858 11.0 14.1 0.009Never, no data 557 6.1

Insurance* Private, self pay, none identified/other 735 9.5 0.1 0.99Medicaid, Medicare 484 8.5

Metropolitan area New York City, Seattle, Phoenix/Tucson 722 11.6 16.1 0.01Los Angeles, Chicago, Miami, Nashville/Memphis 693 6.4

Hospital type Not-for-profit 728 10.2 7.2 0.25County, church operated, for-profit 668 7.8

*Characteristic had an effect strength that was lower in jackknife validity analysis vs training analysis, suggesting that the level of classificationaccuracy achieved in training may not cross-generalize when it is used to classify an independent random sample.

†Selected by optimal discriminant analysis for each characteristic because it maximized training sample sensitivity effect strength.‡Effect strength for sensitivity of the model, a standardized measure indicating the percent of the theoretically possible improvement inclassification accuracy—beyond what is expected by chance—that is achieved by the model. On this measure, 0 � classification accuracyexpected by chance and 100 � perfect (errorless) classification accuracy.




appropriate since the patient had a respiratory rate� 25 breaths/min and the model classifies the pa-tient as alive. Of the 748 patients classified into thesame end point via this particular branch of theclassification tree, 731 were correctly classified re-sulting in a predictive value of 97.7%, with theprobability of death equal to 1 to 0.977, or 0.023,yielding a mortality rate of 2.3%. If the patient’srespiratory rate was � 25 breaths/min and creatininewas � 1.2 mg/dL, then the third-from-the-left endpoint would be appropriate and the model wouldclassify the patient as “dead.” Note that, of 100patients classified into this latter end point, 22patients were correctly classified, resulting in a mor-tality rate of 22.0%.

Of the 1,415 patients, the CTA model classified1,403 patients (99.2%) with nonmissing data, ofwhich 1,196 patients (85.2%) were classified cor-rectly (Table 5). The CTA model yielded relativelystrong effect strength for sensitivity of 45.5, indicat-ing that the model provided 45.5% of the theoreticalclassification improvement possible to achieve beyond

chance. For effect strength, 0 is equal to the classifica-tion accuracy expected by chance, and 100 is equal toperfect (errorless) classification accuracy. All modelperformance indexes and end point predictive valueswere closely approximated in jackknife and bootstrapvalidity analysis, indicating model performance stabilityfor a 50% reduced sample size (Table 6).

Severity of Illness Staging System

The severity of illness staging system derived fromthe CTA model is displayed in Table 7. The predic-tion end points from the CTA model were orderedfrom lowest to highest mortality rate, such that stageis an ordinal scale of severity of illness on whichincreasing integers reflect an increasing likelihood ofmortality (Fig 1). Mortality rates for stage 1 throughstage 5 were 2.3%, 5.8%, 12.9%, 22.0%, and 40.5%,respectively. Among patients with neurologic symp-toms, those in stage 5 with a respiratory rate � 25breaths/min had a threefold higher mortality rate(40.5% vs 12.9%) compared to patients in stage 3

Figure 1. Schematic illustration of the CTA model for predicting mortality. Circles represent nodes,(decision points), arrows indicate branches (decision paths), and rectangles represent prediction endpoints. (As indicated, the model classifies patients having a given attribute profile as either dead oralive.) Numbers underneath nodes give the generalized type I error for the node. Numbers/wordsadjacent to arrows indicate the value of the cut point or category for each node.




Table 4—Univariate Associations of Clinical Characteristics With In-hospital Mortality

Variables



No.Mortality Rate,

%Effect

Strength‡ p Value

Coexisting illnessLiver disease Yes 95 14.7 4.6 0.06

No 1,320 8.6Renal disease Yes 57 21.1 5.9 0.003

No 1,358 8.5Diabetes mellitus* Yes 51 11.8 1.2 0.63

No 1,364 8.9Congestive heart failure Yes 34 14.7 1.6 0.38

No 1,381 8.9Cerebrovascular disease* Yes 21 9.5 0.1 0.99

No 1,394 9.0Factors related to previous HIV treatment

Prior AIDS-related diagnosis Yes 806 11.0 13.8 0.005No 609 6.4

Prior PCP prophylaxis Yes 718 10.2 6.9 0.15No 697 7.9

Any prior antiretroviral therapy Yes 613 6.7 12.4 0.008No 802 10.8

Prior zidovudine use Yes 343 7.6 4.3 0.27No 1,072 9.5

Prior M avium complex prophylaxis Yes 225 16.0 13.4 � 0.001No 1,190 7.7

Prior protease inhibitor use Yes 194 5.2 6.5 0.06No 1,221 9.7

CD4† lymphocyte count,* cells/�L � 50 563 11.7 22.9 � 0.001� 50 634 4.9

Factors related to respiratory statusHistory of non-PCP pneumonia Yes 451 8.2 3.3 0.48

No 964 9.4History of PCP Yes 353 9.4 0.9 0.83

No 1,062 9.0History of COPD Yes 185 6.5 4.1 0.22

No 1,230 9.4History of tuberculosis Yes 170 6.5 3.8 0.26

No 1,245 9.4Pleural effusion Yes 111 9.9 0.8 0.86

No 1,304 9.0Cigarette smoking Current 185 17.3 13.1 0.007

Former, never 1,230 7.8Preadmission cough duration,* d � 2 273 11.0 8.2 0.30

� 2 1,009 7.3Percent oxygen saturation* � 90 161 19.9 18.6 � 0.001

� 90 1,004 7.2Partial pressure of oxygen,* mm Hg � 60 205 17.1 15.5 0.14

� 60 765 8.5Respiratory rate, breaths/minute � 25 910 4.2 38.4 � 0.001

� 25 488 18.0Initial vital signs and laboratory values

Lowest systolic BP,* mm Hg � 95 380 17.1 27.4 � 0.001� 95 1,026 5.8

Lowest diastolic BP,* mm Hg � 60 777 12.0 21.9 � 0.001� 60 619 5.8

Highest heart rate,* beats/min � 112 862 5.2 28.3 � 0.001� 112 547 15.0

WBC count,* 1000 cells/�L � 5,750 666 13.1 24.2 � 0.001� 5,750 733 5.2

Hematocrit,* mg percent � 31 506 15.4 29.3 � 0.001� 31 871 5.2

Sodium,* mg/dL � 132 220 16.8 14.9 0.002� 132 1,174 7.6




with respiratory rate � 25 breaths/min. Among pa-tients without neurologic symptoms and a respiratoryrate � 25 breaths/min, those in stage 4 with creati-nine � 1.2 mg/dL had nearly a fourfold highermortality rate (22.0% vs 5.8%) compared to patientsin stage 2 with creatinine � 1.2 mg/dL.

Comparison With HIV-Associated PCP andNon–HIV-Associated CAP Severity ofIllness Models

We compared the effect strength for sensitivity ofour CTA model to previous CTA models developedfor predicting mortality for HIV-associated PCP.10,16

One previous PCP mortality model, developed using1,339 patients with PCP hospitalized between 1987and 1990, had an effect strength for sensitivity of21.2.16 Another PCP mortality model, developedusing 1,660 patients with PCP hospitalized between1995 and 1997, had an effect strength for sensitivityof 33.1.10 Compared to these PCP mortality CTAmodels, our CTA model had superior effect strength

for sensitivity of 45.5, which represents a 115% and37.5% improvement, respectively.10,16

We also compared CTA model performance withpneumonia severity index performance in our sam-ple of patients with HIV-associated CAP.11,12 Sincethe pneumonia severity index was developed for thegeneral population, it is reasonable to expect that adifferent set of threshold values might be applicableto the present population of patients with HIV.17

Accordingly, we used optimal discriminant analysisto identify the optimal pneumonia severity indexscore cut point for accurately classifying patientmortality status in our sample. The cut point value

Table 5—Classification Accuracy of the CTA Model*

Actual Status

Patient’s Predicted Status

Alive Dead Total

Alive 1,123 153 1,276Dead 54 73 127Total 1,177 226 1,403

*Overall classification accuracy � (1,123 � 73)/1,403 � 85.2%.

Table 6—CTA Model Performance in Training andBootstrap Analysis

VariablesTrainingAnalysis

BootstrapAnalysis*

Total classification accuracy, % 85.2 84.9 � 1.0Sensitivity (dead patients), % 57.5 57.8 � 4.5Sensitivity (alive patients), % 88.0 87.6 � 0.9Mean sensitivity, % 72.7 72.7 � 2.3Effect strength for sensitivity† 45.5 45.4 � 4.6

*Bootstrap validity analysis is based on 1,000 iterations of a 50%resample; summary indices provided include the mean and SD.

†Effect strength for sensitivity of the model is a standardized measureindicating the percentage of the theoretically possible improvementin classification accuracy—beyond what is expected by chance—that is achieved by the model. On this measure, 0 � classificationaccuracy expected by chance and 100 � perfect (errorless) classifi-cation accuracy.

Table 4—Continued

Variables



No.Mortality Rate,

%Effect

Strength‡ p Value

Glucose,* mg/dL � 71 74 29.7 13.5 0.02� 71 1,278 8.0

Blood urea nitrogen,* mg/dL � 20.5 340 24.1 45.7 � 0.001� 20.5 1,048 4.0

Creatinine,* mg/dL � 1.2 1,054 5.0 37.0 � 0.001� 1.2 338 21.6

Albumin,* g/dL � 3.1 528 16.7 46.1 � 0.001� 3.1 594 1.8

Wasting Yes 524 10.3 5.7 0.22No 891 8.3

Neurologic symptoms§ Yes 296 24.4 39.7 � 0.001No 1,119 5.4

*Characteristic had an effect strength that was lower in jackknife validity analysis versus training analysis, suggesting that the level of classificationaccuracy achieved in training may not cross-generalize when it is used to classify an independent random sample.

†Cut point selected by optimal discriminant analysis for each characteristic because it maximized training sample sensitivity effect strength.‡Effect strength for sensitivity of the model, a standardized measure indicating the percent of the theoretically possible improvement inclassification accuracy—beyond what is expected by chance—that is achieved by the model. On this measure, 0 � classification accuracyexpected by chance and 100 � perfect (errorless) classification accuracy.

§Neurologic symptoms were considered present if a physician noted that the patient was either comatose, confused, and/or had symptoms ofneurologic change on day 1 or day 2 of the hospital admission.




that satisfied this criterion was 80.5, meaning thatpatients with pneumonia severity index scores� 80.5 were predicted to be alive (accurate for 832of 857 patients; 97.1%), while patients with pneumo-nia severity index scores � 80.5 were predicted to bedead (accurate for 107 of 548 patients; 19.5%). Thepneumonia severity index score had effect strengthfor sensitivity of 46.4, or 2% greater effect strengththan our CTA model. However, in contrast to ourCTA model, the pneumonia severity index was notstable in jackknife validity analysis as its effectstrength for sensitivity fell to 40.0, or 88% of theCTA model. The pneumonia severity index alsorequires the use of 19 variables, while our CTAmodel provides similar predictive ability and stableestimates while using only three variables.

Discussion

Opportunistic infections, such as PCP, have beenresponsible for a majority of the pulmonary-relatedmorbidity and mortality among patients with HIVinfection.1,2 However, bacterial pneumonia was rec-ognized as a common cause for hospitalizationamong HIV-infected gay/bisexual men early in theHIV epidemic.18 In 1995, Hirschtick et al3 reportedthat among their prospective cohort of 1,130 patientswith HIV infection, the incidence of bacterial pneu-monia was greater than the incidence of PCP. Theyestimated that the mortality attributed to bacterialpneumonia was 6%, and that patients with bacterialpneumonia had a mortality rate ratio of 3.9 com-pared to patients without bacterial pneumonia.3 Anobjective evaluation of the quality of care for HIV-associated CAP is needed in light of the increasingincidence of CAP among patients with HIV infec-tion. We studied 1,415 patients from seven majorgeographic areas in the United States who wereenrolled in the Multi-City Study of Quality of Carefor HIV-related CAP. The overall in-hospital mor-tality rate for these patients was 9.1%. Using infor-

mation easily available at hospital admission, wecreated a five-stage system using three variables forpredicting in-hospital mortality in patients with HIV-associated CAP.

Previous studies of HIV-associated bacterial pneu-monia reported several risk factors associated withmortality. One study of 350 episodes of bacterialpneumonia in 285 patients with HIV infection foundthat CD4� lymphocyte count � 100 cells/�L, neu-tropenia, Po2 � 70 mm Hg, and Karnofsky score� 50 were associated with increased mortality.19 Astudy of 355 patients with HIV infection with bacte-rial CAP found that shock, CD4� lymphocyte count� 100 cells/�L, and chest radiograph findings ofpleural effusion, cavities, or multilobar infiltrateswere risk factors for mortality.20 Similarly, in univar-iate analyses, we found that lower systolic or diastolicBP and CD4� lymphocyte count � 50 cells/�L wereassociated with higher mortality (p � 0.001). In con-trast to previous studies, our results indicated thathypoxemia � 60 mm Hg or the presence of a pleuraleffusion was not associated with higher mortality.Similar to prior studies in HIV-associated PCP,10

there was an increased risk of mortality amongpatients with prior use of M avium complex prophy-laxis (16.0% vs 7.7%, p � 0.001). The increasedmortality risk may be related to the underlyingimmunocompromised state of patients receivingM avium complex prophylaxis or macrolide prophy-laxis may be selecting for drug-resistant bacterialorganisms leading to increased mortality.

The variables used in our staging system reflect theseverity of pulmonary illness (respiratory rate) and twocomorbid medical conditions (renal and neurologicstatus). These factors are similar to those included inour HIV-associated PCP severity-of-illness system, andare routinely recorded in hospital admission notes.10

The PCP model included alveolar-arterial oxygen gra-dient that reflects the severity of pulmonary illness;however, we found that the rate of room air arterialblood gas results recorded for patients with HIV-

Table 7—Severity of Illness Staging System Based on the CTA Model

StageAltered Mental

Status*Respiratory Rate,

breaths/minCreatinine,

mg/dLPatients,No. (%)†

MortalityRate, %

1 No � 25 ‡ 748 (53.3) 2.32 No � 25 � 1.2 259 (18.5) 5.83 Yes � 25 ‡ 170 (12.1) 12.94 No � 25 � 1.2 100 (7.1) 22.05 Yes � 25 ‡ 126 (9.0) 40.5

*Altered mental status was considered present if patients were noted to be comatose, confused, and/or had symptoms of neurologic change.†Percentages given are the No. of patients with the indicated attribute profile divided by the total No. of patients with nonmissing data(n � 1,403).

‡A missing entry indicates that the attribute is not included in the indicated attribute profile (ie, branch of the classification tree).




associated CAP was much lower than with HIV-associated PCP, rendering the alveolar-arterial oxygengradient less useful as a potential predictor of mortalityin HIV-associated CAP. With respect to severity ofpulmonary illness, the CAP model uses respiratory ratethat was readily available from the hospital admissionphysical examination or nursing notes for all of thestudy patients.

Severity of comorbid medical illness was assessedby serum albumin in the PCP model and by neuro-logic symptoms and creatinine level in the CAPmodel. Comments on neurologic symptoms wereincluded in the majority of hospital admission notesfor patients with HIV infection with respiratorycomplaints. As opposed to albumin level, patientsroutinely receive testing for creatinine level duringhospital admission. The upper limit of the normalrange for creatinine level in most laboratories is1.4 mg/dL. However, among the subset of patientswithout neurologic symptoms and respiratory rate of� 25 breaths/min, we found that those with creati-nine � 1.2 mg/dL had a mortality rate of 22% vs5.8% for those with creatinine � 1.2 mg/dL. Themortality rate of 10.3% for this subset of patients wasvery similar to the overall mortality rate of 9.1%. Inprior studies of HIV-associated PCP, we found thatwasting was a risk factor for increased mortality.10

One possible explanation for the lower thresholdvalue for creatinine is that patients with HIV infec-tion have lower muscle mass resulting in lowerbaseline creatinine levels. Therefore, a creatininelevel of 1.2 mg/dL may represent a significant wors-ening of renal function in patients with lower-than-normal baseline values. Renal disease and BUN areboth risk factors for increased mortality among pa-tients with non–HIV-associated CAP as well.11,12

In addition to using routinely measured clinicalfindings and laboratory values, our staging system isboth disease specific and HIV specific. The effectstrength for sensitivity for our CTA model wassimilar to the pneumonia severity index that wasdeveloped for non–HIV-associated CAP.11,12 How-ever, the effect strength for our CTA model wasmore stable in validity analysis. We have previouslyreported similar findings for HIV-associated PCP, inwhich disease-specific models performed better thangeneral HIV severity-of-illness models.9,10 Our stag-ing system uses only 3 variables, compared to 19variables needed to calculate the pneumonia severityindex. Although other variables were significantlyassociated with increased mortality, their addition tothe CTA model did not improve predictive accuracy.The presence of liver disease and congestive heartfailure were significant risk factors for mortality inthe pneumonia severity index, but in our sample ofpatients with HIV infection, renal disease and neu-

rologic symptoms were the only two coexisting ill-nesses associated with increased inpatient mortality.Our disease-specific staging system should be usefulin future studies focused on patients with HIV-associated CAP by providing an efficient and easymethod to measure and compare severity of illness athospital admission.

There are several limitations to our study. Westudied patients from 86 hospitals in seven metro-politan areas, and our staging system performed wellin jackknife and bootstrap validity testing; however,further validation of our staging system is needed ondifferent populations, such as outpatients with HIV-associated CAP and patients from rural areas. Sec-ond, validation of our findings in patients treated inthe current era of highly active antiretroviral therapyis needed because our study was based on retrospec-tive review of medical records from the early highlyactive antiretroviral therapy era (1995–1997) with alow rate of protease inhibitor use. Third, since CAPis caused by a variety of infectious organisms, it ispossible that organism-specific severity-of-illnessstaging systems might perform better than our sys-tem. However, the etiologic cause of pneumonia isdiscovered in a minority of patients with HIV-associated CAP, making organism-specific systemsless applicable to most patients.21 Fourth, our defi-nition of CAP was based on ICD-9 codes for bacte-rial pneumonia and HIV-related disease, so thatdifferences in hospital coding practices may haveresulted in hospital level ascertainment bias. Fifth,dependence on ICD-9 codes may have also resultedin misclassification of pneumonia cases that wereactually caused by PCP or other pathogens.22 Weattempted to minimize misclassification by excludingpatients with cytologically proven PCP or culture-proven pulmonary tuberculosis in the previous 30days and patients with hospitalization for any reasonin the previous 30 days.

Additional limitations to our study are related tothe definitions of “neurologic symptoms” and “symp-toms of neurologic change.” First, the ascertainmentof the presence of neurologic symptoms was depen-dent on whether a physician noted symptoms on day1 or day 2 of the hospital admission. Second, there isa possibility that conditions such as mild peripheralneuropathy or HIV-associated opportunistic infec-tions involving the CNS were included as “symptomsof neurologic change” because chart abstractors werenot required to record symptom etiology. Third, theneurologic symptoms that were recorded may havebeen acute or chronic in nature. However, chartabstractors reported that most physician notes re-porting neurologic symptoms referred to acutechanges in mental status as opposed to chronicneurologic conditions. Also, if a patient had severe




neurologic symptoms that required hospitalizationwithin the past 30 days, they would have beenexcluded from our study population. Therefore, amajority of the neurologic symptoms included in ourdefinition were probably acute in nature and relatedto mental status changes.

In conclusion, we have developed a staging systemfor HIV-associated CAP based on data from 1,415patients with HIV infection who received care in 86hospitals in seven metropolitan areas in the UnitedStates. The staging system categorizes patients intofive stages associated with increasing risk of in-hospital death and had an overall classification accu-racy of 85.2%. The clinical significance of our stagingsystem is that by using three routinely obtainedclinical and laboratory variables (presence of neuro-logic symptoms, respiratory rate, and creatinine lev-el), one is able to estimate patient mortality risk athospital admission. Clinicians may use the stagingsystem to guide clinical decisions such as ICUadmission, aggressive antibiotic treatment, and closemonitoring of neurologic and renal status. Futurestudies and clinical guidelines examining outpatientvs inpatient management for HIV-associated CAPmay utilize the staging system for classifying patientsinto high-risk and low-risk categories. Our disease-specific staging system can also be useful for strati-fying patients in clinical trials evaluating antimicro-bial agents for the treatment of HIV-associated CAPand for case-mix adjustment when evaluating varia-tion in hospital mortality rates.

ACKNOWLEDGMENT: We thank the nurse abstractors ineach of the cities for data collection and the staffs of each of thehospitals for identification of medical records for review. We alsothank the following collaborators: Shirin Ali, Robert Weinstein,Matthew Goetz, Rafael Campo, Jack Dehovitz, and JeffreyJacobson.

References1 Bartlett JG. Pneumonia in the patient with HIV infection.

Infect Dis Clin North Am 1998; 12:807–8202 Murray JF. Pulmonary complications of HIV infection. Annu

Rev Med 1996; 47:117–1263 Hirschtick RE, Glassroth J, Jordan MC, et al. Bacterial

pneumonia in persons infected with the human immunode-ficiency virus. N Engl J Med 1995; 333:845–851

4 Palella FJ, Delaney KM, Moorman AC, et al. Decliningmorbidity and mortality among patients with advanced hu-man immunodeficiency virus infection. N Engl J Med 1998;338:853–860

5 Wallace JM, Hansen NI, Lavange L, et al. Respiratory diseasetrends in the pulmonary complications of HIV infection studycohort. Am J Respir Crit Care Med 1997; 155:72–80

6 Afessa B, Green B. Bacterial pneumonia in hospitalizedpatients with HIV infection: the pulmonary complications,ICU support, and prognostic factors of hospitalized patientwith HIV (PIP) study. Chest 2000; 117:1017–1022

7 Mathews WC, Ferdon E, Bennett CL, et al. Evaluatinginstitutional performance in AIDS-related Pneumocystis ca-rinii pneumonia: a risk-adjustment approach. J Clin Epide-miol 1989; 42:421–425

8 Bennett CL, Garfinkle JB, Greenfield S, et al. The relationbetween hospital experience and in-hospital mortality forpatients with AIDS related PCP. JAMA 1989; 261:2975–2979

9 Bennett CL, Weinstein RA, Shapiro MF, et al. A rapidpreadmission method for predicting inpatient course of dis-ease for patients with HIV-related PCP. Am J Respir CritCare Med 1994; 150:1503–1507

10 Arozullah AM, Yarnold PR, Weinstein RA, et al. A newpreadmission staging system for predicting inpatient mortalityfrom HIV-associated Pneumocystis carinii pneumonia in theearly highly active antiretroviral therapy (HAART) era. Am JRespir Crit Care Med 2000; 161:1081–1086

11 Fine MJ, Hanusa BH, Lave JR, et al. Comparison of adisease-specific and a generic severity of illness measure forpatients with community-acquired pneumonia. J Gen InternMed 1995; 10:359–368

12 Fine MJ, Auble TE, Yealy DM, et al. A prediction rule toidentify low-risk patients with community-acquired pneumo-nia. N Engl J Med 1997; 336:243–250

13 Centers for Disease Control and Prevention. HIV AIDSSurveill Rep 1997; 9:8–9

14 Oken C, Archibald N, Cvitanic M, et al. Multi-city study ofquality of care for HIV-related PCP: successfully collectinghighly sensitive information. Clin Perform Qual Health Care1995; 3:140–146

15 Yarnold PR. Discriminating geriatric and non-geriatric pa-tients using functional status information: an example ofclassification tree analysis via UniODA. Educ Psychol Meas1996; 56:656–667

16 Yarnold PR, Soltysik RC, Bennett CL. Predicting in-hospitalmortality of patients with AIDS-related Pneumocystis cariniipneumonia: an example of hierarchically optimal classifica-tion tree analysis. Stat Med 1997; 16:1451–1463

17 Flanders WD, Tucker G, Krishnadasan A, et al. Validation ofthe pneumonia severity index: importance of study-specificrecalibration. J Gen Intern Med 1999; 14:333–340

18 Cohn DL. Bacterial pneumonia in the HIV-infected patient.Infect Dis Clin North Am 1991; 5:485–507

19 Tumbarello M, Tacconelli E, de Gaetano K, et al. Bacterialpneumonia in HIV-infected patients: analysis of risk factorsand prognostic indicators. J Acquir Immune Defic SyndrHum Retrovirol 1998; 18:39–45

20 Cordero E, Pachon J, Rivero A, et al. Community-acquiredbacterial pneumonia in human immunodeficiency virus-infected patients: validation of severity criteria. Am J RespirCrit Care Med 2000; 162:2063–2068

21 Magnenat J, Nicod LP, Auckenthaler R, et al. Mode ofpresentation and diagnosis of bacterial pneumonia in humanimmunodeficiency virus-infected patients. Am J Respir CritCare Med 1991; 144:917–922

22 Selwyn PA, Pumerantz AS, Durante A, et al. Clinical predic-tors of Pneumocystis carinii pneumonia, bacterial pneumonia,and tuberculosis in HIV-infected patients. AIDS 1998; 12:885–893




DOI 10.1378/chest.123.4.1151 2003;123; 1151-1160Chest

Joan S. Chmiel, Laura Phan and Paul R. YarnoldAhsan M. Arozullah, Jorge Parada, Charles L. Bennett, Maria Deloria-Knoll,

*Community-Acquired PneumoniaA Rapid Staging System for Predicting Mortality From HIV-Associated

July 12, 2011This information is current as of

http://chestjournal.chestpubs.org/content/123/4/1151.full.htmlUpdated Information and services can be found at:

Updated Information & Services

http://chestjournal.chestpubs.org/content/123/4/1151.full.html#ref-list-1This article cites 22 articles, 7 of which can be accessed free at:

References

http://chestjournal.chestpubs.org/content/123/4/1151.full.html#related-urlsThis article has been cited by 2 HighWire-hosted articles:

Cited Bys

http://www.chestpubs.org/site/misc/reprints.xhtmlfound online at: Information about reproducing this article in parts (figures, tables) or in its entirety can bePermissions & Licensing

http://www.chestpubs.org/site/misc/reprints.xhtmlInformation about ordering reprints can be found online:

Reprints

"Services" link to the right of the online article.Receive free e-mail alerts when new articles cite this article. To sign up, select the

Citation Alerts

PowerPoint slide format. See any online figure for directions. articles can be downloaded for teaching purposes inCHESTFigures that appear in Images in PowerPoint format



http://chestjournal.chestpubs.org/content/123/4/1151.full.html#ref-list-1

http://chestjournal.chestpubs.org/content/123/4/1151.full.html#related-urls

http://www.chestpubs.org/site/misc/reprints.xhtml

http://www.chestpubs.org/site/misc/reprints.xhtml


Documents

A Rapid Staging System for Predicting Mortality From HIV-Associated Community-Acquired Pneumonia