Comparison of APACHE II, Trauma Score, and Injury Severity Score as Predictors of Outcome in

Critically Injured Trauma Patients Robert Rutledge, MD, Samir Fakhry, MD, Edmond Rutherford, MD,

Farid Muakkassa, MD, Anthony Meyer, MD, PhD, Chapel Hill, North Carolina

Trauma Score (TS) , APACHE II score, and Injury Severity Score (ISS) have been utilized to quanti- tate severity of illness in various groups of patients. The purpose of this study was to compare the rela- tionship of the APACHE II score, TS, and "comput- er-derived" ISS with outcome in critically injured trauma patients.

Data were recorded prospectively in a computer database for 428 consecutive trauma admissions. Stepwise discriminate analysis was utilized to deter- mine the best predictor of both intensive care unit (ICU) and hospital outcome.

Forty-two patients died in the ICU ( 1 0 % ) , and another 18 patients died after leaving the ICU ( 4 % ) , for a total mortality rate of 14%. The mean p value and partial R 2 value obtained f rom stepwise discriminant analysis of the relationships between APACHE II score, TS, and ISS to ICU and hospital survival are shown.

APACHE II score was the best predictor of both ICU and hospital outcome in these critically iH trau- ma patients. However, when combining all three measures (APACHE II score, TS, and ISS), only a portion of the variance in outcome is explained by the scores (R 2 < 0 . 0 5 ) . We conclude that scoring systems for outcome prediction should be utilized only as an adjunct to clinical assessment in the eval- uation of the severity of illness and mortality risk in critically fll t rauma patients.

From the Department of Surgery, University of North Carolina School of Medicine, Chapel Hill, North Carolina.

Requests for reprints should be addressed to Robert Rutledge, MD, Department of Surgery, CB# 7210 Burnett-Womack, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina 27599.

Manuscript received June 2, 1992, and accepted in revised form December 3, 1992.

T rauma is the most common cause of death in Ameri- cans under the age of 38 years and is the most serious

disease in the United States when viewed from the stand- point of the number of years of life lost. An accurate system for stratifying trauma patients on the basis of injury severity is important for decision-making related to prevention efforts, patient triage decisions, acute care management, outcome comparisons, reimbursement jus- tification, and local, regional, and national quality assur- ance activities.

Champion et al [1-5] have been leaders in the devel- opment and improvement of scoring systems for the tri- age and prediction of outcome in trauma patients. Their efforts have included the development of the Trauma Score (TS), the Revised Trauma Score (RTS), and the TRISS methodology for estimating the probability of survival.

The APACHE II score was developed by Knaus et al [6-15] based on analysis of a large database of physiolog- ic data derived from critically ill medical patients. It has been shown to be statistically associated with outcome in critically ill patients but to have limited usefulness be- cause of problems in predicting outcome in individual patients [16-20].

The TS and its subsequent modification, the RTS, were developed as rapid measures of the physiologic state of the injured patient with the idea that this information would be of use in triage and assessment in the injured patient [3,21], Studies have shown that the original TS was significantly associated with outcome in the injured patient, but it was also associated with problems in pre- dicting outcome in individual patients [6,22-29].

The TS is advantageous in that it is easy to calculate and is noninvasive. It is interesting to note that both the TS and the APACHE II score utilize similar data points, and both stress the assessment of the status of the central nervous system via the Glasgow Coma Scale in assigning risk [30].

The ISS utilizes the final anatomic diagnoses of the injury and scores each injury using the Abbreviated Inju- ry Scale (AIS). The AIS values for each of the three most severely injured body systems are squared, and these are added to give the ISS. A number of studies have used the ISS and have shown that it is statistically associated with patient outcome after injury [31].

The APACHE II score has engendered a great deal of controversy when applied to surgical patients as opposed to medical patients [30-39]. Some supporters of the TS as a useful measurement tool are strong critics of the APACHE II score as a predictive instrument. This is true

244 THE AMERICAN JOURNAL OF SURGERY VOLUME 166 SEPTEMBER 1993

PREDICTORS OF OUTCOME IN TRAUMA PATIENTS

despite the marked overlap in the TS and the APACHE II scores constituent data points.

The purpose of this study was to assess the relative predictive power of the APACHE II, the original TS, and the ISS as predictors of outcome in critically ill trauma patients. The outcome of the injured patient was deter- mined as survival upon discharge from the intensive care unit (ICU) and survival upon discharge from the hospi- tal. Our hypotheses for this study included the following: (1) APACHE II score would be significantly associated with outcome; (2) TS would be significantly associated with ICU outcome; (3) APACHE II and TS would be correlated; (4) APACHE II score would be a better pre- dictor of outcome than TS; and (5) APACHE II and ISS combined would be a better predictor of outcome than TS and ISS combined.

MATERIALS AND METHODS Data were derived from two concurrently functioning

databases at the University of North Carolina (UNC) Hospitals: the UNC Hospitals Trauma Registry and the UNC Hospitals ICU Database, which track patients ad- mitted with injury or admitted to the ICU, respectively. Data were combined and analyzed. Patients were includ- ed if they had an ICD-9 CM diagnosis code between 800 and 959.9 and were admitted to the surgical ICU (SICU) at UNC Hospitals between September 30, 1987, and April 24, 1991. A total of 428 patients fulfilled these criteria, which constituted 11.3% of the 3,767 trauma admissions and 17% of all SICU admissions during this time period. Burn patients were specifically excluded from this analysis.

The ISSs used in this study were not "true" ISS but rather were derived using the methodology developed by Dr. Ellen MacKenzie [40]. With the MacKenzie meth- od, the AIS score for each injury is derived at discharge using the ICD-9 CM diagnosis codes. The derived AIS scores are then used to calculate the ultimate ISS [30]. Our own studies have shown that this method slightly but significantly underscores head injuries in particular, but that the computer-derived ISS and hand-scored ISS are strongly correlated (R = 0.8), with only a small loss in predictive power (R 2 decreased from 0.4 to 0.31, unpub- lished data).

The APACHE II, TS, and Glasgow Coma Scale val- ues used in this study were calculated on the morning after admission to the SICU, based on the worst, reliable component values from the 12-hour period from 10 AM to 10 PM on the day of admission and excluding measure- ments that were deemed erroneous or unreliable.

Means were compared between the two groups using Student's t-test, and frequencies were compared between the two groups using the X 2 test. The ability of the three scores to predict the ICU and the ultimate hospital out- comes were compared using discriminant analysis, and the relative predictive power of each was compared using stepwise, forward, and backward modeling using dis- criminant analysis. Values are presented as the mean 4- SD. Significant differences were believed to be present when the p value was less than 0.05.

RESULTS During the 41 months of the study, 428 trauma pa-

tients were admitted to the SICU (11% of all trauma patients and 17% of SICU admissions). The average age was 25 + 12 years, and 60% were male. Forty-two pa- tients died in the ICU, and another 18 died after ICU discharge, for a total mortality rate of 14%.

The mean APACHE II score was 9.9 4- 6.6 in ICU survivors and 22 4- 8.0 in those who died in the ICU (p = 0.0001). When hospital survival was analyzed, the APACHE II values were similar: 8.6 4- 5.7 and 19.6 4- 8.1 in hospital survivors and nonsurvivors, respectively (p = 0.001).

The mean TS was 13.6 4- 2.9 in ICU survivors and 9.6 4- 5.0 in those who died in the ICU (p = 0.0001). These values were similar when analyzing hospital survival, which was 14.1 4- 2.6 and 11 4- 4.8 in survivors and nonsurvivors, respectively (p = 0.001).

The relation of TS and APACHE II was tested using Pearson's correlation coefficient. APACHE II and TS were significantly negatively correlated (R = -0.68, p = 0.001).

The mean ISS derived by the MacKenzie method was 17.3 4- 10.6 in ICU survivors and 20.1 4- 12.7 (p = 0.0001) in nonsurvivors, and 14.9 4- 7.5 in hospital survi- vors and 18.0 4- 12.6 in patients who died in the hospital (p --- 0.001).

Discriminant analysis was used to determine the rela- tive predictive power of the three scores tested. Predictive power was tested for both ICU survival and for ultimate hospital outcome. Each of the three scores was individual- ly statistically associated with outcome. It is interesting to note, however, that stepwise multivariate modeling iden- tified the APACHE II score as being the best individual predictor of outcome for both ICU stay and for hospital disposition. The TS added slightly to the predictive power of the APACHE II values for ICU survival, and the ISS added to the predictive power of APACHE II values for the final hospital disposition.

COMMENTS Trauma is an important disease in America, which is

being increasingly recognized as such. The impact in terms of the number of lives lost, the handicaps sustained, and the economic ramifications is devastating. Attempts to improve trauma care must be based on accurate com- parisons of severity of injury and the outcome of injury between groups of patients.

The subject of scoring systems has become an active area of research. The need to assign a relative numeric score to illness is important in every area of medicine. Reliable objective measures of illness severity are useful in comparing patients with regard to quality assurance purposes, the modification of reimbursement, triage pur- poses, treatment selection, and outcome assessment. Ex- amples of clinical scoring systems include the national system of diagnosis-related groups (DRGs), the new re- source-based relative value scales (RBRVS), the thera- peutic intervention score system (TISS), and the systems included in this study, TS, APACHE II, and ISS.

THE AMERICAN JOURNAL OF SURGERY VOLUME 166 SEPTEMBER 1993 2 4 5

RUTLEDGEET AL

The scoring systems selected for the present study come from two separate and sometimes competing areas of study. The APACHE II score was developed and test- ed in a population of predominantly medical ICU pa- tients [6-15]. Certain groups of surgical patients were excluded from the initial sampling because of problems with the system's predictive power in patientsundergoing coronary artery bypass surgery [6,41]. The TS and ISS were designed primarily for use in the injured patient regardless of the severity of the patient's illness.

The acute physiology and chronic health evaluation (APACHE) scoring system was first published by Knaus et al [6] to predict the patient's risk of hospital death after admission to the ICU. It was later simplified and pub- lished as the APACHE II in 1985 [41]. The APACHE II scoring system, in contrast to the 34 acute physiologic parameters measured in the original APACHE, relies on only 12 physiologic parameters in addition to age and chronic health status.

The TS and its subsequent modification, the RTS, were developed by Champion et al [1-5]. The TS was designed for field as well as hospital use as an aide to assessing the physiologic derangement caused by the in- jury. The score was designed for use in all injured patients from the least to the most severely injured [2,21]. A number of studies have evaluated the TS both as a tool for hospital triage and as an aid for predicting outcome. The results of these studies may seem somewhat conflicting, but, in summary, the findings show that although statisti- cally associated with outcome, the individual predictive power of the TS is limited [6,22-25,27-29].

The ISS, in contrast to the TS and the APACHE II score, is an anatomic scoring system and was derived in a manner different from the approach used to develop both the APACHE II and the TS. In the cases of the TS and APACHE II, the scores were derived from a multivariate regression analysis that provided the coefficients that gave the weighted point totals for the scores. The ISS, on the other hand, assigns points to different anatomic inju- ries based on the opinion of a committee of experts as to the risk of death for that particular injury. Our study and others have found the ISS, when used alone, to have only modest predictive power. In the present study, the R 2 for the ISS was only 0.31.

One purpose of this study was to compare the relative predictive power of the APACHE II with that of the TS as predictors of outcome in critically ill trauma patients. In addition to having similar constituent data points and weighting, it is interesting to point out that both scores were developed in Washington, DC (APACHE II by Knaus et al at George Washington University and TS by Champion et al at Washington Hospital Center). Despite the similarities between the two scoring systems, there has been a great deal of controversy surrounding their use in trauma patients and critically ill patients.

The present study is one of the first to address the question of the relative value of these two scores by test- ing the predictive powers of both the TS and the APACHE II in the same group of trauma patients. Given the fact that the APACHE II is a superset of the TS, it is

not surprising that we found that the APACHE II was a better predictor of outcome in trauma patients both early and late after injury. In addition, our finding of the great- er predictive power of the ISS on the hospital disposition also seems logical in retrospect. I f our previous experience is correct, the use of the hand scored or "true" ISS would probably have increased its predictive power to make it a slightly better predictor of outcome.

Possibly the most important finding of this study was that none of these scores in any combination achieves very high predictive capability. When using the R 2 as a measure of predictive power, only a portion of the vari- ance in patient mortality is explained by the combined model. This falls far short of the experienced clinician's ability to predict patient outcome, as we have seen in our own unpublished data. Thus, although there is clearly a need for such scores, they remain very limited tools that can be best utilized on large groups of patients. These scores have less value in predicting the outcome of the individual patient and, at present, clearly cannot compete with the judgment of an experienced clinician.

CONCLUSIONS Trauma is an important disease, and accurate scoring

systems would be of great value in the care of trauma patients. We tested APACHE II, TS, and ISS as predic- tors of outcome in 428 injured patients admitted to the SICU. All three scores were statistically significantly as- sociated with early ICU and hospital mortality. The APACHE II score was the best single predictor of out- come by a slight margin, TS added a small amount of predictive power for ICU outcome, and ISS added a small amount of predictive power for hospital outcome. It is important to note that in no case did any model explain more than 50% of the variance seen in mortality. There- fore, these scores are of limited value as predictors of injury outcome until their predictive power can be in- creased significantly. They are best utilized to assess large datasets, and their use in predicting the outcome of individual patients is limited. Although subjective, clini- cal decision-making remains the mainstay of outcome prediction for trauma patients in present-day trauma care.

REFERENCES 1. Champion HR, Sacco W J, Copes WS, Gann DS, Gennarelli TA, Flannagan ME. A revision of the trauma score. J Trauma 1989; 29: 623-9. 2. Moreau M, Gainer P, Champion H, Sacco WJ. Application of the trauma score in the prehospital setting. Ann Emerg Med 1985; 14: 1049-54. 3. Champion HR, Gainer PS, Yackee E. A progress report on the trauma score in predicting a fatal outcome. J Trauma 1986; 26: 927-31. 4. Sacco W J, Champion HR, Gainer PS, Morelli SA, Fallen S, Lawnick MA. The trauma score as applied to penetrating trauma. Ann Emerg Med 1984; 13: 415-8. 5. Champion HR, Sacco W J, Carnazzo A J, Copes W, Fouty WJ. Trauma score. Crit Care Med 1981; 9: 672-9. 6. Knaus WA, Zimmerman JE, Wagner DP. APACHE acute

246 THE AMERICAN JOURNAL OF SURGERY VOLUME 166 SEPTEMBER 1993

PREDICTORS OF OUTCOME IN TRAUMA PATIENTS

physiology and chronic health evaluation: a physiologically based classification system. Crit Care Med 1981; 9: 591-7. 7. Giangiuliani G, Gui D, Bonatti P, Tozzi P, Caracciolo F. APACHE II in surgical lung carcinoma patients. Chest 1990; 98: 627-30. 8. Schein M, Gecelter G. APACHE II score in massive upper gastrointestinal haemorrhage from peptic ulcer, prognostic value and potential clinical applications. Br J Surg 1989: 76: 733-6. 9. Larvin M, McMahon MJ. APACHE II score for assessment and monitoring of acute pancreatitis. Lancet 1989; 2: 201-5. 10. Dobkin JE, Cutler RE. Use of APACHE II classification to evaluate outcome of patients receiving hemodialysis in an intensive care unit. West J Med 1988; 149: 547-50. 11. Jacobs S, Chang RW, Lee B. Audit of intensive care: a 30 month experience using the APACHE II severity of disease classifi- cation system. Intensive Care Med 1988; 14: 567-74. 12. Chang RW, Jacobs S, Lee B. Predicting outcome among inten- sive care unit patients using computerized trend analysis of APACHE II scores corrected for organ system failure. Intensive Care Med 1988; 14: 558-66. 13. Knaus WA, Wagner DP. APACHE: a nonproprietary measure of severity of illness [letter]. Ann Intern Med 1989; 110: 327-8. 14. Fedullo A J, SwinBurne AJ, Wahl GW, Bixby KR. APACHE II score and mortality in respiratory failure due to cardiogenic pulmonary edema. Crit Care Med 1988; 16: 1218-21. 15. Wagner DP, Draper EA, Abizanda Campos R, et al. Initial international use of APACHE. An acute severity of disease mea- sure. Med Decis Making 1984; 4: 297-313. 16. Wilson C, Heath DI, Imrie CW. Prediction of outcome in acute pancreatitis: a comparative study of APACHE II, clinical assess- ment and multiple factor scoring systems. Br J Surgery 1990; 77: 1260-4. 17. Jackson EA, Udwadia ZF, Rothwell PM, Lawler PG. APACHE II expected outcome calculations are subject to inter- observer variation [letter]. Anaesthesia 1990; 45: 888-9. 18. Kruse JA, Thil-Baharozian MC, Carlson RW. Comparison of clinical assessment with APACHE II for predicting mortality risk in patients admitted to a medical intensive care unit. JAMA 1988; 260: 1739-42. 19. Jacobs S, Chang RW, Lee B. One year's experience with the APACHE II severity of disease classification system in a general intensive care unit. Anaesthesia 1987; 42: 738-44. 20. Chang RW, Jacobs S, Lee B. Use of APACHE II severity of disease classification to identify intensive care patients who would not benefit from total parenteral nutrition. Lancet 1986; 1: 1483-7. 21. Morris JA, Auerback PS, Marshall GA, Bluth RF, Johnson LG, Trunkey DD. The trauma score as a triage tool in the prehospi- tal setting. JAMA 1986; 256: 1319-25. 22. Eichelberger MR, Gotschall CS, Sacco W J, Bowman LM, Mangubat EA, Lowenstein AD. A comparison of the trauma score, the revised trauma score, and the pediatric trauma score. Ann Emerg Med 1989; 18: 1053-8. 23. Who will die in the ICU? APACHE II, ROC curve analysis, and, of course, Cleone [letter; comment]. JAMA 1989; 261: 1279-80.

24. Hawkins M-L, Treat RC, Mansberger AR Jr. The trauma score: a simple method to evaluate quality of care. Am Surg 1988; 54: 204-6. 25. Rhee K J, Baxt WG, Mackenzie JR, et aL APACHE II scoring in the injured patient. Crit Care Med 1990; 18: 827-30. 26. Cerra FB, Negro F, Abrams J. APACHE II score does not predict multiple organ failure or mortality in postoperative surgical patients. Arch Surg 1990; 125: 519-22. 27. The trauma score as a triage tool [letter]. JAMA 1987; 257: 484-5. 28. Deane SA, Gaudry PL, Roberts RF, Juul O, Little JM. Trau- ma triage--a comparison of the trauma score and the vital signs score. Aust N Z J Surg 1986; 56: 191-7. 29. Ornato J, Mlinek EJ Jr, Craren E J, Nelson N. Ineffectiveness of the trauma score and the CRAMS scale for accurately triaging patients to trauma centers. Ann Emerg Med 1985; 14: 1061-4. 30. Weingarten S, Bolus R, Riedinger MS, Maldonado L, Stein S, Ellrodt AG. The principle of parsimony: Glascow Coma Scale score predicts mortality as well as the APACHE II score for stroke patients. Stroke 1990; 21: 1280-2. 31. Maher ER, Robinson KN, Scoble JE, et al. Prognosis of criti- cally-ill patients with acute renal failure: APACHE II score and other predictive factors. Q J Med 1989; 72: 857-66. 32. Hopefl AW, Taaffe CL, Herrmann VM. Failure of APACHE II alone as a predictor of mortality in patients receiving parenteral nutrition. Crit Care Med 1989; 17: 414-7. 33. Poenaru D, Christou NV. Clinical outcome of seriously ill surgical patients with intra-abdominal infection depends on both physiologic (APACHE II score) and immunologic (DTH score) alterations. Ann Surg 1991; 213: 130-6. 34. Escarce J J, Kelley MA. Admissions source to the medical intensive care unit predicts hospital death independent of APACHE II score. JAMA 1990; 264: 2389-94. 35. Bohnen JM, Mustard RA, Oxholm SE, Schouten BD. APACHE II score and abdominal sepsis. A prospective study. Arch Surg 1988; 123: 225-9. 36. MacKenzie E J, Steinwachs DM, Shankar BS. An ICD-9 CM to AIS conversion table: development and application. Proceedings of the American Association for Automotive Medicine, 1986:135. 37. Cerra FB, Negro F, Abrams J. APACHE II score does not predict multiple organ failure or mortality in postoperative surgical patients. Arch Surg 1990; 125: 519-22. 38. Giangiuliani G, Mancini A, Gui D. Validation of severity of illness score (APACHE II) in a surgical intensive care unit. Inten- sive Care Med 1989; 15: 519-22. 39. Ivatury R, Nallathambi M, Rao PM, Rohman M, Stahl WM. Open management of the septic abdomen: therapeutic and prognos- tic consideration base on APACHE II. Crit Care Med 1989; 17: 511-7. 40. MacKenzie E J, Steinwachs DM, Shankar B. Classifying trau- ma severity based on hospital discharge diagnoses. Med Care 1989; 27: 412-22. 41. Knaus WA, Draper EA, Wagner DP, Zimmerman JE. APACHE II: a severity of disease classification system. Crit Care Med 1985; 13: 818-29.

THE AMERICAN JOURNAL OF SURGERY VOLUME 166 SEPTEMBER 1993 247