ww.sciencedirect.com

j o u r n a l o f s u r g i c a l r e s e a r c h 1 8 4 ( 2 0 1 3 ) 5 4e6 0

Available online at w

journal homepage: www.JournalofSurgicalResearch.com

Association for Academic Surgery

Preventable mortality: does the perspective matter whendetermining preventability?

Meera Gupta, MD,a Barry Fuchs, MD,b Carolyn Cutilli, RN, MSN,b Jessica Cintolo, MD,a

Caroline Reinke, MD, MPH, MSHP,a Craig Kean, MS,b Neil Fishman, MD,b

Patricia Sullivan, PhD,b and Rachel R. Kelz, MD, MSCEa,*aDepartment of Surgery, University of Pennsylvania, Philadelphia, PennsylvaniabDepartment of Clinical Effectiveness and Quality Improvement, University of Pennsylvania Health System, Philadelphia, Pennsylvania

a r t i c l e i n f o

Article history:

Received 4 January 2013

Received in revised form

13 April 2013

Accepted 15 May 2013

Available online 10 June 2013

Keywords:

Preventable mortality

Mortality review process

360� survey

Multidisciplinary mortality review

* Corresponding author. Department of SurgTel.: þ1 215 662 2030x3; fax: þ1 215 662 7476

E-mail address: Rachel.Kelz@uphs.upenn0022-4804/$ e see front matter ª 2013 Elsevhttp://dx.doi.org/10.1016/j.jss.2013.05.069

a b s t r a c t

Background: We report a novel approach to mortality review using a 360� survey and

a multidisciplinarymortality committee (MMC) to optimize efforts to improve inpatient care.

Methods: In 2009, a 16-item, 360� compulsory quality improvement survey was implemented

for mortality review. Descriptive statistics were performed to compare the responses by

provider specialty, profession, and level of trainingusing the Fisher exact and chi-square tests,

as appropriate.We compared the agreement between theMMC review and provider-reported

classification regarding the preventability of each death using the Cohen kappa coefficient. A

qualitative review of 360� information was performed to identify the quality opportunities.

Results: Completed surveys (n ¼ 3095) were submitted for 1683 patients. The possibility of

a preventable death was suggested in the 360� survey for 42 patients (1.40%). We identified

502 patients (29.83%) with completed 360� surveys who underwent MMC review. The inter-

rater reliability between the provider opinions regarding preventable death and the MMC

review was poor (kappa ¼ 0.10, P < 0.001). Of the 42 cases identified by the 360� survey as

preventable deaths, 15 underwent MMC review; 3 were classified as preventable and

12 were deemed unavoidable. Qualitative analyses of the 12 discrepancies did reveal

quality issues; however, they were not deemed responsible for the patients’ death.

Conclusions: Themortality survey yielded important information regarding inpatient deaths

that historically was buried with the patient. Poor agreement between the 360� survey

responses and an objective MMC review support the need to have a multipronged approach

to evaluating inpatient mortality.

ª 2013 Elsevier Inc. All rights reserved.

1. Introduction some patients die in the hospital as a result of medical errors.

The Institute of Medicine report in 2000 [1], and subsequent

reports in the media [2e5], have publicized the notion that

ery, University of Pennsy..edu (R.R. Kelz).ier Inc. All rights reserved

The heightened awareness of potentially preventable mor-

tality has fueled the development of national initiatives [6e8]

and regional [9,10], institutional [6,11], and departmental

lvania, 3400 Spruce Street, 4 Silverstein, Philadelphia, PA 19104.

.

j o u r n a l o f s u r g i c a l r e s e a r c h 1 8 4 ( 2 0 1 3 ) 5 4e6 0 55

[12,13] programs to address this critical issue in inpatient care

[14]. These efforts have relied on risk-adjusted mortality rates

[15] to account for differences in patient acuity across insti-

tutions [16e18] to successfully gauge the quality of care, as

defined by the inpatient mortality measure.

Institutional efforts to reduce preventable mortality have

included mortality and morbidity conferences and stan-

dardized mortality review. The time-honored departmental

mortality and morbidity conference has often been limited by

a lack of uniformity and structure [19e21]. Additionally, the

nursing staff and other hospital personnel that contribute to

the patients’ clinical course are typically not present for these

discussions [22,23]. Alternatively, mortality review panels

comprised of individuals and smaller groups of physicians

have been noted to overestimate the preventable mortality

and to possess a high degree of variability and error [24].

More recently, interdisciplinary mortality review committees,

similar to those used by many trauma teams, have been

implemented to investigate inpatient deaths within hospitals

[12,25,26].

Although the systematic review of every inpatient death

might provide invaluable information regarding opportunities

for local and institutional improvement, these reviews are

quite labor intensive and rely on the retrospective review

of the clinical documentation. Information gathered from

frontline providers based on the medical record has varied

across specialties and is often difficult to interpret. Clinical

documentation around critical events (i.e., adverse drug

events) might not accurately capture the true nature of the

occurrence, given the concerns regarding litigation [27].

We report a novel approach to gather real-time informa-

tion about preventable inpatient mortality from all members

of the clinical team using a hospital-wide 360� survey. Our

report includes differences in provider responsiveness and

opinions by service line and professional training.

Table 1 e Quality and mortality classification system.

Level Description

1 Mortality expected/unpreventable

2 Practice consistent with standards,

mortality occurred

3 Practice not necessarily consistent

with standards, but still acceptable;

mortality occurred

4 Practice deviates from standards;

mortality possibly preventable

5 Practice deviates from standard;

mortality preventable

Developed by the Hospital of the University of Pennsylvania

Multidisciplinary Mortality Committee, 2009.

2. Methods

The Hospital of the University of Pennsylvania is a 772-bed

tertiary care hospital in eastern Pennsylvania. Approxi-

mately 41,000 admissions occur annually, with an annual

mortality rate of 1.8%e2.0% during the past 5 years. In 2006,

the Hospital of the University of Pennsylvania (HUP) formed

a multidisciplinary mortality committee (MMC) to evaluate

the quality of care delivered to patients who died during

a hospitalization. After examination of the administrative

data, detailed unstructured chart review, and focus groups, it

was clear that (1) using administrative data alone did not

provide enough detail to identify potentially preventable

deaths; (2) detailed systematic chart review of each death

by a trained abstracter was costly and time consuming; and

(3) the opinions of the frontline care providers regarding the

preventability of the death was not easy to appreciate without

directly discussing the case with each participant.

A central chart review was deemed a critical component of

the mortality review process to ensure objectivity. Therefore,

a 30-min structured review of all deaths by a trained regis-

tered nurse was initiated. Each review was discussed in detail

with one of the two physician leaders of the MMC. Beginning

in 2011, the detailed review was completed with classification

of any quality issues identified and the classification of the

death according to a five-level preventable mortality scale.

Table 1 lists the levels of the quality and mortality classifica-

tion system. Although a variety of approaches have been used

to elicit judgments of preventability, no formal classification

for preventable death has been validated [28]. The quality

classification system and levels of preventable mortality

for HUP were developed after an informal review of the pub-

lished data and materials provided by the following organi-

zations: University Health System Consortium, Nebraska

Medical Center, Grady Health System, Albany Medical Center,

Medical University of South Carolina, MCG Health Inc,

University of Arkansas Medical Sciences, University of Wis-

consin Hospital and Clinics, University Hospital and Health

System Mississippi, University of Louisville Hospital, Penn-

sylvania State Hershey Medical Center, Vanderbilt University

Hospital, Medical College of Georgia, Pennsylvania Presbyte-

rian Medical Center, Pennsylvania Hospital, and Brigham and

Women’s Hospital.

In addition to the central mortality review process,

a systematic electronic query of providers at the time of all

inpatient deaths was developed to allow the clinical care team

the opportunity to share concerns regarding the quality of

care provided and to capture information that might not be

apparent during a review of the medical record. At the inpa-

tient death, the certifying resident physician completes an

electronic packet of information to comply with Pennsylvania

state laws. The packet includes the pronunciation of death,

certification of death, paperwork regarding the eligibility of

organ donation, and forms for the medical examiner. In 2009,

a 360�, mandatory, 16-item survey was added to the packet

(see the Appendix). The form is also sent to the attending

physicians of record, resident physicians, nurse practitioners,

physician assistants, nurse managers, and respiratory thera-

pists involved in the patient’s care at the time of death for

completion within 48 h. The survey is intended to capture

information about a patient’s care from the array of providers

queried. As such, we have referred to it as a 360� mortality

survey. The results are captured by the Department of Clinical

Effectiveness and Quality Improvement database and linked

to the administrative data for each patient death. In brief, the

j o u r n a l o f s u r g i c a l r e s e a r c h 1 8 4 ( 2 0 1 3 ) 5 4e6 056

360� survey questions providers regarding the patient’s care

during the hospitalization. The providers are required to

categorize the preventability of the patient’s death (yes, no, or

unknown). The providers are also asked to report concerns

regarding the quality of the care delivered during the patient’s

hospital course. Specific data elements from the 360� survey

include the primary diagnosis, intensive care unit diagnosis,

preventability of death, and adverse events. All results

are considered subjective; however, after additional investi-

gation, they are frequently used to guide subsequent quality

improvement efforts.

We performed a retrospective analysis of the data from the

360� surveys recorded from February 2009 to March 2012. All

patients with completed 360� surveys with at least one pro-

vider that categorized the death as potentially preventable

were included in the analysis. The patient records were linked

to the HUP MMC database to delineate the patient character-

istics, including demographic data, risk of mortality, severity

of illness [29], length of stay, duration in the intensive care

unit, and the classification of the level of preventable death.

Descriptive statistics were performed on the results from

the 360� survey. The responses were compared by provider

specialty and provider levels of training using the Fisher exact

and chi-square tests, as appropriate. P < 0.05 was considered

significant.

We compared the agreement between the mortality

committee classification and provider classification using the

Cohen kappa coefficient [30e32]. The cases that were assigned

a level 4 or 5 by the MMC were classified as preventable. The

kappa statistic ranged from �1 to þ1. The interpretation of

kappa values was as follows: 0e0.20, negligible agreement

beyond chance; 0.21e0.40, fair agreement; 0.41e0.60,

moderate agreement; and >0.60, substantial agreement [33].

For discordant cases, a qualitative review of the 360� surveyresponses was performed to identify helpful information for

future quality improvement initiatives. Comments were coa-

lesced into common themes by keyword identification [32].We

used the MMC classification system to aggregate the qualita-

tive comments. The MMC classification system was divided

into the following categories: provider competency, delay in

treatment, delay in diagnosis, provider communication of

patient information, inadequate treatment, and other [34].

All data was transferred into STATA format using Stat/

Transfer, version 11.0, statistical program, and analysis was

performed using STATA, version 12.0/IC statistical software

(StataCorp, College Station, TX) [35,36].

3. Results

3.1. 360� Survey results and preventable mortality

During the study period, 2483 patients died within the hospital

and 1683 patients had 360� surveys completed, for a response

rate of 67.8%. Of the 3095 360� surveys completed for 1683

patients, 42 patient deaths (1.4%) were identified by the care

providers as preventable. Of the patient deaths deemed

preventable, most of the patients were men (54.8%) and white

(57.1%). Themean age at deathwas 63.4� 16.9 y. Most patients

with a preventable death identified by the providers had been

emergently admitted (90.5%), with a classification for severity

of illness on admission of “extreme” (66.7%) or “major” (26.2%)

and a mortality risk classification of “extreme” (54.8%) or

“major” (35.7%). The mean observed total length of stay was

11.3 � 18.1 d, and the mean observed duration in an intensive

care unit was 7.7 � 16.1 d. Of the 42 patients, 11 (26.2%) expe-

rienced “early death.” defined as death within 48 h of admis-

sion. The two most common primary diagnoses from the

administrative data for patients whose deaths were consid-

ered preventablewere in two categories: cardiovascular events

(21.4%) and trauma, neurologic, and toxicity events (21.4%).

The two most common secondary diagnosis categories were

organ failure (28.6%) and sepsis and/or infectious (25.0%). The

twomost common reasons for organ failurewere severe sepsis

in 12 (28.6%) and respiratory failure in nine (21.4%) patients.

Of the 3095 360� surveys, most were completed by resident

physicians. Of the respondents, 47%were residents, 24% were

attending physicians, 24% were nurses, 6% were respiratory

therapists, and <0.1% were advanced providers (nurse prac-

titioners and physician assistants). The absolute number

of patients with surveys completed by each level of provider

was as follows: 736 by attending physicians, 1434 by resident

physicians, 723 by nurses, 4 by nurse practitioners or physi-

cian assistants, and 170 by respiratory therapists.

The proportion of deaths deemed preventable by the

360� surveys responses differed across specialties, with 2%

reported by surgical practitioners and 1% reported bymedicine

practitioners (P < 0.01). The provided classified 18% of

the medicine cases and 25% of the surgical cases “I do not

know”when determining the preventability of death (P< 0.01).

Therefore, although most inpatient deaths were not prevent-

able, uncertainty in determining the preventability of death

does appear to exist in a large proportion of deaths under

review by the primary providers from both specialties.

The determination of preventable death was also reported

by provider level within each specialty (Fig.). During the study

period, 589 deaths and 979 deaths occurred in the medicine

and surgical subspecialties, respectively. The medicine sub-

specialty overall 360� survey response rate (2.2 surveys per

patient) was higher than that of the surgical subspecialty (1.4

surveys per patient). No significant difference was found in

the determination of preventable death between same level of

providers when comparing attending providers, nurse prac-

titioners and physician assistants, nurses, and respiratory

therapists across specialties (P > 0.05). However, a significant

difference was found between the medicine and surgical

resident physicians in the identification of preventable death

(P < 0.01; Table 2). When comparing physician versus non-

physician providers within specialties, the attending physi-

cian and resident physician providers were more likely to

report a death as preventable than were other providers,

including nurse practitioners, physician assistants, nurses,

and respiratory therapists (P < 0.01).

3.2. Interrater reliability for preventable mortality

We identified 502 patients (29.8%) with completed 360� sur-

veys who had undergone a central detailed MMC review

complete with classification of the preventability of the death.

Of these 502 patients, 42 patient deaths (8.4%) were deemed as

Fig. e Preventable death response by specialty and provider level. (Color version of figure is available online.)

j o u r n a l o f s u r g i c a l r e s e a r c h 1 8 4 ( 2 0 1 3 ) 5 4e6 0 57

possibly preventable by the providers who completed the 360�

surveys. Of these 42 patient deaths, only three (7.1%) were

deemed definitely preventable by the HUP MMC. Two cases

were classified as “level 5: practice deviates significantly” and

one case was classified as “level 4: practice deviates from

standards;mortality possibly preventable.” Of the remaining 39

deaths that were not ruled as preventable by the MMC stan-

dards, one was considered possibly preventable on the 360�

survey and was ruled “level 3: practice not necessarily consis-

tent with standards, but still acceptable; mortality occurred” by

theMMC. The remaining 38 deaths received a ruling of “level 2:

practice consistent with standards, mortality occurred.” The

inter-rater reliability between provider opinions regarding

preventable deaths and the mortality review committee clas-

sification was poor (kappa ¼ 0.10, P < 0.01).

Table 2 e Preventable death classification responses by specia

Provider specialtyand response

Attending Resident Nurse prphysici

Surgery

Yes 5 (2.58) 9 (1.97) 0

No 158 (81.44) 341 (74.45) 1

Do not know 31 (15.98) 108 (23.58) 0

Medicine

Yes 8 (1.64) 13 (1.43) 0

No 424 (86.89) 772 (85.02) 2

Do not know 56 (11.48) 123 (13.55) 0

P valuey 0.12 <0.01

NA ¼ not applicable.

Data presented as n (%).

*Within specialty provider comparisons using Fisher’s exact and chi-squyProvider comparison by specialty using Fisher’s exact and chi-square te

3.3. Quality measures associated with preventablemortality

Of the 42 deaths identified by the 360� survey as preventable

deaths, only three were classified as preventable. Qualitative

analyses of the discordant cases revealed that deaths reported

as preventable by providers on the 360� survey often identified

opportunities for systemwide quality improvement (e.g., early

recognition of deterioration). These cases were not classified

by the MMC as preventable, despite the quality opportunities,

because the detailed MMC review deemed the quality issue as

unlikely to have changed the risk of mortality on admission

for the patient. Consider the following representative provider

(360�) response regarding the subjective description of

a preventable death that was not deemed preventable by the

lty and provider level of training.

actitioner oran assistant

Nurse Respiratorytherapist

P value*

<0.01

(0.00) 1 (0.94) 1 (1.75)

(100.00) 66 (62.26) 27 (47.37)

(0.00) 39 (36.79) 29 (50.88)

(0.00) 4 (0.69) 0 (0.00)

(100.00) 427 (73.75) 42 (43.30)

(0.00) 148 (25.56) 55 (56.70)

NA 0.14 0.56

are tests, as appropriate.

sts, as appropriate.

j o u r n a l o f s u r g i c a l r e s e a r c h 1 8 4 ( 2 0 1 3 ) 5 4e6 058

MMC: “I believe this patient was not transferred safely from

the emergency room to the medical intensive care unit, and

had the patient been adequately resuscitated before transfer,

it is possible that death would have been avoided.” Although

the provider comments clearly identified an opportunity for

improvement in the process of care, in this case, the patient

had presented to the emergency department with irreversible

multisystem organ failure and moribund, and the death was

therefore classified as a nonpreventable death by the MMC.

Subjective provider responses of the discordant patients

not classified as experiencing a preventable death by the

HUP MMC attributed them to hospital-acquired infections,

resuscitation and airway management, management during

transportation, patient disease, postprocedural complica-

tions, and patient access to medical therapy.

4. Discussion

The electronic 360� mortality survey performed at a patient’s

death provides frontline caregivers with the opportunity to

share information regarding the inpatient stay that would not

be readily available by chart review or from the administrative

claims data. In so doing, the hospital MMC is provided with

a comprehensive view at inpatient deaths from the provider

perspective in a format suitable for aggregated study

providers to be able to contribute to quality improvement

initiatives on behalf of their deceased patients.

Our data have highlighted the critical role of direct patient

providers in the mortality review process. The results of

our study support the role of subjective reporting as a vital,

and often missing, element in the global understanding of

key events surrounding a patient death [37,38]. We found that

residents and nurses provided the most qualitative informa-

tion regarding inpatient mortality and that the sparse com-

ments shared by attending physicians were often particularly

instructive. The information from the 360� survey was useful

when equipoise existed in determining whether a patient

deathwas preventable and helped identify root causes of error

unrelated to the patient’s death that might have occurred.

A mortality review has been recognized by many institu-

tions as a formal process to explore possible breaches in the

quality of care that could have potentially led to death [39].

The willingness of physicians to label deaths as potentially

preventable is one step toward minimizing the occurrence of

these rare, but agonizing, events. Given the medical expertise

of physician providers and their knowledge of the events

surrounding patient deaths, we suggest that physicians

should be integral members of the mortality review process.

Preventable deaths are rare, perhaps even underestimated

by a formal mortality review and overestimated by care

providers, but they do occur [24]. In our study, the subjective

360� responses overestimated the preventability of the inpa-

tient deaths compared with the objective HUP MMC detailed

reviews. The emotional toll of a patient death might leave

providers feeling responsible and inadequate [40,41], thus

prompting them to assume responsibility for the poor

outcomes despite their adherence to best practice guidelines.

The second victim phenomenon can lead to provider burnout

and depression [42], making the findings of the MMC

important for the organization and the health of the practi-

tioners in the system.

It has been reported that different consensus systems

addressing preventable deaths in a mortality review will yield

different results, with overall low reliability [28,37,43]. The

poor agreement between the MMC findings and those of the

subjective provider review reflect the differences between the

two approaches, as well as the importance of each perspective

[18]. Concerns regarding provider objectivity have led to the

development of more systemic mortality review through the

use of standardized approaches by multidisciplinary teams

[12,25,26,44e47]. Most of this work has focused on deaths

within specific medical specialties and a paucity of data

remains regarding the application of such interdisciplinary

mortality review committees to assess hospital-wide

preventable mortality across medical specialties.

Our study had a few limitations. First, not all patients with

360� surveys completed by clinical providers underwent

a detailed chart review by the HUPMMC. Although a complete

review would have substantially increased our sample size,

we have no reason to suspect that the rate of agreement

would have changed significantly. Second, despite the

protection of the 360� results under the quality infrastructure

[48,49], some providers remain concerned about the discov-

erable nature of the information and, therefore, were not

forthcoming with their concerns. The number of responses

per patient indicated that not all providers chose to share their

perspective. Therefore, the survey might not have reported

the plurality of views implied by the “360�” title. It is possible

that the views expressed did not benefit from the opinions of

multiple caregivers of different types collaborating on the

same patient and were subject to observer bias. However, the

nature of the data was subjective, and, therefore, this limita-

tion would not diminish the value of the qualitative infor-

mation obtained. Finally, the 5-level mortality classification

system used by our institution has not yet been validated,

necessitating additional study and, ultimately, the standard-

ization of this scale within the objective arm of a formal

mortality review process.

The mortality review process is continuously evolving to

meet the inherent challenges of identifying and mitigating all

preventable deaths. Care provider recognition of the impor-

tance of a standardized mortality review process facilitates

identification of quality improvement opportunities. To bridge

the gap between the 360� mortality survey and the MMC

review process, we have incorporated representatives from

different specialty-specific wards and house staff into the

MMC leadership team. These representatives were respon-

sible for providing updates to both the committee and the

clinical care teams on a regular basis regarding efforts to

reduce inpatient deaths and their effectiveness. For this to be

successful in the future, medical education should provide

formal leadership training to enhance physicians’ ability to

serve in this capacity.

The 360� mortality survey yielded important information

regarding inpatient deaths that historically was buried with

the patient. The survey process facilitates provider participa-

tion in the quality improvement process. The survey is inte-

gral to the mortality review and permits a blending of facts

from objective reviewers and the subjective perspective of the

j o u r n a l o f s u r g i c a l r e s e a r c h 1 8 4 ( 2 0 1 3 ) 5 4e6 0 59

actual care participants. The two processes yield comple-

mentary information. We strongly encourage all hospitals to

engage frontline providers in efforts directed at under-

standing inpatient mortality and to work with a centralized

multidisciplinary team to strive for zero preventable deaths in

the future. The organization and all providers must invest in

the process if preventable mortality is to be eradicated from

the inpatient setting.

Supplementary data

Supplementary data associated with this article can be

found, in the online version, at http://dx.doi.org/10.1016/j.jss.

2013.05.069.

r e f e r e n c e s

[1] Institute of Medicine. To err is human: building a saferhealth system. Washington, DC: National Academy Press;2000.

[2] Preventing fatal medical errors. New York Times; 1999. A22.[3] Hospital-caused deaths. New York Times; 2006.[4] Gawande A. A Lifesaving checklist. The New York Times;

2007.[5] President Bill Clinton to address health care leaders at

Inaugural Patient Safety, Science, & Technology Summitdacall to action to eliminate preventable deaths and reducecosts. PR Newswire US; 2012.

[6] Behal R, Finn J. Understanding and improving inpatientmortality in academic medical centers. Acad Med J Assoc AmMed Coll 2009;84:1657.

[7] Wachter RM, Pronovost PJ. The 100,000 lives campaign:a scientific and policy review. Jt Comm J Qual Patient Saf2006;32:621.

[8] Berwick DM, Hackbarth AD, McCannon CJ. IHI replies to “the100,000 lives campaign”: a scientific and policy review. JtComm J Qual Patient Saf 2006;32:628.

[9] Kilpatrick SJ, Prentice P, Jones RL, Geller S. Reducingmaternal deaths through state maternal mortality review. JWomen Health 2012;21:905.

[10] Berg CJ, Harper MA, Atkinson SM, et al. Preventability ofpregnancy-related deaths: results of a state-wide review.Obstet Gynecol 2005;106:1228.

[11] Hogan H, Healey F, Neale G, Thomson R, Vincent C, Black N.Preventable deaths due to problems in care in English acutehospitals: a retrospective case record review study. BMJ QualSaf 2012;21:737.

[12] Teixeira PG, Inaba K, Hadjizacharia P, et al. Preventable orpotentially preventable mortality at a mature trauma center.J Trauma 2007;63:1338.

[13] Stamou SC, Camp SL, Stiegel RM, et al. Quality improvementprogram decreases mortality after cardiac surgery. J ThoracCardiovasc Surg 2008;136:494.

[14] Rutstein DD, Berenberg W, Chalmers TC, Child CG III,Fishman AP, Perrin EB. Measuring the quality of medicalcare: a clinical method. N Engl J Med 1976;294:582.

[15] Dubois RW, Brook RH, Rogers WH. Adjusted hospital deathrates: a potential screen for quality of medical care. Am JPublic Health 1987;77:1162.

[16] Roemer MI, Moustafa AT, Hopkins CE. A proposed hospitalquality index: hospital death rates adjusted for case severity.Health Serv Res 1968;3:96.

[17] Duckett SJ, Kristofferson SM. An index of hospitalperformance. Med Care 1978;16:400.

[18] Dubois RW, Rogers WH, Moxley JH III, Draper D, Brook RH.Hospital inpatient mortality: is it a predictor quality? N Engl JMed 1987;317:1674.

[19] Higginson J, Walters R, Fulop N. Mortality and morbiditymeetings: an untapped resource for improving thegovernance of patient safety? BMJ Qual Saf 2012;21:576.

[20] Deis JN, Smith KM, Warren MD, et al. Transforming themorbidity and mortality conference into an instrument forsystemwide improvement advances in patient safety: newdirections and alternative approaches (vol 2: culture andredesign). Rockville (MD): Agency for Healthcare Quality andResearch; 2008.

[21] Berenholtz SM, Hartsell TL, Pronovost PJ. Learning fromdefects to enhance morbidity and mortality conferences. AmJ Med Qual 2009;24:192.

[22] Aboumatar HJ, Blackledge CG Jr, Dickson C, et al. Adescriptive study of morbidity and mortality conferencesand their conformity to medical incident analysismodels: results of the morbidity and mortalityconference improvement study, phase 1. Am J Med Qual2007;22:232.

[23] Szekendi MK, Barnard C, Creamer J, Noskin GA. Using patientsafety morbidity and mortality conferences to promotetransparency and a culture of safety. Jt Comm J Qual PatientSaf 2010;36:1.

[24] Hayward RA, Heisler M, Adams J, Dudley RA, Hofer TP.Overestimating outcome rates: statistical estimationwhen reliability is suboptimal. Health Serv Res 2007;42:1718.

[25] Morton AP, Moore EE, Wohlauer MV, et al. Revisiting earlypostinjury mortality: are they bleeding because they aredying or dying because they are bleeding? J Surg Res 2013;179:5.

[26] Lozen YM, Cassin BJ, Ledgerwood AM, Lucas CE. Thevalue of the medical examiner as a member of themultidisciplinary trauma morbidity-mortality committee.J Trauma 1995;39:1054.

[27] Classen DC, Jaser L, Budnizt DS, et al. Adverse drug eventsamong hospitalized Medicare patients: epidemiology andnational estimates from a new approach to surveillance. JtComm J Qual Patient Saf 2010;36:12.

[28] MacKenzie EJ, Steinwachs DM, Bone LR, Floccare DJ,Ramzy AI. Inter-rater reliability of preventable deathjudgments: the Preventable Death Study Group. J Trauma1992;33:292.

[29] Rutledge R, Osler T. The ICD-9-based illness severity score:a new model that outperforms both DRG and APR-DRG aspredictors of survival and resource utilization. J Trauma1998;45:791.

[30] Juul S, Frydenberg M. An introduction to Stata for healthresearchers. 3rd edition. College Station (TX): StataCorp LP;2010. p. 179.

[31] Rosner B. Fundamentals of biostatistics. 7th edition. PacificGrove, CA: Brooks/Cole Cengage Learning; 2011.

[32] Gordis L. Epidemiology. Philadelphia: Saunders Elsevier;2009.

[33] Fleiss JL. Statistical methods for rates and proportions. 2ndedition. New York: John Wiley & Sons; 1981.

[34] Cutilli C. HUP mortality database analysis. Institutionalpresentation. Hospital of the University of PennsylvaniaMortality Review Committee Meeting; 2012.

[35] Stata/IC 12.0 statistical software, www.stata.com, CollegeStation (TX): StataCorp LP; 1985e2011.

[36] Circle Systems. Stat/Transfer: version 11. Seattle: CircleSystems, Inc; 2011.

j o u r n a l o f s u r g i c a l r e s e a r c h 1 8 4 ( 2 0 1 3 ) 5 4e6 060

[37] Wilson DS, McElligott J, Fielding LP. Identification ofpreventable trauma deaths: confounded inquiries? J Trauma1992;32:45.

[38] Hofer TP, Hayward RA. Identifying poor-quality hospitals:can hospital mortality rates detect quality problems formedical diagnoses? Med Care 1996;34:737.

[39] Zimmerman R, Pierson S, McLean R, et al. Aiming for zeropreventable deaths: using death review to improve care andreduce harm. Healthcare Q 2010;13:81.

[40] Aleccia J. Nurse’s suicide highlights twin tragedies of medicalerrors. Health Care on msnbc.com; June 27, 2011.

[41] Scott SD, Hirschinger LE, Cox KR, McCoig M, Brandt J,Hall LW. The natural history of recovery for the healthcareprovider “second victim” after adverse patient events. QualSaf Health Care 2009;18:325.

[42] WuAW.Medical error: thesecondvictim.Thedoctorwhomakesthe mistake needs help too [editorial]. BMJ 2000;320:726.

[43] Moty C, Michel P. Organizational and operational work ofdeath analysis committees: review of experiences. PresseMed 2001;30:259.

[44] Lau H, Litman KC. Saving lives by studying deaths: usingstandardized mortality reviews to improve inpatient safety.Jt Comm J Qual Patient Saf 2011;37:400.

[45] Vargas-Rosendo R, Aleman-Velazquez P, Jasso-Gutierrez L.[The Mortality Committee: a hospital need]. Bol Med HospInfantil Mexico 1992;49:683.

[46] MacKenzie EJ. Review of evidence regarding trauma systemeffectiveness resulting from panel studies. J Trauma 1999;47(3 Suppl):S34.

[47] Settervall CH, Domingues Cde A, Sousa RM, Nogueira Lde S.Preventable trauma deaths. Rev Saude Publica (Bogota) 2012;46:367.

[48] Office of Regulatory Affairs. Guidelines for determining IRBsubmission and defining human research. Philadelphia:University of Pennsylvania Institutional Review Board;http://www.upenn.edu/regulatoryaffairs/Documents/irbgui-1.pdf. 2009 [accessed 28.12.12].

[49] OHRP Quality Improvement Activities. Frequently askedquestions, http://www.hhs.gov/ohrp/qualityfaq.html[accessed 26.12.12].