The Nursing Outcomes Classification:Validation by Rehabilitation Nurses
Cindy A. Scherb, MS RNCarla Gene Rapp, MNSc RN CRRNMarion Johnson, PhD RNMeridean Maas, PhD RN FAAN
Measuring patient outcomes is im-portant to rehabilitation nurses andthe patients they serve. This article de-scribes researchconducted at the Uni-versity ofIowa College ofNursing todevelop the Nursing Outcomes Clas-sification (NO C) and the validationofthis research by surveys conductedthrough specialty nursing organiza-tions, particularly the Association ofRehabilitation Nurses. Nurses re-sponded to surveys designed to vali-date (a) the importance ofoutcomeindicators to the achievement ofanoutcome and (b) nursing's contribu-tion to the achievement ofthe indica-tors. The results ofthe surveys indi-cated that rehabilitation nursesbelieve that nursing makes a sub-stantial contribution to most outcomesand indicators.
Address correspondence to CindyScherb, 4440 State Highway 22,Kiester, MN 56051.
It is necessary to use standardized language to articulate what nurses do. The devel-opment of standardized language for nursing will allow nurses to communicate moreeffectively with other nurses, nationally and internationally, as well as with members ofother disciplines. Although comprehensive classifications of nursing diagnoses (NorthAmerican Nursing Diagnosis Association [NANDA], 1996) and nursing interventions(Iowa Intervention Project, 1996) have been developed, standardized outcomes that aresensitive to the effects of nursing interventions (i.e., nursing-sensitive outcomes) werenot available until work on the Nursing Outcomes Classification (NOC) began at theUniversity ofIowa College of Nursing in 1991. This article describes the developmentofNOC and the specific results of surveys to assess the content validity of the NOC out-comes. One group of surveys was completed by expert nurses who are members of theAssociation of Rehabilitation Nurses (ARN).
The work of the NOC research team is important to nurses because no comprehen-sive standardized language for nursing outcomes exists (Maas, Johnson, & Moorhead,1996). As healthcare reform issues remain in the national spotlight, standardized lan-guage that identifies and measures nurse-influenced patient outcomes is essential fornursing to compete in the managed care environment (Jennings, 1991; Marek, 1989).Consumers are demanding to know what outcomes they can expect from the dollars theyspend on health care, and they are holding all healthcare professionals accountable forthe achievement of these outcomes (Hegyvary, 1991; Kelly, Huber, Johnson, McCloskey,& Maas, 1994). Because of the pressure on healthcare providers to demonstrate results,nurses must be able to demonstrate and justify their contributions to patient care (Jones,1993).
Rehabilitation nurses have realized the need to identify outcomes pertinent to theirpatient populations. The need for quality rehabilitation services is the driving force be-hind the documentation of outcomes obtained from rehabilitation nursing interventions(McCourt, 1993). The Rehabilitation Nursing Foundation (RNF) published 21 Reha-bilitation Nursing Diagnoses: A Guide to Interventions and Outcomes in 1995, and in1996 RNF set a research agenda for rehabilitation nursing. One of the five priorities RNFidentified was "rehabilitation nurse-sensitive outcomes and costs in the continuum ofcare and interdisciplinary seUing(s)" (Nelson, 1996, p. 3).
BackgroundThe primary purpose of the NOC research, which is ongoing, is to conceptualize, la-
bel, define, and classify patient outcomes and indicators sensitive to nursing care. Oth-er.purposes are to evaluate the validity and usefulness of the classification in field testsand to define and test measurement procedures for the outcomes and indicators (IowaOutcomes Project, 1997).
For purposes of this research, "a nursing-sensitive patient outcome is defined as avariable patient or family caregiver state, behavior, or perception that is responsive tonursing intervention and conceptualized at middle levels of abstraction (e.g., mobility
174 Rehabilitation Nursing' Volume 23, Number 4' Jul/Aug 1998
level, nutritional status, health beliefs)" (Iowa Outcomes Proj-ect, 1997, p. 21). The outcomes are stated as concepts that canbe measured along a continuum, rather than as goals. Most ofthe nursing-sensitive patient outcomes characterize the resolu-tion of nursing diagnoses, but not all outcomes are necessarilynursing diagnosis-specific (Maas et al., 1996). The outcomesincluded in NOC are nursing-sensitive outcomes in that theyare influenced by nursing care; however, nursing is not the onlydiscipline responsible for the outcomes (Johnson et al., 1996).
Rehabilitation nurses have realized theneed to identify outcomes pertinent to
Each outcome is defined specifically by a group of associ-ated indicators. An indicator is defined as "variable patient orfamily caregiver states, behaviors, or perceptions at a low lev-el of abstraction that are responsive to nursing intervention andused for determining a patient outcome (e.g., for the outcomeMobility Level, indicators include 'joint movement: 'transferperformance: 'ambulation: walking')" (Iowa Outcomes Proj-ect, 1997, p. 21). Thus, indicators are needed to measure specificoutcomes and to reflect the dimensions or aspects of more gen-eral outcomes.
All outcome labels and indicators are measured on a 5-pointLikert-type scale, with 5 being the most desirable patient state.A 5-point scale allows for demonstration of variability in a pa-tient state, behavior, or perception. The outcomes are designedto be measured over time. Measurements are made, at a mini-mum, on admission or during the initial nurse visit (after nurs-ing diagnoses are established) and upon discharge, transfer, orreferral to another agency or healthcare provider (when the sta-tus of each nursing diagnosis is documented).
The NOC research team includes 17 investigators and addi-tional clinical content experts representing eight focus groups.The focus group categories are based on the Medical OutcomesStudy (MOS) (Tarlov et al., 1989) and on work completed with-in nursing by Lang and Clinton (1984) and Marek (1989). NOCfocus group categories include social and role status; physicalfunctional status; safety status; perceived well-being; family care-giver status; physiological status; psychological and cognitivestatus; and health attitudes, knowledge, and behavior. Outcomelabels for the focus group categories are shown in Figure 1.
Each focus group was responsible for identifying outcomelabels by reviewing nursing and allied health research and oth-er pertinent literature. A concept analysis was completed to re-fine outcome labels, clarify definitions, and develop indicatorsfor each outcome. Each focus group was responsible for re-viewing the literature to identify a recommended method ofmeasuring its outcomes (Johnson et al., 1996).
NOC was published in March 1997 with 190 nursing-sen-sitive outcome labels, definitions, indicators, measurementscales, and corresponding background readings. An example,"Self-Care: Activities of Daily Living (ADL)," is shown in
Table 1. Outcomes continue to be identified and developedthrough literature review and feedback from nurses who are fa-miliar with the classification (Iowa Outcomes Project, 1997).
Another step in the development and validation ofNOC wasa survey of master's degree-prepared nurses who representeddifferent clinical specialties and settings and who provided carefor clients in various age groups. ARN was one of the special-ty groups from which these nurses were drawn. Outcomes per-tinent to rehabilitation were selected for inclusion in the sur-vey. The nurses were asked to rate the importance of eachindicator to its outcome label and each indicator's sensitivityto nursing intervention (Maas et al., 1996).
MethodsSample: Nurses were selected to receive surveys based on
their clinical expertise and their familiarity with research tech-niques and the need for classification systems. The names andaddresses of expert rehabilitation nurses were obtained fromARN, and a random sample of 118 nurses was drawn.
Figure 1. Examples of Outcome Labels by FocusGroup Category
Social and Role Status Psychosocial Adjustment: Life Change Social Involvement
Physical Functional Status Mobility Level Self-Care: Activities of Daily Living
Safety Status Infection Status Safety Behavior: Home Physical Environment
Perceived Well-Being Acceptance: Health Status Quality of Life
Family Caregiver Status Caregiver Emotional Health Caregiver-Patient Relationship
Physiological Status Energy Conservation Endurance
Psychological and Cognitive Status Body Image Self-Esteem
Health Attitudes, Knowledge, and Behavior Compliance Behavior Knowledge: Treatment Regimen
Rehabilitation Nursing > Volume 23, Number 4 Jul/Aug 1998 175
Nursing Outcomes Classification
Surveys: The surveys were developed to include outcomesfrom similar specialty areas. Each survey contained between 4and 10 outcome concepts and 6 to 15 nursing-sensitive indica-tors for each outcome. The nurse experts were asked to rate eachindicator as to the importance of the indicator for measuringthe outcome (1 =not at all important, 5 =very important) andthe contribution of nursing to the achievement of the indicator(i.e., nursing sensitivity) (l =no contribution, 5 =contributionis mainly nursing). The nurses were also asked to suggest anyadditional indicators and, if they wished, to modify the indica-tors presented in the survey.
Each nurse received a survey based on his or her area of prac-tice. Fourteen NOC outcome labels were determined to be per-tinent for rehabilitation nurses' practice. Because the surveyswere designed to contain 4 to 10 outcome labels, two differentsurveys were sent to the rehabilitation nurses. Survey A con-tained 4 outcome labels and associated indicators, and SurveyB contained 10 outcome labels and associated indicators.
The nurses were asked to return the completed survey in astamped, addressed envelope within 2 weeks of receiving it. (Apostcard reminder was sent 3 weeks after the survey mailing.)Sixty-eight nurses received Survey A; 31 (45%) completed andreturned it. Fifty nurses received Survey B; 22 (44%) complet-ed and returned it. Fourteen of the nurses who received SurveyB identified rehabilitation as their specialty area of practice.Others identified general adult medicine (1), specialty medi-cine (1), specialty surgery (1), community health (2), and oth-er (2); one person did not indicate a specialty. Fifteen of thenurses who responded to Survey A identified rehabilitation as
Table 1. Example of an NOC Nursing-Sensitive Outcome
their specialty area. Other specialties were general adult med-icine (4), specialty medicine (1), gerontology (6), administration(2), home health (1), education (1), and admissions (1). The av-erage number of years of practice in the identified specialty was11.87 years for those who responded to Survey A and 10.93years for those who responded to Survey B. Forty-three of the53 respondents to the surveys were currently certified by a pro-fessional organization, and 35 of the 43 certified nurses wereCertified Registered Rehabilitation Nurses (CRRNs).
Survey resultsContent validity and sensitivity to nursing interventions were
estimated using a modification of the Fehring (1987) method-ology. Nurse experts were asked to rate each indicator for itsimportance in measuring the nursing-sensitive outcome and forthe contribution of nursing to achievement of the indicator (i.e.,sensitivity). Weighted ratios for each indicator statement werecalculated by adding all the individual responses and dividing thetotal by the total number of responses. The weights suggested byFehring are 5 = 1.00,4 = 0.75,3 = 0.50,2 = 0.25, and 1 = 0.1fthe weighted ratio scores for importance and contribution werehigher than 0.80 for a given indicator, the indicator was con-sidered to be a critical indicator. If the weighted ratio scoreswere higher than 0.60, the indicator was still considered im-portant and was retained. If the weighted ratio scores were low-er than 0.59, the research team evaluated whether the indicatorshould be retained. Most indicators were above the 0.60 level,and many were above the 0.80 level. The overall outcome con-tent validity and outcome sensitivity values were calculated by
Self-Care: Activities of Daily Living (ADL)Definition: Ability to perform the most basic physical tasks and personal care activities
Dependent, Requires assistive Requires assistive Independent with Completelydoes not participate person and device person assistive device independent
Self-Care: ADL 2 3 4 5
IndicatorsEating 2 3 4 5Dressing 2 3 4 5Toileting 2 3 4 5Bathing 2 3 4 5Grooming 2 3 4 5Hygiene 2 3 4 5Oral hygiene 2 3 4 5Ambulation: Walking 2 3 4 5Ambulation: Wheelchair 2 3 4 5Transfer performance 2 3 4 5Other (specify) 2 3 4 5
From Nursing Outcomes Classification (NOC), by Iowa Outcomes Project, M. Johnson and M. Maas (Eds.), 1997, St. Louis:Mosby. Reprinted with permission.
176 Rehabilitation Nursing> Volume 23, Number 4 luI/Aug 1998
averaging the individual indicator importance and contributionscores. Table 2 shows the overall content validity and nursingsensitivity scores for each outcome label.
The research team reviewed the expert nurses' suggestionsand made changes as appropriate. The results of the surveys in-dicated that rehabilitation nurses believe that the indicators areimportant to evaluation of the outcome concepts, that nursing'scontribution to most outcomes and indicators is substantial, andthat no important indicators were missing.
Summary and nursing implicationsARN members assisted in validating outcome labels and in-
dicators pertinent to the rehabilitation specialty.The nurses sub-stantiated that the indicators and outcome labels are sensitiveto nursing intervention and that the indicators are important inmeasuring the outcomes. Survey findings from rehabilitationnurses and members of other nursing specialty groups "provideevidence that the selection, development, and refinement ofnursing-sensitive outcomes and indicators, based on the quali-tative and concept analyses, were valid" (Iowa Outcomes Proj-ect, 1997, p. 39).
Through these survey results, rehabilitation clinicians haveshown that NOC is pertinent to the practice of rehabilitationnursing. However, rehabilitation nurses were surveyed only onselected NOC outcomes, and many more NOC patient outcomescan be used in determining the outcomes of rehabilitation nurs-ing care.
Rehabilitation nurse clinicians have already taken a step to-ward identifying interventions and outcomes pertinent to theirnursing practice through 21 Rehabilitation Nursing Diagnoses:A Guide to Interventions and Outcomes (RNF, 1995). Com-municating these outcomes using standardized language pro-vided by NOC will allow for comparison of nursing practiceacross settings a...