Kottner

Anusca

Jana Cenb Facc De

International Journal of Nursing Studies xxx (2009) xxx–xxx

A R

Artic

Rece

Rece

Acce

Keyw

EPU

Pres

Risk

Rep

Reli

*

fax:

G Model

NS-1475; No of Pages 6

Plud

002

doi:

interrater reliability study of the assessment of pressure ulcer risking the Braden scale and the classification of pressure ulcers in a homere setting

Kottner a,*, Ruud Halfens b,c, Theo Dassen a

tre for Humanities and Health Sciences, Department of Nursing Science, Charite-Universitatsmedizin Berlin, Augustenburger Platz 1, 13353 Berlin, Germany

ulty of Health, Medicine and Life Sciences, Department of Health Care and Nursing Sciences, Universiteit Maastricht, The Netherlands

partment of Nursing Science, Universitat Witten/Herdecke, Germany

What is already known about the topic?

� The validity of pressure ulcer prevalence data can beseriously affected by measurement error.

T I C L E I N F O

le history:

ived 28 January 2009

ived in revised form 12 March 2009

pted 28 March 2009

ords:

AP

sure ulcers

assessment

roducibility

ability

A B S T R A C T

Background: Measurement error can seriously affect the validity of pressure ulcer risk

assessment and of pressure ulcer classification.

Objectives: Determination of interrater reliability and agreement of pressure ulcer risk and

pressure ulcers using the Braden scale and the EPUAP system.

Design and setting: Duplicate assessments by trained nurses during two nationwide

pressure ulcer prevalence surveys in the years 2007 and 2008 in The Netherlands in the

home care setting.

Participants: Home care clients which participated in 2007 (n = 352) and 2008 (n = 339) in

the pressure ulcer prevalence surveys.

Methods: The Braden scale was used to assess pressure ulcer risk. Skin examination was

conducted to detect pressure related tissue damages and to classify them according to the

EPUAP.

Results: In 2007 and 2008, Intraclass Correlation Coefficients for Braden scale sum scores

were 0.90 (95% CI: 0.88–0.92) and 0.88 (95% CI: 0.85–0.91) respectively, and

corresponding Standard Errors of Measurement were 1.00 and 0.98. 95% limits of

agreement were �2.8 to 2.8 and �2.7 to 2.7 respectively. The items ‘‘moisture’’, ‘‘sensory

perception’’ and ‘‘nutrition’’ contained largest amounts of measurement error. Proportions

of agreement for the classification of pressure ulcers were 96% and interrater reliability

was 0.81 and 0.79. Most disagreements were observed for the classification of grade 1

pressure ulcers.

Conclusions: The standardized study procedure applied in the annual nationwide pressure

ulcer prevalence surveys leads to reliable and reproducible results regarding pressure

ulcer risk and pressure ulcer prevalence in the home care setting. Researchers and

practitioners should be careful when drawing inferences from single pressure ulcer risk

factors included in the Braden scale. Descriptions of the items ‘‘moisture’’, ‘‘sensory

perception’’ and ‘‘nutrition’’ should be made more clearly and unambiguous.

� 2009 Elsevier Ltd. All rights reserved.

Corresponding author. Tel.: +49 30 450 529 054;

+49 30 450 529 900.

E-mail address: [email protected] (J. Kottner).

Contents lists available at ScienceDirect

International Journal of Nursing Studies

journal homepage: www.elsevier.com/ijns

ease cite this article in press as: Kottner, J., et al., An interrater reliability study of the assessment of pressure ulcer risksing the Braden scale and the classification of pressure ulcers in a home care setting. Int. J. Nurs. Stud. (2009),oi:10.1016/j.ijnurstu.2009.03.014

0-7489/$ – see front matter � 2009 Elsevier Ltd. All rights reserved.

10.1016/j.ijnurstu.2009.03.014

mailto:[email protected]

http://dx.doi.org/10.1016/j.ijnurstu.2009.03.014

http://www.sciencedirect.com/science/journal/00207489


J. Kottner et al. / International Journal of Nursing Studies xxx (2009) xxx–xxx2

G Model


� Empirical evidence regarding the interrater reliability ofthe Braden scale and its items is sparse.� Most interrater reliability studies of pressure ulcer

classifications are based on the assessment of imageswhich are not comparable to real skin assessment.� There is no empirical evidence to support the use of the

EPUAP pressure ulcer classification in clinical practice.

What this paper adds

� Braden scale sum scores lead to reproducible pressureulcer risk estimates when used nationwide by a varietyof different raters in the home care setting.� Nurses were able to differentiate and classify pressure

ulcers grade 2, 3, and 4 according to the EPUAP.� Single nurses are able to provide reliable pressure ulcer

data and it seems not necessary to do a second controlmeasurement.

1. Introduction

Pressure ulcers are a serious health problem especiallyamong older care dependent people. In the Netherlands,8.2% of nursing home residents and 3.9% of home careclients suffered from at least one pressure related tissueinjury in the year 2007 (Halfens et al., 2007).

Pressure ulcer prevalence rates provide valuableinsights into the magnitude of that health problem andthey can be used in planning for health resources andfacilities and for quality monitoring (Bours et al., 2004;Defloor et al., 2005). The validity of pressure ulcerprevalence data can be seriously affected by measurementerror. Often reported sources of error are over- andunderreporting (e.g. patients have ulcers which are notdocumented in the records) (Gunningerg and Ehrenberg,2004; Keong et al., 2004), non-response bias (Lahmannet al., 2006), and low interrater reliability (Allcock et al.,1994; Fletcher, 2001).

Since 1998, annual nationwide pressure ulcer preva-lence surveys are conducted in the Netherlands. Accordingto a standardized study protocol, data regarding pressureulcer risk and pressure ulcers are systematically collectedby trained nurses in various health care settings (Bourset al., 1999, 2002; Halfens et al., 2007, 2008).

The Braden scale is used to assess the pressure ulcerrisk (Bergstrom et al., 1987). Compared to other riskassessment scales it shows the best balance betweensensitivity and specificity and its score is a good predictorfor pressure ulcer risk (Pancorbo-Hidalgo et al., 2006). Thescale has six items reflecting contributing factors forpressure ulcer development: sensory perception, moist-ure, activity, mobility, nutrition, friction and shear.Individual items are scored on rating scales and thensummed up. Total scores range between 6 indicatingmaximum risk and 23 indicating no risk. It is assumed thatinterrater reliability for the Braden scale is high but thereis only limited evidence to support this (Kottner andDassen, 2008a). Kottner and Dassen (2008b) demon-strated that interrater reliability for Braden sum scores inthe nursing home setting is high, but it could also beshown that clinical relevant differences between raters

occurred. Furthermore, in research it is common practiceto make associations between single Braden scale itemsand other criteria of interest or to investigate whethersingle items are more able to predict pressure ulcer risk ascompared to the sum score (e.g. Bergquist, 2001; Fisheret al., 2004). In clinical practice item scores are used toprovide orientation to preventive measures that might betaken to prevent pressure ulcers (Bergstrom, 2008;Magnan and Maklebust, 2009). If Braden scale items arenot reliable these inferences might be misleading.Compared to all Braden scale items it could be shownthat ‘‘sensory perception’’ and ‘‘nutrition’’ containedlargest amounts of measurement error (Kottner andDassen, 2008b). These results are supported by findingsfrom Magnan and Maklebust (2009). They reportedreliability being lowest for ‘‘moisture’’ and ‘‘nutrition’’scores.

Pressure ulcers are defined according to the EuropeanPressure Ulcer Advisory Panel (EPUAP) as areas oflocalised damage to the skin and underlying tissuecaused by pressure, shear, friction and or a combinationof these (EPUAP, 1998). They are classified into fourgrades. Pressure ulcers grade 1 are defined as non-blanchable erythema of intact skin; grade 2 as partialthickness skin loss involving epidermis, dermis, or both,clinically visible as an abrasion or blister; grade 3 as fullthickness skin loss involving damage of subcutaneoustissue that may extend down to, but not through,underlying fascia; grade 4 as extensive destruction,tissue necrosis, or damage to muscle, bone, or support-ing structures with or without full thickness skin loss.Interrater reliability of the EPUAP classification wasinvestigated in several studies but results are contra-dictory (Kottner et al., 2009). Most reliability studieswere based on the assessment of images under artificialassessment situations. Therefore, results are hardlycomparable with the assessment of real skin andwounds in clinical practice (Beeckman et al., 2008;Kottner et al., 2009).

According to the Dutch protocol and adapted by theEPUAP (2005), skin of the surveyed patients is assessedby a team of two trained observers working indepen-dently. When identifying a pressure ulcer, both nurseshave to agree on the grade. It is assumed that thisenhances data quality. While this procedure is adoptedin the hospital and nursing home setting it is impossiblefor home care institutions to comply with this studyprotocol. Data collection in home care settings requiresmuch more effort and personal costs as compared tohospitals and nursing homes: most often nurses workalone, they have to cover time consuming distances tovisit the clients and they do not meet all clientseveryday. Therefore, also the measurements within theprevalence surveys are conducted by nurses alone. Tocontrol the data quality it is required that staff nursesthat are wound care specialists make a second inspec-tion of a random sample of patients which were alreadysurveyed (Bours et al., 1999).

The aim of this paper was to examine the degree ofinterrater reliability of the (1) Braden scale and itsconstituting items, (2) of the pressure ulcer diagnosis,

Please cite this article in press as: Kottner, J., et al., An interrater reliability study of the assessment of pressure ulcer riskusing the Braden scale and the classification of pressure ulcers in a home care setting. Int. J. Nurs. Stud. (2009),doi:10.1016/j.ijnurstu.2009.03.014


andcoll

2. M

2.1.

quadatnuronpreAddconass

tooskinranandindwewhweweorgraticha

Uniteestu

2.2.

mebetratethereliablGol200

itscalc

Tab

Dem

twic

Fem

Age

M

M

BMI

M

M

J. Kottner et al. / International Journal of Nursing Studies xxx (2009) xxx–xxx 3

G Model


Plud

(3) of the classification of pressure ulcers among dataecting nurses in the home care setting.

ethods

Design and sample

Each participating home care institution appointed alified coordinator who had the primary responsibility fora collection. Prior to data collection all data collectingses were trained by the institutional coordinator. Baseda PowerPoint presentation Braden scale items and

ssure ulcer classification were discussed and explained.itionally, nurses received written instruction manualstaining images of pressure ulcers, explanations ofessment scales and data collection forms.In 2007 and 2008, 5056 and 7922 home care clientsk part in the prevalence study. They were assessed and

examination was conducted by trained nurses. Adom sample of 352 clients of 27 institutions (year 2007)

339 clients of 21 institutions (year 2008) wereependently assessed a second time by nurses whore specially qualified in wound management. Nurseso conducted the first rating did not know which clientsre selected for a second rating and the second ratersre unaware of the results of the first ratings. Due toanisational conditions, time intervals between bothngs varied between 1 and 3 days. Demographicracteristics of the samples are shown in Table 1.Permission to conduct the study was given by theversity Hospital Maastricht’s Medical Ethical Commit-. Only those home care clients were included in thedy whose informed consent was given.

Statistical analysis

To gain detailed insight into the amount of measure-nt error due to different raters, we differentiatedween the concepts of interrater reliability and inter-r agreement. Agreement refers to the degree how closescores for repeated measurements were, whereas

ability refers to the degree how the instrument wase to differentiate among individuals (Stratford anddsmith, 1997; De Vet et al., 2006; Streiner and Norman,8).

To evaluate interrater reliability of the Braden scale anditems, Intraclass Correlation Coefficients (ICC) wereulated. As the home care clients were rated by a

different set of raters randomly selected from a populationof raters, a one-way random effects model seemed to beappropriate (Shrout and Fleiss, 1979). Proportions ofobserved agreement (po) and Standard Errors of Measure-ment (SEM) and were used to indicate interrater agree-ment. Proportions of observed agreement were calculatedby summing the frequencies of exact agreement in themain diagonal cells divided by the total number ofobservations. It indicates the proportion of exact con-cordance between raters across all ratings (Fleiss et al.,2003). However, this statistic is not appropriate forcalculating interrater agreement for the Braden scaleitems and Braden scale sum scores because only exactagreement is taken into account (Kottner and Dassen,2008a). The SEM is more appropriate in measuringinterrater agreement for interval-level data because itindicates how close the scores between raters within therated subjects were. The SEM expresses the measurementerror in the same units as the original measurement(Stratford and Goldsmith, 1997; De Vet et al., 2006;Streiner and Norman, 2008). We obtained SEM-values bycalculating the square root from the error variance fromthe ANOVA table. Furthermore, Bland–Altman plots and95% limits agreement were used to indicate agreement forthe Braden scale sum scores (Bland and Altman,1999).Interrater agreement and reliability for the diag-nosis pressure ulcer (yes/no) and for the classification ofpressure ulcers was calculated by po and Scott’s p statistic(Scott, 1955). The latter approach and its algebraicexpression is equal to the k statistic (Cohen, 1960), withthe difference among them lying in the definition of theproportion of expected or ‘chance’ agreement (pe). Giventhat p1 is the proportion of all ratings by rater 1 in onecategory (marginal total of rater 1) and p2 is the proportionof all ratings made by rater 2 in the same category(marginal total of rater 2) the proportion of expectedagreement in the p statistic is pe = ((p1 + p2)/2)2. In the kstatistic pe = (p1 p2). In Scott’s p statistic the individualrater’s marginal proportions are replaced by a commonestimate. Regarding our sampling this approach seemedappropriate since effects according to single raters werenot separable. Scott’s p corresponds to a one-way ANOVAfor continuous measurements ignoring rater effects andassuming the same probability of positive ratings of raters(Bodian, 1994; Dunn, 2004; Shoukri, 2004).

3. Results

Pressure ulcer risk assessment according to the Bradenscale was completed in 288 (year 2007) and 292 (year2008) cases. Interrater reliability and agreement coeffi-cients for the Braden scale and its items are shown inTable 2.

po-values ranged between 0.81 for ‘‘mobility’’ and 0.91for ‘‘friction & shear’’. ICC-values ranged between 0.64 (95%CI: 0.57–0.71) for the item ‘‘moisturetion’’ and a maximumof 0.91 (95% CI: 0.89–0.93) for the item ‘activity’. In bothyears lowest SEM-values were obtained for ‘‘friction &shear’’ and ‘activity’. For the item ‘‘moisture’’ SEM-valueswere highest in both years. Regarding Braden scale sumscores raters exactly agreed in 66% and 63% cases. ICC-

le 1

ographic characteristics of home care clients which were assessed

e.

2007 (n = 352) 2008 (n = 339)

ale (%) 68.0 62.8

(years)

ean (S.D.) 77.8 (11.8) 77.4 (13.4)

edian (IQR) 80 (74–86) 80 (73–86)

(kg/m2)

ean (S.D.) 27.5 (6.8) 27.0 (6.7)

edian (IQR) 26.2 (23.2–30.3) 26 (23.1–29.3)




G Model


values were 0.90 (95% CI: 0.88–0.92) and 0.88 (95% CI:0.85–0.91) and corresponding SEM-values were 1.00 and0.98 respectively. Differences between both ratings inrelation to their means are shown in Figs. 1 and 2. The sizeof the circles indicates the frequency of scores. In 2007, themaximum difference between scores was �6 to 8; in 2008the maximum difference was �7 to 9. 95% limits ofagreement were �2.8 to 2.8 in 2007 and �2.7 to 2.7 in2008.

Classifications of pressure ulcers are shown in Tables 3and 4. In 2007, regarding the presence versus the absenceof pressure ulcers raters exactly agreed in 338 cases(po = 0.96, p = 0.87 (95% CI: 0.77–0.93)). Interrater relia-bility across all five categories was p = 0.81 (95% CI: 0.73–0.88). In 2008, seven clients refused to have their skinexamined a second time. They were omitted from the

analysis. In the remaining sample (n = 332) interrateragreement for the diagnosis ‘pressure ulcer (yes/no)’ waspo = 0.96; interrater reliability was p = 0.89 (95% CI: 0.79–0.95). Interrater reliability across all five categories wasp = 0.79 (95% CI: 0.72–0.87). In both years most frequentdisagreements were observed regarding the presence orabsence of grade 1 pressure ulcers.

4. Discussion

Interrater agreement and interrater reliability was highfor the Braden scale sum scores in both years. About 90% ofvariance was due to measured differences according tothat instrument. Except for few cases most score differ-ences were between �3 and 3. Exact agreement (nodifference) would be ideal, but regarding the wide range ofpossible Braden scale scores (6–23) these smaller differ-ences can be regarded as being of minor clinical relevance.On the other hand, if differences increase up to four or evenmore, it is likely that different courses of action are taken(Kottner and Dassen, 2008b). There were no relationsbetween the amount of differences and the means ofscores. Regarding the Braden scale items some differencesbecame apparent. In 2008, interrater reliability for‘Moisture’ was lowest, and disagreement expressed bythe SEM was highest compared to all other items. In bothyears reliability was also low for ‘‘sensory perception’’ and‘‘nutrition’’. These results are comparable with previousresearch findings from other settings (Bours et al., 1999;Halfens et al., 2000; Kottner and Dassen, 2008b; Magnanand Maklebust, 2009). It is highly probable that relativehigh amounts of measurement error were due to pooroperational definitions of those items (Kottner and Dassen,2008b; Magnan and Maklebust, 2009). Our results suggestthat researchers and practitioners should be careful whendrawing inferences from single items like ‘‘moisture’’ or

Table 2

Interrater reliability and agreement coefficients for the Braden scale and

its items.

Year Items po ICC(1,1) (95% CI) SEM

2007 (n = 288) Sensory

perception

0.87 0.71 (0.65–0.76) 0.34

Moisture 0.88 0.81 (0.77–0.85) 0.36

Activity 0.87 0.91 (0.89–0.93) 0.26

Mobility 0.82 0.86 (0.83–0.89) 0.34

Nutrition 0.83 0.78 (0.73–0.82) 0.35

Friction & Shear 0.91 0.89 (0.87–0.92) 0.24

Total score 0.66 0.90 (0.88–0.92) 1.00

2008 (n = 292) Sensory

perception

0.85 0.74 (0.69–0.79) 0.34

Moisture 0.85 0.64 (0.57–0.71) 0.40

Activity 0.87 0.88 (0.85–0.90) 0.27

Mobility 0.81 0.82 (0.78–0.85) 0.38

Nutrition 0.84 0.79 (0.75–0.83) 0.29

Friction & Shear 0.90 0.83 (0.79–0.86) 0.25

Total score 0.63 0.88 (0.85–0.91) 0.98

Fig. 1. Braden score: difference of ratings versus average of scores of both ratings in 2007 (n = 288).



‘‘nuquilowbyDas

diationdiffpregracanthebe

Tab

Clas

Seco

N

Gr

Gr

Gr

Gr

To

Tab

Clas

Seco

N

Gr

Gr

Gr

Gr

To

J. Kottner et al. / International Journal of Nursing Studies xxx (2009) xxx–xxx 5

G Model


Plud

trition’’, because the amount of measurement error iste high. On the other hand, measurement error wasest for ‘‘activity’’. Again, these findings are supportedprevious research from other settings (Kottner andsen, 2008b; Magnan and Maklebust, 2009).Interrater agreement and interrater reliability for thegnosis of pressure ulcers and pressure ulcer classifica-

was high. In general, data collectors were able toerentiate between individuals with and withoutssure ulcers and between different pressure ulcerdes. Disagreements regarding grade 1 pressure ulcersbe explained by difficulties regarding the diagnosis andnature of this phenomenon. It is well known that it maydifficult to accurately differentiate between blanching

and non-blanching erythema (Halfens et al., 2001; Bethell,2003). Further, when preventive measures are appliedeffectively these early signs of skin damage are formedback (Vanderwee et al., 2007). This may have been the casein our study, because the time interval between bothassessments was not known exactly. However, our resultssupport the assumption that precision of pressure ulcerprevalence estimates could be increased when grade 1pressure ulcers are excluded (EPUAP, 2005).

5. Limitations

Our study had limitations. First, local study coordina-tors selected home care clients for the second assessmenton their own. They were required to do this randomly butthis was not performed randomly in a strict sense.Consequently, there may be a selection bias. On the otherhand, the proportions of home care clients with pressureulcer in our subsample correspond to the observedpressure ulcer prevalence rates in the total samples quitewell (Halfens et al., 2007, 2008). Second, nurses knew thata second rater may check their assessment results. Thismay had an effect on precision and diagnostic accuracy.There is evidence that interrater reliability declines underless controlled conditions (Topf, 1988).

6. Conclusions

Results indicate that the standardized study procedureapplied in the annual nationwide pressure ulcer preva-lence surveys in The Netherlands leads to reliable andreproducible results regarding pressure ulcer risk andpressure ulcer prevalence in the home care setting. Theimpact of measurement errors due to different ratersseems to be small. Consequently, a second controlmeasurement seems not necessary. This would savefinancial and human resources in future prevalencesurveys. Practitioners must be careful when using theBraden scale items ‘‘sensory perception’’, ‘‘moisture’’, and

le 3

sification of pressure ulcers in 2007 (n = 352).

First rating

No Grade 1 Grade 2 Grade 3 Grade 4 Total

nd rating

o 308 5 0 0 0 313

ade 1 4 15 1 0 0 20

ade 2 0 1 8 0 0 9

ade 3 1 0 1 4 1 7

ade 4 0 0 0 0 3 3

tal 313 21 10 4 4 352

le 4

sification of pressure ulcers in 2008 (n = 332).

First rating

No Grade 1 Grade 2 Grade 3 Grade 4 Total

nd rating

o 292 1 2 0 0 295

ade 1 3 15 0 1 0 19

ade 2 1 1 6 1 0 9

ade 3 0 0 1 3 1 5

ade 4 0 0 0 2 2 4

tal 296 17 9 7 3 332

Fig. 2. Braden score: difference of ratings versus average of scores of both ratings in 2008 (n = 292).




G Model


‘‘nutrition’’ for care planning because reproducibility is toolow. The relative precision of the Braden scale could beenhanced when the descriptions of the items are put moreclear and unambiguous.

Acknowledgements

The annual pressure ulcer prevalence surveys weresupported by the participating institutions.

Conflict of interest. There are no conflicts of interest to declare.

Ethical approval. Permission to conduct the study was given

by the University Hospital Maastricht’s Medical Ethical

Committee.

References

Allcock, N., Wharrad, H., Nicolson, A., 1994. Interpretation of pressure-sore prevalence. Journal of Advanced Nursing 20 (1), 37–45.

Beeckman, D., Schoonhoven, L., Boucque, H., Van Maele, G., Defloor, T.,2008. Pressure ulcers: e-learning to improve classification by nursesand nursing students. Journal of Clinical Nursing 17 (13), 1697–1707.

Bergquist, S., 2001. Subscales, subscores, or summative score: evaluatingthe contribution of Braden scale items for predicting pressure ulcerrisk in older adults receiving home health care. Journal of Wound andContinence Nursing 28 (6), 279–289.

Bergstrom, N., 2008. On accuracy. Journal of Wound Ostomy and Con-tinence Nursing 35 (2), 211–212.

Bergstrom, N., Braden, B.J., Laguzza, A., Holman, V., 1987. The Braden scalefor predicting pressure sore risk. Nursing Research 36 (4), 205–210.

Bethell, E., 2003. Controversies in classifying and assessing grade I pres-sure ulcers. Journal of Wound Care 12 (1), 33–36.

Bland, J.M., Altman, D.G., 1999. Measuring agreement in method compar-ison studies. Statistical Methods in Medical Research 8, 135–160.

Bodian, C.A., 1994. Intraclass correlation for two-by-two tables underthree sampling designs. Biometrics 50 (1), 183–193.

Bours, G.J.J.W., Halfens, R.J.G., Abu-Saad, H.H., Grol, R.T.P.M., 2002. Pre-valence, prevention, and treatment of pressure ulcers: descriptivestudy in 89 institutions in The Netherlands. Research in Nursing andHealth 25 (2), 99–110.

Bours, G.J.J.W., Halfens, R.J.G., Candel, M.J., Grol, R.T.P.M., Abu-Saad, H.H.,2004. A pressure ulcer audit and feedback project across multi-hospital settings in the Netherlands. International Journal for Qualityin Health Care 16 (3), 211–218.

Bours, G.J.J.W., Halfens, R.J.G., Lubbers, M., Haalboom, J.R.E., 1999. Thedevelopment of a national registration form to measure the preva-lence of pressure ulcers in The Netherlands. Ostomy Wound Manage-ment 45 (11), 28–40.

Cohen, J., 1960. A coefficient of agreement for nominal scales. Educationaland Psychological Measurement 20 (1), 37–46.

De Vet, H.C.W., Terwee, C.B., Knol, D.L., Bouter, L.M., 2006. When to useagreement versus reliability measures. Journal of Clinical Epidemiol-ogy 59 (10), 1033–1039.

Defloor, T., Clark, M., Witherow, A., Colin, D., Lindholm, C., Schoonhoven,L., Moore, Z., 2005. EPUAP statement on prevalence and incidencemonitoring of pressure ulcer occurrence 2005. EPUAP Review 6 (3),74–80.

Dunn, G., 2004. Statistical Evaluation of Measurement Errors, 2nd ed.Hodder Arnold, London.

European Pressure Ulcer Advisory Panel, 1998. Pressure Ulcer TreatmentGuidelines (available at: http://www.epuap.com/gltreatment.html).

European Pressure Ulcer Advisory Panel (EPUAP), 2005. EPUAP state-ment on prevalence and incidence monitoring of pressure ulcer

occurrence 2005 (available at: http://www.epuap.org/review6_3/page5.html).

Fisher, A., Wells, G., Harrison, M., 2004. Factors associated with pressureulcers in adults in acute care hospitals. Advances in Skin & WoundCare 17 (2), 80–90.

Fleiss, J.L., Levin, B., Paik, M.C., 2003. Statistical Methods for Rates andProportions, 3rd ed. Wiley, New Jersey.

Fletcher, J., 2001. How can we improve prevalence and incidencemonitoring? Journal of Wound Care 10 (8), 311–314.

Gunningerg, L., Ehrenberg, A., 2004. Accuracy and quality in the nursingdocumentation of pressure ulcers: a comparison of record contentand patient examination. Journal of Wound Ostomy and ContinenceNursing 31 (6), 328–335.

Halfens, R.J.G., Van Achterberg, T., Bal, R.M., 2000. Validity and reliabilityof the Braden scale and the influence of other risk factors: a multi-centre prospective study. International Journal of Nursing Studies 37(4), 313–319.

Halfens, R.J., Bours, G.J., Van Ast, W., 2001. Relevance of the diagnosis‘stage 1 pressure ulcer’: an empirical study of the clinical course ofstage 1 ulcers in acute care and long-term care hospital populations.Journal of Clinical Nursing 10 (6), 748–757.

Halfens, R., Meijers, J.M.M., Neyens, J.C.L., Offermans, M.P.W., 2007.Rapportage resultaten landelijke prevalentiemeting zorgproblemen2007 [Report of the results of the nationwide prevalence study of careproblems]. Universiteit Maastricht, Maastricht.

Halfens, R., Schols, J.M.G.A., Meijers, J.M.M., Neyens, J.C.L., Offermans,M.P.W., 2008. Rapportage resultaten landelijke prevalentiemetingzorgproblemen 2008 [Report of the results of the nationwide pre-valence study of care problems]. Universiteit Maastricht, Maastricht.

Keong, N., Ricketts, D., Alakeson, N., Rust, P., 2004. Pressure sores follow-ing elective total hip arthroplasty: pitfalls of misinterpretation.Annals of the Royal College of the Surgeons of England 86 (3),174–176.

Kottner, J., Dassen, T., 2008a. Interpreting interrater reliability coefficientsof the Braden scale: a discussion paper. International Journal ofNursing Studies 45 (8), 1238–1246.

Kottner, J., Dassen, T., 2008b. An interrater reliability study of the Bradenscale in two nursing homes. International Journal of Nursing Studies45 (10), 1501–1511.

Kottner, J., Raeder, K., Halfens, R., Dassen, T., 2009. A systematic review ofinterrater reliability of pressure ulcer classification systems. Journalof Clinical Nursing 18, 315–336.

Lahmann, N., Halfens, R.J.G., Dassen, T., 2006. Effect of non-response biasin pressure ulcer prevalence studies. Journal of Advanced Nursing 55(2), 230–236.

Magnan, M.A., Maklebust, J., 2009. The effect of web-based Braden scaletraining on the reliability of Braden subscale ratings. Journal ofWound Ostomy and Continence Nursing 36 (1), 51–59.

Pancorbo-Hidalgo, P.L., Garcia-Fernandez, F.P., Lopez-Medina, I.M.,Alvarez-Nieto, C., 2006. Risk assessment scales for pressure ulcerprevention: a systematic review. Journal of Advanced Nursing 54(1), 94–110.

Scott, W.A., 1955. Reliability of content analysis: the case of nominal scalecoding. Public Opinion Quarterly 19, 321–325.

Shoukri, M.M., 2004. Measures of Interobserver Agreement. Chapman &Hall/CRC, Boca Raton.

Shrout, P.E., Fleiss, J.L., 1979. Intraclass correlations: uses in assessingrater reliability. Psychological Bulletin 86 (2), 420–428.

Stratford, P.W., Goldsmith, C.H., 1997. Use of the standard error as areliability index of interest: an applied example using elbow flexorstrength data. Physical Therapy 77 (7), 745–750.

Streiner, D.L., Norman, G.R., 2008. Health Measurement Scales: A PracticalGuide to Their Development and Use, 4th ed. Oxford University Press,Oxford.

Topf, M., 1988. Interrater reliability decline under covert assessment.Nursing Research 37 (1), 47–49.

Vanderwee, K., Grypdonck, M., Defloor, T., 2007. Non-blanchable erythemaas an indicator for the need for pressure ulcer prevention: a rando-mized-controlled trial. Journal of Clinical Nursing 16 (2), 325–335.


http://www.epuap.com/gltreatment.html


Documents

Kottner