Conceptualizing performancein accreditationPERNELLE A. SMITS1, FRANCOIS CHAMPAGNE2, DAMIEN CONTANDRIOPOULOS2, CLAUDE SICOTTE2

AND JOHANNE PREVAL2

1PhD Program in Public Health, University of Montreal, Faculty of Medicine, Montreal, Quebec, Canada, and 2GRIS—Groupe deRecherche Interdisciplinaire en Sante, University of Montreal, Montreal, Quebec, Canada

Abstract

Objectives. To compare the conceptualization of performance underlying different accreditation manuals.

Data sources. Accreditation manuals were selected from the 2003 WHO report titled ‘Quality and Accreditation inHealthcare Services’. We used manuals from WHO-listed countries that most influenced the standards: Canada, France, theUSA and Australia. The fifth manual is published by the Pan American Health Organization (PAHO).

Extraction methods. Standards from each manual were classified by two independent reviewers. The coding grid, which wasbased on a Parsonian-based integrative framework on performance, was composed of performance dimensions and theirinterlinks/alignments.

Principal findings. The four dimensions of quality, goal-attainment, adaptation to the external environment and values, alongwith their alignments, were given differing levels of importance in the five manuals. The Australian manual emphasizes allfour dimensions and their alignments. The PAHO accreditation focuses mainly on quality. The manuals from Canada, Franceand the USA fall somewhere between the two accreditation extremes of complete versus one-dimensional. Finally, we presenta taxonomy of the conceptualization of performance in accreditation manuals that distinguishes between quality-oriented andalignment-oriented accreditation manuals.

Conclusions. Specific conceptualizations of performance underlying accreditation manuals may not be neutral. Perhaps, morenormative accreditation manuals are associated with authoritative management styles, or more balanced accreditation manualswith comprehensive management styles. Our comparative analysis is a first step toward better understanding the relationshipbetween the conceptualization of performance and the management style adopted in a particular healthcare organization.This relationship could help explain the variation observed in healthcare organization performance.

Keywords: accreditation, framework, management, Parsonian perspective, performance

Introduction

Accreditation is a procedure that is being used with increasingfrequency around the world. It encompasses elements of self-assessment, field survey, reporting [1] and subsequent follow-up[2]. Accreditation is constructed around norms or standardsrelated to the inputs, processes and outputs with which organiz-ations must comply in order to receive accreditation. Originally,the primary goal of healthcare organization accreditation was toimprove the performance of health systems through the stan-dardization of practices and quality improvement [3]. It thenalso became a locus for social change [4]. Since 1990, thenumber of programs being used worldwide has doubled [5].Although the accreditation processes of different countriesshare a common goal of improving the performance of health-care organizations, each country has developed its own particu-lar solution to achieving this goal.

There has been very little research comparing the contentof accreditation manuals since the descriptive work ofSegouin [6] and the taxonomy proposed by Scrivens [7].Seguoin gives an overall description of accreditation pro-cesses, including information such as history, cost, length ofthe accreditation process, the grading system for review visitsand the content of accreditation manuals. Scrivens elaborateson the dimensions used in constructing accreditation pro-cesses, such as grading schema, voluntary and compulsoryparticipation, the employment status of surveyors, the pur-poses of accreditation and the focus of standards, but doesnot provide any details on methodology. Neither Seguoinnor Scrivens analyze accreditation standards per se.Accreditation targets the inputs, processes and outputs

that an organization is supposed to reach. The level of attain-ment for each target in accreditation manuals is measured bythe organization’s performance. An organization that

Address reprint requests to: Smits Pernelle, Sante Publique, University of Montreal, CP6128 Succ., Centre Ville, 1420 MontRoyal—3rd Floor, Montreal, Quebec, Canada H3C 3J7. Tel: þ1 514 343 7365; Fax: þ1 514 343 2207; E-mail: pernelle.smits@umontreal.ca

International Journal for Quality in Health Care vol. 20 no. 1

# The Author 2007. Published by Oxford University Press on behalf of International Society for Quality in Health Care; all rights reserved 47

International Journal for Quality in Health Care 2008; Volume 20, Number 1: pp. 47–52 10.1093/intqhc/mzm056Advance Access Publication: 17 November 2007

performs well with respect to one target may perform lesswell with respect to another. Accordingly, a conceptualizationof performance focused on a single target or dimensioncould impair the overall performance of the organizationbecause some areas are underrepresented. Therefore, wehave based our analysis of performance on a multidimen-sional definition of performance. Organizational perform-ance can be defined in different ways depending on theschool of thought: the attainment of objectives in the rationalgoal model, the achievement of quality in the internalprocess model, adaptation to the external environment in theopen model or relationships in the human relation model.There exist very few models that integrate the differentcomplementary definitions of performance. Indeed, manyperformance measurement systems have been documented[8, 9]. Some conceptualize performance in business-liketerms [10, 11] and exclude staff satisfaction, whereas othersare more comprehensive but difficult to operationalize, suchas the EFQM Business Excellence Model [8]. Moreover, theEFQM and the Balanced Score Card [12] are aimed atprivate organizations and do not necessarily incorporate thespecificities of public healthcare organizations [13].For our analysis, we needed a comprehensive model of per-

formance adapted to the healthcare field. We therefore used theParsonian-based integrative model of performance in health-care organizations that was developed by Sicotte et al. [14] andused to construct a performance assessment framework of hos-pitals in Europe [15]. This model has many advantages. First,the complexity of organizational performance is reflected in itsdefinition as a multidimensional concept. Second, the assess-ment of organizational performance examines more than justthe dimensions per se. It takes into account the relationshipsbetween these dimensions: what is the level of productionreached on the basis of available resources? Is the innovationcapacity of the organization appropriate given its stated goals?The concept of performance needs to include the relationshipsbetween dimensions. We believe the model of performancedeveloped by Sicotte is well-suited to evaluating organizationalperformance in all its complexity.The aim of this paper is to analyze the concept of per-

formance underlying five recognized accreditation manualsfrom the countries of Australia, the USA, France, Canadaand from the Pan American Health Organization (PAHO).We examine which dimensions of performance were includedin these accreditations (manual) through a descriptive analysisof the manuals. After selecting the manuals, we analyzed theavailable ones using the integrative framework on healthcareorganization performance. We classified the standards ofeach accreditation manual according to the framework’sdimensions of performance.

Methodology

Selection of accreditations

We used a quantitative method for pre-selecting recognizedaccreditations based on the volume of citation for each

WHO-listed country that most influenced the selection ofstandards (WHO report) [16]. We counted the frequency ofcitation of program X listed as influential (frequency ¼number of times program X is cited/total number ofanswers from responding agencies). The results for frequencyof citation were 34% for the US program, 24% for theCanadian, 24% for the Australian, 9% for the English and3% for the Japanese and Polish.For our in-depth analysis, we selected the most commonly

cited programs for which a hospital accreditation manual wasavailable to us. We were not able to analyze the accreditationmanual used in Great Britain, as we had no version of it andit was not available to the public. The Japanese manual wasalso excluded, as a translated version was not available. TheFrench manual was included to extend the analysis to aEuropean country. The PAHO manual was included toextend the analysis to low- and middle-income countries. Thefinal accreditations retained for our analysis are as follows, bycountry/organization:(i) Canada: Achieving Improved Measurement Program,

Canadian Council on Health Services Accreditation[17].

(ii) USA: Comprehensive Accreditation Manual for Hospitals:The Official Handbook, Joint Commission onAccreditation of Healthcare Organizations [18].

(iii) France: Accreditation for Healthcare Organizations: SecondAccreditation Procedure, Agence Nationaled’accreditation et d’evaluation en sante [19].

(iv) Australia: EQuIP Guide of the Australian Council onHealthcare Standards (3rd edn) [20].

(v) PAHO: La garantia de calidad [21], which we translatedto the best of our ability.

Please note that, in this paper, we refer to the manuals bycountry (except in the case of the PAHO manual) for thesake of simplicity. We recognize that more than one accredi-tation standard may be used in a country (for example, theJCAHO and NCQA standards in the USA).

Selection of the framework of analysis

We rely on a Parsonian description of action and its appli-cation to the concept of performance, as it applies to thespecific case of healthcare organizations [14]. This model(Fig. 1) was an attempt to synthesize the common coreelements addressed in the healthcare organization perform-ance literature. It is based on the idea of achieving an equili-brium among four dimensions of performance: (i) adaptation(A), the organization’s capacity to survive and grow in thechanging environment; (ii) goals (G), the results pursued interms of efficiency, effectiveness, the attainment of outcomesand stakeholder satisfaction; (iii) integration (I) or production,the care and services produced by the healthcare organizationin terms of volume of care and mechanisms and (iv) latency(L) or values and culture, the sense-making in the organiz-ation and its social environment. Values and culture refer toorganizational climate, resolution of staff conflicts, rewardingsystem and staff motivation. These four dimensions arerelated to each other through six inter-linked systems oralignments [14].

P. A. Smits et al.

48

This framework will be used to order the diverse stan-dards of accreditation and to provide the analytical ground-work for our comparison of accreditations and our analysisof the conceptualization of performance underlying the fiverecognized accreditations.

Analysis of the standards in the selectedaccreditations

Two persons independently read the selected manuals. Usingthe integrative framework, they both independently classifiedthe standards on the basis of their understanding of themanual’s content with respect to each dimension and align-ments of the Sicotte framework of analysis. After this classifi-cation step, the two persons discussed any differences inorder to arrive at one, mutually agreed upon classification ofthe standards. One person did a comparative analysis of themost versus least common dimensions and alignments used.This comparison was based on the number of standardscategorized in each dimension of the framework of analysis.It produced a taxonomy of accreditation manuals on thebasis of their conceptualization of performance.

Results

Representation of the dimensions of performanceand the alignment between dimensionsin the accreditations

Table 1 shows the importance of each dimension of per-formance as defined by the integrative model [14] for eachof the five accreditation manuals. The more the plus (þ)signs, the more fully represented a dimension in the accredi-tation. A plus/minus (þ/2 ) sign signifies a quasi-absenceof standards for that particular dimension, whereas a minus(2) sign signifies a complete absence of standards.Table 2 shows the presence (P) or absence (A) of

alignments.

Results for the USA. The US accreditation manual [18]emphasizes the dimension of integration/production(Table 1): ‘the hospital has a process to ensure that aperson’s qualifications are consistent with his/her jobresponsibilities’ (from the section titled ‘Management ofHuman Resources’; falls under the subdimension of qualityin Fig. 1). Less emphasis is given to the dimensions ofadaptation to the environment, goal attainment and values.Tactical and allocation alignments are both present

(Table 2) to highlight the importance of linking goals to theproduction of services and of relating adaptation to the pro-duction of services.

Results for PAHO. Using the same framework to categorizethe standards used, we found that the PAHO accreditation[21] omits goals and values. There is a considerable focus onthe production of services, especially the quality aspect and alesser focus on adaptation to the environment (Table 1).However, goals are taken into account through theiralignment with adaptation and with production (Table 2).

Results for France. The French accreditation [19] places littleemphasis on values, stressing instead the importance ofquality and, to a lesser extent, adaptation and goals (Table 1).There is some linking between adaptation and goals throughstrategic alignment, and between adaptation and productionthrough allocation alignment (Table 2).

Results for Canada. The Canadian accreditation [17] givessimilar importance to adaptation, production and values. Itplaces somewhat less importance on goals (Table 1). There issome alignment of values and adaptation, of values andgoals and of adaptation and goals (Table 2).

Figure 1 Representation of a conceptual framework for theanalysis of healthcare organization performance (adaptedfrom Sicotte et al. [14]).

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Table 1 The importance of the dimensions of performance in the five selected accreditation manuals

Integratedframework

USA PAHO France Canada Australia

A Present þþ þ/2 þ þþ þG Present þ – þ þ 2

I Present þþ þ þþ þ þþ þ þþ þþL Present þ – þ/2 þþ þ/2

Each dimension is either absent (2) or quasi-absent (þ/2 ) or more and more present (þ to þþþ ).

Conceptualizing performance in accreditation

49

Results for Australia. The Australian accreditation [20]makes reference to the dimensions of adaptation and quality,although making almost no mention of values andcompletely omitting goal attainment (Table 1). However,goals and values are not totally absent, as they are linked tothe other four dimensions through the presence of all sixalignments (Table 2).To summarize, all of the selected accreditations emphasize

the production of services, especially quality and coordi-nation of services and adaptation to the environment. Someomit values or goal attainment. The dimensions of perform-ance, when not mentioned directly, are included in relation toother dimensions through the alignments. For example, theattainment of goals is assessed by several means: a standardof accreditation can be specific to the dimension of goals, asit is the case in the manuals from France, the USA andCanada (i.e. ‘human resources achieving positive outcomes’[17]), or it can be linked to the alignment of goals with theother dimensions, as it is the case in the PAHO andAustralian manuals (i.e. ‘management of human resourcessupports the delivery of quality, safe care, and services/human resources planning supports the organization’scurrent and future ability to provide quality, safe care andservices’ [20]).

Taxonomy of accreditations

We propose a taxonomy of the standards of accreditationmanuals to compare them with respect to their conceptualiz-ation of performance. We chose to classify the variousmanuals using two axes reflecting the importance they give toboth the individual dimensions and to the alignments betweendimensions.The results given earlier show that quality is the only

dimension present in every accreditation manual (Table 1).Moreover, accreditations vary in how much emphasis theyplace on production, especially its quality and on the align-ments between dimensions. Therefore, we decided toorganize the taxonomy around these two axes: normative

quality-oriented axis and balanced alignment-oriented axis. InFig. 2, one axis represents a quality-oriented accreditation,ranging from normative to non-normative accreditation, andthe other represents an alignment-oriented axis ranging frommore (þ) to less (2) balanced accreditation.A normative accreditation is one that places a relatively high

emphasis on the dimension of production, especially on thesubdimension of quality, compared with the emphasis placedon the other three dimensions. The accreditation manuals ofPAHO, France and the USA are highly normative accredita-tions that focus on production. Those from Canada andAustralia, which are positioned at the bottom of Fig. 2, are lessnormative accreditations since they are less focused on quality.A balanced accreditation is one that places great emphasis

on alignments (Table 2). We hypothesize that accreditationmanuals composed of many alignments will achieve higherorganizational performance. On the basis of the horizontalposition along the axis representing degree of alignmentbetween dimensions, the most balanced alignment-orientedaccreditation manual is the one from Australia, less balancedare the manuals from the PAHO, France, Canada andthe USA.Ideally, accreditation should be balanced and take quality

into account. We would expect balanced accreditationmanuals on the right-hand side of the diagram to achievehigher performance than accreditation manuals positionedon the left-hand side. Whether accreditation manuals situatedin the top right-hand corner will lead to higher performancethan those situated in the bottom right-hand corner or viceversa remains to be seen. We hypothesize that the omissionof normative dimensions, as represented by the bottom halfof the diagram, is not favorable. However, accreditationmanuals positioned in the top half of the diagram and there-fore emphasizing normative procedures could also be associ-ated with negative outcomes. We recommend that more

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Table 2 The presence/absence of alignments in the fiveselected accreditation manuals

Integratedframework

USA PAHO France Canada Australia

SA P A P P P PTA P P P A A POA P A A A A PCA P A A A P PAA P P A P A PLA P A A A P P

P, presence of the alignment; A, absence of the alignment; SA,strategic alignment; TA, tactical alignment; OA, operationalalignment; CA, contextual alignment; LA, legitimization alignment;AA, allocative alignment.

Figure 2 Visual representation of a taxonomy ofaccreditation based on the conceptualization of performance.The taxonomy is based on two continuums: balancedaccreditation on the horizontal axis and normativeaccreditation on the vertical axis. An accreditation manualcould occupy a high (þ) or low (2) position on eachcontinuum.

P. A. Smits et al.

50

empirical studies on the actual performance of accreditationspositioned at both extremes—top and bottom halves of thediagram—be carried out in order to determine the idealposition for an accreditation in the taxonomy.

Discussion

To improve the performance of healthcare organizations, weneed to have a clear understanding of the definition ofperformance. Indeed, the definition of performance doesnot refer to just one, mutually agreed-upon dimension. Theintegrative framework we chose allows us to classify andanalyze accreditations according to a number of differentdimensions.We found that some dimensions were common across all

accreditations studied—e.g., the production of services.Some, e.g., values, are scarcely mentioned in any accredita-tions. The most common standard across all accreditationswas quality, reflecting a normative view of the performanceof healthcare organizations. The least common dimensionwas ‘values and culture’. The other dimensions of goalattainment and adaptation to the environment were empha-sized to varying degrees in the different manuals.The Parsonian-based perspective of performance presents

four dimensions that healthcare organizations have to fulfill inorder to achieve high performance levels. Therefore, we cantheoretically hypothesize that an accreditation built around allfour dimensions produces the highest level of performance. Inthis respect, on the basis of our framework of analysis, theCanadian manual scores above the others. The US manualoccupies an intermediate position, followed by the French andAustralian manuals. The PAHO manual is the least complete.An organization that performs well with respect to one

criterion may perform less well with respect to one or moreother criteria [22]. A conceptualization of performancefocused solely on one criterion or dimension could impairthe organization’s overall performance because of the pooremphasis placed on other important areas. We can thereforehypothesize that accreditations that include alignmentsbetween the dimensions will lead to better performance. TheAustralian accreditation manual appears to be the most com-plete, balanced manual since it contains all of the alignmentsof our integrative model. The Canadian manual appears tobe less complete, and the French, US and PAHO manualsare the least complete. On the basis of our analysis of theconceptualization of performance underlying accreditationmanuals, we were able to create a taxonomy of accreditationmanuals reflecting the underlying concepts of performancebased on two axes: normative quality-oriented accreditationand balanced alignment-oriented accreditation.The results of this research apply to the particular

manuals studied and are not a static description of a coun-try’s vision of healthcare organization performance. ThePAHO manual that we analyzed dates from 1992, whereasthe other manuals date from the past six years. Countriesmodify their manuals every 3–5 years. The various Hispaniccountries may have devised different protocols to assess

management and so may not be using the one presented inthe PAHO manual. Another limit of our analysis relates tothe weighting applied by accreditation agencies to particularimportant standards. We did not take such weighting intoconsideration in our analysis. Finally, the results may havebeen different had we chosen a different framework of analy-sis to classify accreditation standards.Our comparative analysis of the diversity of performance

conceptualizations parallels research into how the conceptionof management varies across countries [23]. Our analysis canbe a useful starting point for further studies on the relationshipbetween management style and performance in healthcareorganizations. One interesting avenue would be to comparethe management style of an organization with its performanceconceptualization model as identified by the typology.To date, no studies have empirically evaluated the extent to

which the dimensions and alignments considered in an accredi-tation manual improve organizational performance. Suchresearch would provide valuable information that could be usedto improve the performance of our healthcare organizations.Specific conceptualizations of performance underlying

accreditation manuals may not be neutral. More normativeaccreditation manuals may be associated with authoritative man-agement styles, and more balanced accreditation manuals withcomprehensive management styles. Our comparative analysis isa first step toward better understanding the relationship betweenthe conceptualization of performance and the management styleadopted in a particular healthcare organization.

Acknowledgements

We would like to thank Myriam Hivon and Susan Lemprierefor useful comments, and two anonymous reviewers fortheir constructive criticism and suggestions.

Funding

GETOS Chair at the Universite de Montreal supported thiswork financially.

References

1. Hayes J, Shaw C. Implementing accreditation systems. Int J QualHealth Care 1995;7:165–71.

2. Jovanovic B. Hospital accreditation as method for assessingquality in healthcare. Arch Oncol 2005;13:156–7.

3. Robert MD, James S, Jack G et al. A history of the jointcommission on accreditation of hospitals. J Am Med Assoc1987;258:21.

4. Pomey M, Contandriopoulos AP, Francois P et al. A tool fororganizational change in hospitals? Int J Health Care Qual Assur2004;7:113–24.

5. Shaw CD. Editorial: evaluating accreditation. Int J Qual HealthCare 2003;15:455–6.

Conceptualizing performance in accreditation

51

6. Segouin C. L’accreditation des etablissements de sante. De l’experienceinternationale a l’application francaise. Paris: Douin Editeurs, 1998.

7. Scrivens EA. A taxonomy of the dimensions of accreditationsystems. Soc Policy Admin 1996;30:114–24.

8. Neely A, Mills J, Platts K et al. Performance measurementsystem design: developing and testing a process-basedapproach. Int J Oper Prod Manage 2000;20:1119–45.

9. Tangen S. Performance measurement: from philosophy to prac-tice. Int J Prod Perform Manage 2004;53:726–37.

10. Cross KF, Lynch RL. For good measure. CMA Magazine, April1992;20–3.

11. Sink D, Tuttle T. Planning and Measurement in Your Organisation ofthe Future. Norcross, GA: Industrial Engineering andManagement Press, 1989.

12. Kaplan R, Norton DP. The balanced scorecard—measures thatdrive performance. Harv Bus Rev 1992;10:71–9.

13. Holmes JS, Amin Gutierrez de Pineres S, Kiel DL. Reformingpublic administration: applying the balanced scorecard approachto government agencies in developed and developing countries.Int J Public Admin 2006;29:1125–45.

14. Sicotte C, Champagne F, Contandriopoulos AP et al.Conceptual framework for the analysis of health care organiz-ations’ performance. Health Serv Manage Res 1998;11:24–48.

15. Veillard J, Champagne F, Klazinga N et al. A performanceassessment framework for hospitals: the WHO regional office

for Europe PATH project. Int J Qual Health Care2005;17:487–96.

16. WHO. Quality and Accreditation in Health Care Services: A GlobalReview. World Health Organization, 2003.

17. CCHSA. AIM: Achieving Improved Measurement: AccreditationProgram. Ottawa: Canadian Council on Health ServicesAccreditation, 2001.

18. JCAHO. Comprehensive Accreditation Manual for Hospitals: TheOfficial Handbook (CAMH). Joint Commission on Accreditationof Healthcare Organizations, 2006.

19. ANAES. Accreditation for Healthcare Organizations: SecondAccreditation Procedure, 2004. http://www.has-sante.fr/portail/types/FileDocument/doXiti.jsp?id=c_269762 (20 August 2007,date last accessed).

20. ACHS. The EQuIP Guide: A Framework to Improve Quality andSafety of Health Care. 3rd edn. Australia: The Australian Councilon Healthcare Standards, 2002.

21. PAHO. La garantia de calidad. Acreditacion de hospitales para americalatina y el caribe. Pan American Health Organization, 1992.

22. Cameron K. Effectiveness as paradox: consensus and conflictin conceptions of organizational effectiveness. Manage Sci1986;32:539–53.

23. Pindur W, Rogers SE, Kim PS. The history of management: aglobal perspective. J Manage Hist 1995;1:59–77.

Accepted for publication 17 October 2007

P. A. Smits et al.

52