34506359

8/13/2019 34506359

1/14

DISTINCTIVERESEARCH PATTERNS

ON PUBLIC SECTOR

PERFORMANCE

MEASUREMENT OF

PUBLIC

ADMINISTRATION

AND ACCOUNTING

DISCIPLINES

G. Jan van Helden, Age Johnsen and

Jarmo Vakkuri

G. Jan van Helden

Faculty of Economics and Business, PO Box 800,9700 AV Groningen, The NetherlandsTel: 31 503633673, Fax: 31 503638252E-mail: [email protected]

Age Johnsen

Faculty of Social Sciences, Oslo University College,P.O. Box 4 St Olavs plass, NO-0130 Oslo, NorwayE-mail: [email protected]

Jarmo Vakkuri

Department of Economics and Accounting,University of Tampere, FI-33014, FinlandE-mail: [email protected]

Abstract

This article explores distinctive research

patterns of public administration and ac-

counting disciplines concerning public sector

performance measurement (PSPM). Our re-

view shows that accounting researchers from

Europe investigate reasons for limited PM

use and factors explaining a rational or

symbolic PM use, inspired by organization

theory and institutional theory and conduct-ing case/field studies. Public administration

researchers from Europe and the USA prefer

to study PM design and PM impact respec-

tively, mainly using surveys in combination

with various theories, like political theory.

Public administration research from the USA

examines the types of performance indicators

in PM systems and contingent factors for PM

design. Public administration research from

Europe shows an interest in evaluating publicsector reforms like Best Value and explaining

learning processes for improvement. We

argue that PSPM research could benefit from

interdisciplinary efforts and intensified mutual

communication between public administra-

tion and accounting.

Key wordsAccounting, performance measurement, pub-

lic administration, research patterns

Vol. 10 Issue 5 2008 641651

8/13/2019 34506359

2/14

INTRODUCTION

Performance measurement has been a core issue in the public sector for many decades,but its importance has been reinforced by the more recent New Public Managementand Reinventing Government movements with their emphasis on planning and controlof outputs and outcomes.

Several disciplines (as well as professions) are relevant in studies of public sectorperformance measurement (PSPM), notably economics, public administration,accounting and sociology. We focus on two research disciplines, one rooted in publicadministration and the other in (public sector management) accounting. These twoacademic disciplines do not necessarily overlap, indicating an absence of mutualcommunication. This impression inspired us to conduct a literature review on PSPMresearch. It goes beyond the scope of this short article to investigate comprehensivelydifferences and similarities in PSPM research between public administration andaccounting. Therefore, our analysis focuses on distinctive research patterns of thesedisciplines in terms of content as well as use of theories and methods. Our literaturereview includes articles on PSPM that were published in the 20005 volumes of fourpublic administration journals and four accounting journals.

After a justification of the research design, the findings of our review are analysedand discussed.

RESEARCH DESIGN

Our review focuses on public sector performance measurement. In this article thepublic sector includes various tiers of government (i.e. local, central and intermediate),governmental agencies (such as the police force and schools) and also non-governmentalpublic sector organizations, mainly from the health care sector.

Performance measurement (PM) is understood as an instrument for indicatingefficiency and effectiveness and due to the public sector context also equity. PM isprimarily used for increasing decision-making rationality in organizations. However,ritualistic uses of PM information cannot be excluded. PM not only impliesmeasurement and analysis, but also reporting the resulting information to relevantadministrative and political bodies. PM serves various functions, particularlyaccountability, policy (re)design, as well as planning and control (Mayston 1985;Likierman 1993; de Bruijn 2002; Johnsen 2005). In this sense performancemeasurement largely overlaps with performance management. It is both measurement

of performance and management with measures of performance.We selected four public administration (PA) journals Journal of Public AdministrationR h d Th (JPART) P bli Ad i i t ti R i (PAR) P bli Ad i i t ti

642 Public Management Review

8/13/2019 34506359

3/14

(JMAR), Financial Accountability and Management (FAM) and Management AccountingResearch (MAR). Table 1 shows the origins of the journals (US or European) and the

disciplines (PA or AC).All four PA journals rank among the highest in their field, according to their impact

scores. AOS is one of the four international top AC journals (it also has a substantialimpact score). Given that the American top journals in AC Accounting Review,Journalof Accounting Research and Journal of Accounting and Economics are strongly financialaccounting oriented and publish hardly any articles about the public sector, we includedtwo other AC journals JMAR and MAR with very good reputations (Lowe andLocke 2005). Finally, FAM was selected as an AC journal because it is a well-respectedinternational niche journal on public sector management and accounting.

Our analysis is based on a set of PSPM articles, selected from the 20005 volumes ofthe journals. Our initial set of articles was seventy-nine, but the analysis focuses onsixty-eight articles.1 First, because PSPM research from US authors published in ACjournals is underrepresented, we had to disregard this group of authors. Second, alsoarticles coming from authors outside Europe or the USA mainly from Australia andNew Zealand were excluded. The analysis thus distinguishes three groups of authors:

(1) Accounting researchers from Europe.(2) Public administration researchers from Europe.(3) Public administration researchers from the USA.

Research methods employed in the articles were classified as: literature review;analytic; survey; documentary analysis; case/field study; reflection; and others. Thiscategorization was inspired by earlier literature reviews by Shields (1997) and vanHelden (2005). Although these categories refer to different research dimensions, suchas empirical research (documentary analysis, case/field studies and surveys) andtheoretical research (analytic and literature reviews), they can be derived from the

selected articles. Moreover, whenever two categories are apparent in one article forinstance a literature review and a survey this is accounted for (see countingprocedures later).

Based on prior reviews (Shields 1997; van Helden 2005) we identified the followingcategories for theories: economics; organization theory; neo-institutional sociology;political theory; PM theory; other theories; and no theory. PM theory can be regarded

Table 1: Journal selection according to origins and disciplines of journal

Public administration Accounting

van Helden et al.: Public sector performance measurement 643

8/13/2019 34506359

4/14

as a second level (more applied) theoretical stance, whereas the other theories aredistinctive disciplines.

We distinguished PM stages and PM topics. PM stages relate to the various stageswithin a PM life cycle; particularly design, implementation, use and impact. Similardistinctions are made by Likierman (1993) and Johnsen (2005), and in empirical studiesof PM some stages of this life cycle are recognizable (Cavalluzzo and Ittner 2004; vanDooren 2005). The pilot stage of our review revealed that this categorization of PMstages although sensible incompletely covered the research topics. We decided toinclude the PM topic as well as the PM stage. Accordingly, each PM stage involvesvarious research topics. For the research topics a pre-determined set of categorizationscould not be developed. Only a retrospective categorization was possible.

Our counting procedure in dealing with articles that use a combination of two ormore methods (or theories) was as follows. If two methods (theories) are addressed,each counted for one half, and if three methods (theories) are addressed, each countedfor one third and so on.

ANALYSIS

We discuss the nature of public sector performance measurement research by identifyingthe main themes addressed in our sample of articles. Although articles from both AC andPA researchers display an interest in all the PM stages that is, design, implementation,use and impact each category of authors has its own favourite PM stage: AC researchersfrom Europe are strongly interested in PM use, European PA researchers in PM impactsand PA researchers from the USA in PM design. Focusing on these distinctive researchinterests, we will review the current body of knowledge in PSPM research.2

US public administration researchers on PM design

Research on PM design is prominent among American PA researchers: 43 per cent ofthe articles address PM design, whereas the three other PM stages implementation,use and impact only receive attention from about 20 per cent of the articles each.These researchers address two main questions, that is, the types of performanceindicators (including the way they are specified) and contingent factors that influencePM design.

Edwards and Thomas (2005) describe the development and operation of a

performance measurement system, called the Atlanta Dashboard, in comparison withother such systems in US cities. Although the Dashboard loosely follows the ideas of theB l d S d it d lib t l h i t b f th i l


8/13/2019 34506359

5/14

public sector needs to be balanced among appropriate, competing values acknowledgingthe multiple levels of managerial accountability to citizens, elected officials, public

employees and professional standards. Kelly and Swindell (2002) examine correlationsfrom a sample of cities and municipalities in the USA with standardized measures andcitizen survey results for four service areas. Their study did not find a systematiccorrelation between internal (administrative) and external (citizen satisfaction)performance measures.

Some articles intend to find answers to rather pragmatic questions about performancemanagement. Rubenstein et al. (2003) advocate, for instance, the use of performancemeasures adjusted for the effect of non-controllable factors and the average value ofcontrollable factors (policy variables), instead of using raw performance measures. Folz(2004) recommends that best performing benchmarking partners have to be selectedfrom organizations having a certain target service level.

Another group of articles is concerned with survey research that aims to findrelationships between contingent variables and PM design. In their nation-wide survey ofUS counties Wang and Berman (2001) found, for instance, that central managementinvolvement significantly determines deployment of output measures, while missionorientation as well as external support from politicians and citizens significantlyinfluences deployment of outcome measures. Research by Julnes and Holzer (2001)reveals that PM design of organizations at various layers of US government is primarilyinfluenced by external requirements, resources, goal orientation, informationcapabilities and involvement of internal groups (see also Melkers and Willoughby 2005).

As indicated before, PA authors from the USA are conducting surveys for theircontingency studies, whereas studies on the design of Performance Indicator (PI)systems are mostly based on case studies. Some articles have a reflective nature. BothPM theory and political theory are well represented in PA papers by American authors.

European AC researchers on PM use

About 50 per cent of the articles written by European AC researchers concerns PM use.Three aspects are important to this research.

The first group of articles examines why politicians and public managers useperformance information only to a limited extent. Ter Bogt (2004) observes thataldermen of Dutch municipalities prefer informal and verbal communication to budgetsand reports with output-oriented performance information (see also ter Bogt and vanHelden 2000). This author also shows that politicians evaluate top managers based on a

broad set of criteria in which quantitative performance information only plays a minorrole (ter Bogt 2003).A d f ti l i ti t th t t t hi h PM i ti l


8/13/2019 34506359

6/14

of the organizational hierarchy, which makes a symbolic PM use likely. Only when one ofthe stakeholders is dominant in comparison with the others, a cascading down of

performance information serving the interests of the dominant stakeholder throughoutthe hierarchy will lead to a rational PM use. Some studies point at specific forms of PMuse also challenging rationality (compare Modell 2001). In their analysis of the UK BestValue project Bowerman et al. (2000) introduce the idea of defensive benchmarking,meaning that public sector organizations use benchmarking information to avoid critiquefrom external bodies rather than as an incentive for improvement. Llewellyn andNorthcott (2005) examine social, institutional and political processes through which UKhospitals move to the level of average performance.

A third group of authors investigates contingencies of PM use. Van Dooren (2005)conducts a survey on design and use of sections within the Ministry of the Flemish Commu-nity in Belgium. This study shows that PM use is positively influenced by the ability toidentify PIs, organizational size (e.g. larger organizations adopt more PIs than smallerorganizations), decision-making authority and goal orientation. Budding (2004) finds that,opposite to insights from other contingency studies, external uncertainty does not influ-ence PM use in Dutch municipalities, whereas factors like a results-oriented organizationalclimate, a supportive attitude of superiors and creation of game spirit have an influence.

European AC researchers most frequently apply case/field studies as methods of datacollection. Their research is mainly based on organization theories (such as strategicmanagement theory and management control theory) and neo-institutional theory.

European PA researchers on PM impact

Within this last group of articles 40 per cent address PM impact, which can beunderstood as the effect of PM on the accomplishment of goals, at the organizational,programme or sectoral level, but also including functional (e.g. learning) or

dysfunctional (e.g. manipulation) effects of PM. As elucidated below, some articlesaim to evaluate reform programmes of government, while others endeavour to enrichunderstanding of the impact of PM on organizational learning.

Martin and Hartley (2000) evaluate Best Value (BV) adoption in English and Welshlocal government at an early stage of development. They report a strong support forBV, although pilot authorities expressed a higher level of confidence in andunderstanding of BV than non-pilot authorities. Using theories on accountability andbased on a wide range of data sources, Boyneet al.(2002) review the BV project withinWelsh municipalities. Their assessment was more critical, and they conclude that in

general the BV pilot performance plans did not provide useful information to internaland external stakeholders. In a case study of partnerships within the NHS, Alcock(2004) hi t t t f t i i t t d li ki


8/13/2019 34506359

7/14

Hartley and Allison (2002) present an analysis of a network consisting of localauthorities in the UK that participated in a Better Value Development Programme.

Their study clarifies that participants valued the exchange and transfer of tacitknowledge, through sharing and comparing experiences. Sanderson (2002) reflects onthe impacts of NPM like reforms, in which performance measurement plays a mainrole. This article concludes that the evaluation capacity of organizations (includingcapacities for reflection, dialogue and research) should be improved.

PA researchers from Europe relatively frequently undertake surveys and case/fieldstudies. They often use applied theories, such as theories on change, accountability orbenchmarking.

Figure 1 links the most important PM stages for the groups of authors, asindicated above, to the corresponding PM topics and to the use of theories and researchmethods.

DISCUSSION

Our analysis demonstrates that PA and AC can be regarded to some extent as twodifferent academic tribes involving distinct research objectives and identities (Becherand Trowler 2001). This can account for a certain specialization in studying a commondomain, and contribute to knowledge accumulation on PSPM. According to ouranalysis a specific setting of specialization has developed in PSPM research: US PAresearchers concentrate on issues of PM design, European AC researchers on PM useand European PA researchers on PM impacts. The different foci may benefit PSPMresearch in some regard. However, a lack of mutual communication between thedisciplines their fundamental research orientations are fairly similar can also be ahindering factor. Researchers in one discipline, for instance, may suggest novel themesor research approaches, the novelty of which would have been questioned if those

researchers had been aware of relevant knowledge production in the other discipline.An indication for this lack of communication can be derived from our review. Wesearched for those authors who publish two or more articles, separately or as co-authorwith others. The outcome of this search reveals that authors mainly stick to theirdiscipline this holds for fifteen from eighteen authors which corroborates ourexpectation of non-overlapping academic circles. A lack of publications by the sameauthors in journals from different disciplines is, however, only a weak indication of thelack of communication between those disciplines. This issue requires further analysis ofcitations within and across the PA and AC disciplines.

There are some examples that clarify the risks of a lack of communication betweendisciplines that study the same phenomenon, PSPM. Cavalluzzo and Ittner (2004)bli h d t d i ti j l hi h d t it l t


8/13/2019 34506359

8/14knowledge about case-based accounting research using institutional theories (ter Bogt2003 M d ll 2004)

Figure 1: Distinctive research patterns of groups of authors


8/13/2019 34506359

9/14

(for example: ter Bogt and van Helden 2000; Bowermanet al.2001; Modell 2001; Boyneet al.2002; Sanderson 2002), whereas much research in the North-American tradition is

more normative (Rubensteinet al.2003; Folz 2004). Such research is, generally speaking,more supportive of public management reforms. This would indicate that as regards toattitudes of PSPM researchers on public management reforms the tribal division would alsobe associated with continents, not only with the disciplines of PA and AC.

There is also little interest in PSPM research within the AC discipline in NorthAmerica. The major interest in PSPM comes from PA. This underpins a point ofconcern that American AC researchers seem to ignore the public sector.

The results of our review may be bounded by the journal selection and by ouromission of analysis of books. It has been difficult to find good international AC journalsthat publish public sector research. Our review also includes a very small number ofAmerican AC academics.

We envisage several directions for future research. It would be of importance toconduct a more in-depth study of the research themes on PSPM. This assessment couldsubsequently relate to similarities and differences between disciplines and differentgeographical areas, for instance of the European or North-American perspectives.Despite these new directions for research, our review provides an overall picture of themost distinctive types of research on PSPM conducted by both AC and PA researchersfrom various continents.

ACKNOWLEDGEMENT

The authors acknowledge the constructive comments received when earlier versions ofthis article were presented at the Second Transatlantic Dialogue: A Performing PublicSector, Leuven, 13 June 2006, and at the Fourth EIASM International Conference onAccounting, Auditing and Management in Public Sector Reforms, Siena, 79

September 2006. The authors are indebted to Bob Scapens and two anonymousreviewers for their helpful comments on earlier versions of this article.

NOTES1 The list of articles in the review is available on request from the corresponding author.

2 Classification results of all articles in the review are available on request from the corresponding author.

REFERENCES

Alcock, P. (2004) Targets, Indicators and Milestones, Public Management Review, 6: 2 pp2117.

Becher, T. and Trowler, P. A. (2001) Academic Tribes and Territories: Intellectual Enquiry and the Cultures of Disciplines


8/13/2019 34506359

10/14

Boyne, G., Gould-Williams, J., Law, J. and Walker, R. (2002) Plans, Performance Information and

Accountability: The Case of Best Value, Public Administration. 80: 4 pp691710.

Brignall, S. and Modell, S. (2000) An Institutional Perspective on Performance Measurement and Management inthe New Public Sector, Management Accounting Research. 11: 3 pp281306.

Budding, T. (2004) Accountability, Environmental Uncertainty and Government Performance: Evidence from

Dutch Municipalities, Management Accounting Research. 15: 3 pp285304.

Cavalluzzo, K. S. and Ittner, C. D. (2004) Implementing Performance Measurement Innovations: Evidence from

Government, Accounting Organizations and Society. 29: 3/4 pp24367.

De Bruijn, H. (2002) Managing Performance in the Public Sector, London: Routledge.

Edwards, D. and Clayton Thomas, J. (2005) Developing a Municipal Performance-Measurement System:

Reflections on the Atlanta Dashboard, Public Administration Review. 65: 3 pp36976.

Folz, D. F. (2004) Service Quality and Benchmarking the Performance of Municipal Services, Public Administration

Review. 64: 2 pp20920.Hartley, J. and Allison, M. (2002) Good, Better, Best? Inter-Organizational Learning in a Network of Local

Authorities, Public Management Review. 4: 1 pp10118.

Johnsen, A. (2005) What Does 25 Years of Experience Tell Us about the State of Performance Measurement in

Public Policy and Management?, Public Money and Management. 25: 1 pp1522.

Julnes, P. deLancer, and Holzer, M. (2001) Promoting the Utilization of Performance Measures in Public

Organizations: An Empirical Study of Factors Affecting Adoption and Implementation, Public

Administration Review. 61: 6 pp693708.

Kelly, J. M. (2005) The Dilemma of the Unsatisfied Customer in a Market Model of PA, Public Administration

Review. 65: 1 pp7684.

Kelly, J. M. and Swindell, D. (2002) A Multiple-Indicator Approach to Municipal Service Evaluation CorrelatingPerformance Measurement and Citizen Satisfaction across Jurisdictions, Public Administration Review. 62: 5

pp61021.

Likierman, A. (1993) Performance Indicators: 20 Early Lessons from Managerial Use, Public Money and

Management. 13: 4 pp1522.

Llewellyn, S. and Northcott, D. (2005) The Average Hospital, Accounting, Organizations and Society. 30: 6 pp55583.

Lowe, A. and Locke, J. (2005) Perceptions of Journal Quality and Research Paradigm: Results of a Web-Based

Survey of British Accounting Academics, Accounting, Organizations and Society. 30: 1 pp8198.

Martin, S. and Hartley, J. (2000) Best Value for All: An Empirical Analysis of Local Governments Capacity to

Implement Best Value Principles, Public Management Review. 2: 1 pp4356.

Mayston, D. J. (1985) Non-Profit Performance Indicators in the Public Sector, Financial Accountability andManagement. 1: 1 pp5174.

Melkers, J. and Willoughby, K. (2005) Models of Performance-Measurement Use in Local Governments:

Understanding Budgeting Communication and Lasting Effects, Public Administration Review. 65: 2 pp180

90.

Modell, S. (2001) Performance Measurement and Institutional Processes: A Study of Managerial Responses to

Public Sector Reform, Management Accounting Research. 12: 4 pp43764.

(2004) Performance Measurement Myths in the Public Sector: A Research Note, Financial Accountability and

Management. 20: 1 pp3955.

Rubenstein, R., Schwartz, A. E. and Stiefel, L. (2003) Better than Raw: A Guide to Measuring Organizational

Performance with Adjusted Performance Measures, Public Administration Review. 63: 5 pp60715.Sanderson, I. (2002) Performance Management, Evaluation and Learning in Modern Local Government, Public

Administration. 79: 2 pp297313.


8/13/2019 34506359

11/14

Ter Bogt, H. J. (2003) Performance Evaluation Styles in Governmental Organizations: How Do Professional

Managers Facilitate Politicians Work?, Management Accounting Research. 14: 4 pp31132.

(2004) Politicians in Search of Performance Information? Survey Research on Dutch Aldermens Use ofPerformance Information, Financial Accountability and Management. 20: 3 pp22152.

Ter Bogt, H. J. and van Helden, G. J. (2000) Accounting Change in Dutch Government: Exploring the Gaps

Between Expectations and Realizations, Management Accounting Research. 11: 2 pp26379.

Van Dooren, W. (2005) What Makes Organisations Measure, Financial Accountability and Management. 21: 3

pp36383.

Van Helden, G. J. (2005) Researching Public Sector Transformation: The Role of Management Accounting,

Financial Accountability and Management. 21: 1 pp99133.

Wang, X. and Berman, E. (2001) Hypotheses about Performance Measurement in Counties: Findings from a

Survey, Journal of Public Administration Research and Theory. 11: 3 pp40328.


8/13/2019 34506359

12/14

8/13/2019 34506359

13/14

8/13/2019 34506359

14/14

Documents

34506359