1

The Use of Performance Information in Cutback Budgeting: The Case of Estonia

Ringa Raudla (ringa.raudla@ttu.ee), Senior Research Fellow, Ragnar Nurkse School of Innovation and Governance, Tallinn University of Technology, Estonia Riin Savi (riin.savi@ttu.ee), Junior Researcher, Ragnar Nurkse School of Innovation and Governance, Tallinn University of Technology, Estonia Ermo Liedemann (Ermo.Liedemann@riigikontroll.ee), Audit Manager, Performance Audit Department, National Audit Office of Estonia

Paper prepared for the 2013 ECPR General Conference in Bordeaux, 4-7 September 2013 Section: The Changing Face of Executive Politics: Crisis and Austerity

Panel: Budgetary Responses to the Fiscal Crisis: Changes in Budgeting Practices and Institutions

2

Abstract: Based on the existing literature on cutback budgeting and performance budgeting it is possible to make diverging predictions about the role of performance information in budgetary decision-making during fiscal stress and fiscal crisis. This paper contributes to theoretical discussions by outlining what kind of (rivalling) predictions different theoretical approaches would make about the use of PI in budgetary decision-making in times of fiscal stress and fiscal crisis. This paper also contributes to the empirical literature by examining the role of performance information in the central government budgetary decision-making in Estonia during the recent economic and fiscal crisis (in 2008-2009) and its aftermath (during a continuing atmosphere of fiscal stress in 2010-2012). As the Estonian case demonstrates, in a situation of fiscal crisis, when the decision-makers are intent on adopting sizable fiscal consolidation packages, the incrementalist prediction – that performance would not be used in budgetary decision-making – is corroborated. In the aftermath of a period of fiscal crisis, however, when fiscal stress continues (but is not so extreme anymore), the Estonian case shows that the understanding provided by interactive-dialogue theory of performance information use is the most accurate: performance information is sometimes used in budgetary negotiations, broadly speaking, but different actors perceive its role and impact differently. 1. Introduction If history is any indication, it can be expected that the current era of austerity in European countries (and elsewhere) is likely to bring about an increased focus on performance-based budgeting. The perceived need to cut back government expenditures provides a fertile ground for calls to “eliminate waste”, to “increase efficiency” − and to use performance management and/or performance budgeting to achieve that (see e.g., Marcel 2013; Stiefel et al. 1999; Willoughby 2004). Indeed in normative discussions on expenditure cutbacks (both in the past and also currently), various consultants, think-tanks, professional associations and some academics have argued that performance information (PI) should be used for making “rational” and “intelligent” decisions about where the cuts should fall (for examples, see, Marcel 2013; MacManus 1984; McTighe 1979; Murray and Efendioglu 2009; Olden et al. 2012; Plant and White 1982). When studying the links between fiscal stress and performance budgeting in a positive rather than normative fashion there are (at least) three possible approaches. First, one could examine whether fiscal (di)stress has brought about public sector reform initiatives that (re)-emphasize performance budgeting. Indeed, many studies make (although often just in passing) reference to the fact that performance budgeting reforms in different countries were triggered or inspired by a period of (fiscal) crisis (e.g. Dunsire and Hood 1989; Glassberg 1978; Levine et al. 1981; Robinson and Brumby 2005; Schick 1990; Straussman 1979).1 Second, one could examine 1 As Pollitt and Bouckaert (2011, pp. 26-27) note though, the link between fiscal crises and (performance) budgeting reforms is not a linear one. While in some instances such reforms have clearly originated in a major fiscal crisis (like the New Zealand reforms of 1980s), in other cases crises have not brought about a fundamental change in the budget system. As Pollitt and Bouckaert (2011, p. 27) explain, on the one hand, austerity would render undertaking a major reform more difficult, because it cannot “be lubricated with new money”; on the other hand, however, a sense of crisis can make it easier to consider more radical reform options, especially if these reforms can be presented to help to “solve” the crisis (see also Pollitt 2010).

3

whether the existence of performance budgeting system influences the level of fiscal discipline and the likelihood of a fiscal crisis emerging in the first place (see, e.g. Robinson and Brumby 2005). Third, one could investigate whether and how the existing structures and systems of performance budgeting in place are utilized during the times of (fiscal) crisis for designing austerity measures and undertaking cutbacks. This paper focuses on the last question. This paper seeks to contribute to both theoretical and empirical gaps in the existing literature. First, the paper contributes to theoretical discussions by outlining what kind of (rivalling) predictions different theoretical approaches would make about the use of PI in budgetary decision-making in times of fiscal stress and fiscal crisis. While the theoretical approaches covered in this paper (organizational learning theory, agency theory, incrementalism, interactive-dialogue theory) discuss the role of PI in budgetary decision-making in general, one cannot find, in the existing literature, a systematic (and comparative) delineation of what these theoretical perspectives would anticipate in the context of fiscal crisis and fiscal stress. Second, when looking at the existing empirical literature, only a handful of studies concentrate on the question of whether and how PI is taken into account in designing austerity measures and dealing with fiscal stress. This paper contributes to the empirical literature by examining the role of PI in the central government budgetary decision-making in Estonia during the recent economic and fiscal crisis and its aftermath. Specifically, the paper focuses on the following research questions: 1) What role did PI play in the cutback decisions and the design of austerity packages in the executive branch of the Estonian central government during the fiscal crisis in 2008-2009?; 2) What role did PI play in the central government budgetary decision-making in the immediate aftermath of the crisis (during a continuing atmosphere of fiscal stress) in 2010-2012?2 Methodologically, this paper utilizes a mixed-method approach by combining quantitative and qualitative analysis. For the quantitative analysis, the responses of the Estonian senior public officials to the “Executive Survey on Public Sector Reform in Europe – Views and Experiences from Senior Executives” (undertaken under the FP7 COCOPS project) are used. For the qualitative analysis, case studies were undertaken in nine Estonian ministries (out of eleven). The sources of data for the case studies included documents and 17 semi-structured interviews with public officials. The qualitative study also draws on an audit, carried out by the National Audit Office (2012) in Estonia, which examined, inter alia, the question of whether and how PI is used in budgetary negotiations and decision-making by the Ministry of Finance and the line ministries. The paper is organized as follows. Section 2 outlines the theoretical discussion on the role of PI in budgetary decision-making during fiscal crisis and fiscal stress. Section 3 gives an overview of the existing empirical studies that have examined this question. Section 4 presents the empirical findings about Estonia. Section 5 concludes.

2 A number of studies have looked at the recent fiscal consolidation in Estonia, exploring why and how the austerity measures were undertaken (see, e.g. Raudla 2013; Raudla and Kattel 2011, 2013; Jõgiste et al. 2012) but none of these studies is explicitly focused on the role of PI in budgetary decision-making process. While Jõgiste et al. (2012) do briefly touch upon the issue of the use of PI in devising austerity measures their article is confined to discussing whether impact assessments were undertaken by the Ministry of Finance officials when proposing the cuts for various areas.

4

2. Theoretical Discussion In the existing literature on performance budgeting, one can find various theoretical approaches that have been used to conceptualize the role of performance information3 in budgetary decision-making. Organizational learning theory, agency theory, incrementalism, and interactive-dialogue theory are probably the most prominent ones. Although all these approaches have discussed the role of PI in budgetary decision-making in general, one cannot find, in the existing literature, a systematic delineation of what these theoretical perspectives would anticipate in the context of fiscal stress and fiscal crisis.4 In the following, the predictions of these different theoretical perspectives about whether and how PI would be used during fiscal stress and fiscal crisis will be outlined − with the help of insights from the studies on cutback management and cutback budgeting.5 Organizational Learning Theory From the perspective of organizational learning theory (see, e.g. Askim 2008) it can be argued that PI would become especially important during fiscal stress and during fiscal crisis given that PI can provide valuable analytical input to the decision-making process and help policy-makers impose expenditure constraints or undertake cuts on a more “rational” basis (Jimenez 2013a; Straussman 1979). In particular, PI would allow the decision-makers to opt for selective and targeted cuts − instead of across-the-board cuts − with larger hits taken by programmes that are not performing well (Jimenez 2013a; Straussman 1979). Also, it would also expect organizations to use PI to identify priorities and use such “rankings” for imposing constraints or implementing cuts in those sectors that are shown to be of lower importance (Jimenez 2013a; Plant and White 1982; Robinson and Brumby 2005). Agency Theory According to agency theory, which emphasizes informational asymmetries between principals and agents and the role of PI in alleviating these asymmetries, it could be expected that if the need to constrain expenditure growth or to achieve budget cuts brings about relaxed input controls and/or the delegation of more detailed decision-making on the budget cuts to lower level officials (i.e., the “agents”), the cabinet and/or central budget office (i.e., the “principals”) may feel it necessary to make the line ministries more accountable for the results, as a quid pro quo for increased flexibility (Brumby 1999; Cothran 1993). This has been the general logic of

3 Under “performance information” this paper means “systematic information describing the outputs and outcomes of public programmes and organizations … generated by systems and processes intended to produce such information” (Pollitt 2006, p. 39). 4 In this paper, the distinction between “fiscal stress” and “fiscal crisis” is based on how policy-makers subjectively perceive and construct the fiscal situation of the country. Such a focus on subjective perceptions is chosen because the interpretations of the situation strongly influence the courses of action chosen. “Fiscal stress” refers to a situation of “chronic scarcity” of budgetary resources (Schick 1980), in which the available revenues are not sufficient to cover the growth of expenditures, as a result of which budgetary negotiations are centered around constraining expenditure growth. “Fiscal crisis” refers to a situation of “acute” or “total scarcity” (Schick 1980), in which revenues are falling and the resulting gap between expenditures and revenues is perceived to be so large as to constitute a “crisis” by the political decision-makers, necessitating an adoption of austerity measures. 5 For an overview of the literature on cutback management, see Raudla et al. (2013).

5

performance-based budgeting reforms: in return for relaxed input controls (i.e., less detailed control of the specific line items and more extensive use of lump sums and global budgets), managers are made accountable for outcomes and outputs, with the help of performance measurement and reporting systems (Brumby and Cangiano 2001; Cothran 1993; Schick 1988, 1990). It can be conjectured that this “logic” would become re-emphasized during fiscal consolidation episodes when the need for more extensive managerial decision-making in using and (re)-allocating resources is needed (Raudla et al. 2013). Incrementalism The theory of incrementalism6 would predict that PI is not used during fiscal crisis or fiscal stress. It can be conjectured that the factors which make performance-based budgeting difficult in “normal” times − and on which incrementalism has elaborated upon extensively − would be even more pronounced during times of austerity. First, the incrementalist approach emphasizes that the challenge in linking resource allocation decisions to PI is that budgeting is inherently political (Caiden 1998; Joyce 1996, 2008; Kong 2005; Rubin 1993; Schick 1990; Wildavsky 1966, 1969, 1978; Wildavsky and Caiden 1988;). As Joyce (1993, p. 16) puts it, in a democratic system, “no set of budget techniques can substitute for political decisions about ‘who wins’ and ‘who loses’”. The same is likely to apply in a context of fiscal stress and fiscal crisis. Second, it can be conjectured that according to incrementalist perspective, the problems arising from what Pollitt (2001) calls “comfort of ambiguity” are likely to be even more prevalent during fiscal strain than in times of stable revenue growth. An important challenge in linking budgetary appropriations to PI is that measuring performance entails clarifying and agreeing on goals for the government as a whole and also for the specific government organizations (Joyce 1997). Pollitt (2001, pp. 13-14) has argued that because budget-making entails “complex and sensitive distribution deals” and is by nature a “delicate and frequently adversarial process”, decision-makers have to appeal to rather “vague and general values” − when discussing budgetary allocations − rather than “careful comparative evaluations of rival programmes or the specification of precise operational priorities and targets”. Especially during times of fiscal crisis, pursuing extensive debates on the specific goals and targets of government organizations would be too polarizing (Schick 1988) and counteract the general aspiration to reduce the level of conflict in governmental decision-making in an already tense environment. As Schick (1988, p. 529) has noted, “evaluations stir up conflict at a time that government officials desperately need support for the tough choices they face”. In such an environment, across-the-board cuts may be viewed as more feasible given their perceived fairness and lower level of political conflict such cuts entail (Behn 1980; Levine 1978, 1979). Third, incrementalist approaches have argued that because of cognitive limitations, decision-makers would not be able to undertake comprehensive consideration of PI during the budget process Joyce 2008; Moynihan 2006; Van Dooren 2011),. This argument would be especially applicable during fiscal crisis: when a sense of urgency prevails, PI may give rise to information overload and could be perceived as a distraction for decision-makers (Joyce 2008; Moynihan 6 For an overview of the theory of incrementalism in budgeting, see Swain and Hartley (2001).

6

2006; Van Dooren 2011), as a result of which PI would not be inserted into budgetary negotiations by the budget actors. Finally, the incrementalist perspective would remind us that budgetary decision-makers face legal constraints: even if they wanted to change the foreseen budgetary allocations “according” to PI, they would be prevented from doing so because the appropriated amounts are mandated by existing laws (Pitsvada & LoStracco 2002). Hence, cutbacks are likely to focus on those expenditures that are “cuttable” or “controllable” (Downs and Rocke 1984; Schick 1983). In sum, the incrementalist perspective would predict that instead of following the “rationalistic” approach (including the utilization of PI), budgetary decision-making during fiscal crisis (and also fiscal stress) is likely to be predominantly decremental (i.e., characterized by across-the-board cuts or constraints) with (perhaps) some prioritization of certain sectors based on broad political preferences (rather than careful consideration of PI). Interactive-dialogue Theory Between these (relatively clear-cut) theoretical standpoints, pointing to either definite “use” or “non-use” of PI during fiscal crisis and fiscal stress is the interactive-dialogue theory of performance information use (see, e.g. Moynihan 2006, 2008a,b). According to this theory, the nature of PI is inevitably ambiguous: PI in itself does not tell us why an organization or program performed as it did, or which external factors influenced the outputs, outcomes and results. Thus, the meaning of PI is constructed and its utilization depends on the context, being influenced by what the motivations of the users of PI are and whether PI is useful for their goals. (Moyhnihan 2006, 2008a,b) Thus, when predicting the extent of utilization of PI in cutback budgeting, it is necessary to ask whether different budget actors would be motivated to use PI in budget dialogues during fiscal strain and how useful PI would be for making cutback decisions. With regard to the motivations of budget actors to use PI in the environment of fiscal stress and fiscal crisis and the usefulness of PI for their purposes, on the one hand, indeed, budget actors might want to use PI in order to either explain the necessity of the expenditure constraints or cuts (if they are “guardians of the purse” in the Ministry of Finance) or to protect their budgets from these steps (if they are “advocates” from the line ministries). There are several issues or challenges, however, that may render decision-makers less willing to use PI when discussing and deciding on the austerity packages and budgetary cutbacks (compared with a relatively stable fiscal environment). These issues are likely to constitute at least some obstacles during fiscal stress but are likely to be even more severe barriers to using PI during fiscal crisis. First, in budgetary negotiations between the central budget office (like the MoF) and the line ministries (especially during fiscal crisis but to some extent also during fiscal stress) the “constraint versus empowerment” problem of integrating financial management and performance management systems (Pollitt 2001, p. 15) may become a more serious barrier to PI use than is the case in times of “fiscal normalcy” (i.e., continued stable revenue growth). Given that PI could give the ministries and agencies “ammunition” for advocating their programmes and explaining why budget constraints or cuts should not fall on them, the officials of the central budget office or Ministry of Finance (MoF) may be reluctant to raise discussions over PI during budgetary negotiations concerning the budget cuts and choose to focus the discussion on “cutting

7

the dollars” rather than on “cutting the programs” (see Schick 1983). Even if line ministries tried to use PI to protect themselves from budget cuts, MoF officials would be more motivated to focus the debates on budgetary aggregates and the size of cutbacks, rather than on programs (and their performance) when devising austerity measures. Second, given that budgeting is a highly time-constrained process (see, e.g., Halachmi 2005; Hatry 2002; Hou et al. 2011; Kong 2005; Pollitt 2001), there may be limited opportunities for developing dialogue routines that would focus on the discussion of PI and its implications for resource allocation. While this may pose obstacles to using PI already in times of normalcy and mild fiscal stress, it can be expected that in the context of fiscal crisis, the time-pressures become even more significant because policy-makers feel that decisions need to be taken quickly. Third, as pointed out by many studies on performance-based budgeting, decision-makers face conceptual problems when trying to link PI to allocation decisions. Establishing links between resource allocation and achieved (or predicted) performance is complicated, inter alia, by attribution problems, rival causes, and time lags (e.g., Caiden 1998; Kong 2005; Pollitt 2001; Stiefel et al. 1999).7 During fiscal crisis, these conceptual dilemmas become even more aggravated. For example, the performance indicators of some programs may fall because of the fiscal environment; hence, these indicators may send “wrong” signals about how the programs are “performing”: they may be a sign of increased demand rather than of poor performance. Finally, given that gathering and interpreting PI requires analytical, financial, human and technological resources (see, e.g. Caiden 1998; Curristine 2005a; De Lancer Julnes and Holzer 2001; Frank and D’Souza 2004; Halachmi 2005; Jordan and Hackbart 1999; Van Dooren 2005), which may become more scarce during times of fiscal stress (and even scarcer during fiscal crisis), the capacity of the officials involved in budgetary decision-making to produce and to analyse PI may become strained (see, e.g. Behn 1980; Caiden 1998; Jimenez 2013b; Levine 1978; Massey and Straussman 1981). In sum, the interactive-dialogue theory would conjecture that the use of PI in budgetary decision-making in cutback budgeting is likely to be more limited than predicted by the organizational learning or agency theories but it would not exclude the possibility of PI playing at least some (significant) role in the budget process, as incrementalism would. Based on the arguments outlined above, interactive dialogue theory would argue that the use of PI in cutback budgeting would depend on the severity of the fiscal strain: the more severe fiscal stress becomes, the less likely it is that PI would be used in budgetary decision-making. 3. Existing Empirical Studies While there are several studies examining the link between fiscal stress and the initiatives of performance budgeting reforms (see, e.g. Dunsire and Hood 1989; Glassberg 1978; Levine et al. 1981; Pollitt and Bouckaert 2011; Robinson and Brumby 2005; Schick 1990; Straussman 1979)

7 Probably the most severe conceptual dilemma in trying to link budgetary appropriations to observed performance is whether to increase or decrease funding to a poorly performing organization or program: poor performance indicators may signal that additional resources are needed for improved performance (e.g. Caiden 1998; Curristine 2005a,b; Joyce 1997; Moynihan 2008a,b).

8

and a significant (and increasing) number of studies investigating the role of PI in budgetary decision-making in general (see, e.g. for recent overviews see, for example, Ho 2011; Melkers and Willoughby 2005; Zaltsman 2009),8 only a handful of empirical studies focus explicitly on the question of how PI is used during a period of fiscal stress and fiscal crisis. Among the existing studies, only four have examined national governments. Most of them find that the use of PI in a context of fiscal stress or crisis is either non-existent or very limited. Schick (1988), in his study of OECD countries during the 1980s, concludes that the governments’ capacity to reallocate resources, from lower priority programmes to higher-priority ones (during preparation of the budget) was weakened by fiscal stress. As he notes, an important reason for that in several countries were the inflexibilities brought about by the mandated nature of many expenditures (as a result of which cuts were made to discretionary programmes, irrespective of their performance). All in all, Schick (1988, p. 528) reckons that “cutback budgeting has not generally been based on assessment of existing programs” because “in an environment dominated by the quest for savings, governments are tempted to take the cutbacks that are readily within reach, politically feasible and safe, and easy to implement.” In a study of developments in budgetary decision-making in the Canadian federal government, Good (2011) observes that when the government faced the need to undertake fiscal consolidation in mid-1990s, there was a shift to utilizing PI (in the form of program reviews) for undertaking cuts (after the limits of across-the-board cuts had been achieved). In early 2000s, the cuts imposed, however, again followed an incremental logic (ibid). Troupin et al. (2013) examine cutback budgeting in the Belgian Federal government (in 2009-2012) and conclude that the budget cuts have, for the most part, followed the cheese-slicing tactics rather than being based on PI – this has been the result of the political nature of budgetary decision-making and insufficient quality of PI. The empirical studies undertaken on budgeting in the US states provide further evidence of the limited or non-existent role of PI during fiscal crisis or fiscal stress. Lauth’s (1985) study of the role of PI in the budgetary process in Georgia finds no evidence that PI played any important role in budget decision-making during a period of fiscal stress. Willoughby’s (2004) survey of 212 public officials indicates that PI was not viewed effective for cost or programme cutting or for changing in early 2000s when the U.S. states had to undertake efforts to balance their budgets.9 Hou et al. (2011) argue, based on their longitudinal study of 11 US states, that performance-based budgeting is used more extensively during periods of economic boom compared with downturns. They show that while in some states the recent crisis has increased the salience of PI, in most, PBB has not turned out to be a useful budget tool during the tight fiscal climate.

8 For most part, these studies find that although PI may play a role in influencing the nature of budgetary discussions and facilitate communication between the different actors involved in the budget process, performance information does not usually play a significant role in directly influencing the appropriation levels (e.g. Carlin and Guthrey 2003; Connelly and Tompkins 1989; Curristine 2005a,b; Joyce 1997; Kelly and Wanna 2004; Melkers and Willoughby 2001, 2005; Raudla 2012; White 2012; Willoughby and Melkers 2000). There are some studies, however, that provide evidence of more extensive use and impact of PI in budgeting (e.g. Gilmour and Lewis 2006; Jordan and Hackbart 1999; Zaltsman 2009). 9 She also finds that the level of perceived effectiveness of PI in influencing cost savings or changing appropriation levels were lower among budget officials than among agency staff.

9

The studies undertaken on the local government level in the US provide somewhat diverging evidence. In a study of departments at the city of Indianapolis, Ho (2011) finds that in the context of fiscal stress, PI was used by the managers in budgetary decision-making at the program level in order to reallocate resources and “to do more with less”. In a survey of 1400 US municipalities between 2009-2011 Jimenez (2013a) observes that the cities with performance management systems made more extensive use of targeted cuts than those without such systems.10 At the same time, the email interviews he conducted with the representatives of about 90 cities (reporting extensive use of performance management) indicate that during fiscal crisis, budgetary decisions appear to be “divorced from performance considerations”, primarily because of the political nature of budgeting. 4. The Case of Estonia 4.1. Research Strategy Ascertaining an impact of PI on budgetary allocations can be quite challenging and focusing only on those instances where PI has directly influenced specific appropriations in the budget bill would be too narrow (see, e.g. Zaltsman 2009). Thus, this study takes a relatively broad view of the “impact” and “role” of PI in the budgetary process and also considers the different phases and points where PI could enter into budgetary decision-making in the executive branch: the discussions and formulation of the budget requests within the line ministries, budgetary negotiations between the line ministries and the MoF officials, and budget deliberations in the cabinet. This paper utilizes a mixed-method approach by combining quantitative and qualitative analysis. For the quantitative analysis, the responses of the Estonian senior public officials to those parts of the “Executive Survey on Public Sector Reform in Europe – Views and Experiences from Senior Executives” (undertaken under the FP7 COCOPS project)11 that pertain to the use of PI (both in general and during the crisis) will be used. For the qualitative analysis, case studies were undertaken in seven ministries (out of eleven). The sources of data for the case studies included budget documents and 17 semi-structured interviews carried out with public officials involved in the budget process (5 officials in the MoF and 12 in the line ministries)12 between spring 2010 and summer 2013. In the interviews, the public officials were asked about the use of PI in budgetary decision-making during the acute crisis in 2009 and also in the aftermath, i.e., in the environment of chronic (but not anymore extreme) scarcity.

10 In a similar study, Jimenez (2013b) finds that although the effects of the existence of (comprehensive) performance management system on the likelihood of targeted cuts was not statistically significant as whole, the more severe the fiscal stress was, the more likely it was that performance management practices in a city led to targeted budget cuts. 11 More detailed information on the survey can be found in Hammerschmid et al. (2013). 12 In the line ministries, mostly officials working in the finance department (and in some cases also strategic development department) were interviewed, given that they are the officials most closely involved in preparing the budget in any given ministry.

10

In addition to the survey and the interviews conducted by academic researchers this paper also draws on an audit carried out by the Estonian National Audit Office (NAO), which examined, inter alia, whether and how PI is used in budgetary negotiations and decision-making in the MoF and the line ministries.13 The audit entailed the analysis of all relevant documents (including the performance plans and reports of the ministries, budget requests, analyses of the requests, memos, minutes of budget meetings, negotiation agendas etc). In addition to the analysis of documents, altogether 35 interviews were conducted with officials involved in the budget process. Such a mixed-method approach provides a useful opportunity to examine whether and to what extent the general attitudes about performance management and the role of PI among public officials are reflected in actual budgetary decision-making. It has been noted that an important challenge in doing survey research on the role of PI in decision-making is that the public officials may have different conceptions of what “performance information” means, which limits the comparability of the answers given (Askim 2007). As noted by a number of studies, the answers from survey research may overstate the use of PI in budgetary decision-making (Burke and Costello 2005; Frank and d’Souza 2004). Complementing survey research with an in-depth case study approach can help to address this shortcoming. 4.2. Background Information on Estonia 4.2.1. Developments in the Fiscal Environment in 2008-2012 The global financial crisis hit Estonia hard: the economic boom of 2000–2007 was followed by a dramatic downturn in Estonia, with its GDP falling by 4.2% in year 2008 and by 14.1% in 2009 (Eurostat). The Estonian government responded to the “economic crisis” as if it was also a “fiscal crisis” and undertook several rounds of budget cuts and tax increases in order to curtail the deficit, starting in 2008. In 2008, an austerity package was adopted in the middle of the year, cutting expenditures by about 2% of GDP. In 2009, the fiscal consolidation measures, adopted in three austerity packages, amounted to more than 9% of GDP, with 2/3 of it falling on the expenditure side (Raudla and Kattel 2013). For 2010, expenditure reduction measures amounting to 2.3% of GDP were foreseen with decisions adopted in 2009 (OECD 2012).14 Cuts were applied to all expenditure categories, though operating expenses and transfers took a larger hit than investments. As Raudla and Kattel (2013) show, the expenditure measures in the austerity packages combined cheese-slicing measures with more targeted approaches: while operational expenses were subjected to across-the-board cuts, more selective cuts were applied to transfers and investments. In the case of operational expenses, a “cascading” cheese-slicing approach to making the cuts was used, meaning that while the cabinet and the MoF adopted a top-down approach in determining how much each ministry had to cut (for example, 8% in January 2009 and 7% in June 2009), it refrained from dictating the specific content of how to

13 The audit covered the Ministry of Finance, the Government Office and five line ministries (the the Ministry of Economic Affairs and Communications, the Ministry of the Environment, the Ministry of the Interior, the Ministry of Education and Research and the Ministry of Social Affairs). 14 Detailed description of budgetary decision-making in 2008 and 2009 is provided by Jõgiste et al. (2012) and Raudla (2013a).

11

curb the operating expenses, leaving it for the individual ministries to decide how to achieve these cuts (see Raudla 2013a). Among transfers, the main cuts were the following: sickness benefits were cut back, planned increases in unemployment benefits were postponed, the increase of old-age pension was curtailed and contributions to the funded pension scheme were postponed.15 In the case of investments, non-EU funded investments were decreased and EU-funded investments were accelerated. Though the Estonian economy returned to growth in 2010, the GDP per capita at the end of 2012 was still below the level of the pre-crisis period (Eurostat) and the fiscal environment since 2010 has been significantly tighter compared to the boom period of 2000-2007 (during which the ministries got used to positive supplementary budgets being passed in the middle of the fiscal year). Also, given the sovereign debt crises in Europe, which emerged in 2010-2012, the fiscal discourse in Europe during this period has been focused on fiscal discipline and austerity, which was also reflected in Estonia − in the form of a continued focus on constraining expenditure growth and securing sustainability of public finances. 4.2.2. Performance Budgeting System in Estonia Estonia has undertaken a number of steps to integrate PI into the budget process, primarily through pursuing closer links between strategic plans and budgets. Between 2002 and 2005, a system for developing strategic plans, performance plans and performance reports – and their role in the budget cycle – was put in place (via amendments to the organic budget law and an adoption of a regulation on strategic planning). As a result, the annual budget cycle in the executive branch contains the following elements.16 The first step in the annual budget cycle is the preparation of the State Budget Strategy (SBS), adopted by the government each year for the next four years on a rolling basis. As an input to the SBS, the ministries have to submit their development plans (together with a financial plan) which are consolidated into the SBS by the MoF. The development plans outline the goals of area of government (i.e., the parent ministry and their subordinate agencies) for the next four years, indicators that reflect the progress towards achieving these goals and relevant actions to be undertaken to achieve these goals. After the SBS has been approved, the line ministries prepare action plans (which are essentially the first year action plans of the 4-year development plans) and budget bids for implementing these plans and submit these documents to the MoF. These action plans also indicate goals, activities, and expected results. After negotiations between the MoF and line ministries, the cabinet approves the budget and submits it to the parliament.17 After the end of the fiscal year, the ministries also have to submit reports on the execution of the action plans, which, in principle should be taken into account when deciding on the next year’s budget. In sum, the budgeting system in Estonia can be categorized as presentational performance budgeting (with some elements of performance-informed budgeting). PI is presented together with financial information and, in principle, it is expected that PI is considered when

15 For more detailed overview of the content of the austerity packages, see Raudla and Kattel (2013) and OECD (2012). 16 For a more detailed description of the budget process in Estonia, see Kraan et al. (2008). 17 For a discussion of legislative phase of the budget process in Estonia, see Raudla (2012).

12

appropriations are discussed. The provisions in the organic budget law and also in the regulation that concerns strategic plans and reports imply that the Ministry of Finance should take PI into account when putting together the draft budget. Since 2007 the MoF has been preparing a performance budgeting reform that would establish closer links between PI and budget allocations but so far the reform has not materialized.18 4.3. Findings of the COCOPS Survey Although the COCOPS survey questionnaire did not directly ask the question whether and how PI information was used for budgetary decision-making during and after the recent crisis, it was conducted in the relative aftermath of the crisis (in 2012) and also included a number of questions that dealt with cutbacks, use of management instruments, and the role of PI. Hence it can help to shed some light on the research questions of this paper. The data reported below focuses on answers given by the public officials who work in the ministries (in order to secure comparability with the qualitative research discussed in section 4.4).19 When asked about general saving strategies during the recent crisis (across the board cuts vs targeted cuts vs productivity savings), 52% of the ministry officials answered that across-the-board cuts were applied; whereas 39% claimed that predominantly targeted cuts were applied and 7% chose the category of cuts based on productivity gains. In addition, Estonian public officials were asked about the specific cutback approaches used to undertake targeted cuts. As can be seen from Table 1, the general perception of the ministry officials is that political priorities were cut less during the crisis (ca 81% agreement); that better performing areas were not cut less (ca 57% agreement); that the cuts did not depend on the personal relations between the department head and the budget office (ca 69% agreement). With regard to the dependence on the cuts on the communication skills of the unit manager, majority of the respondents (ca 58%) reflected that it did (rather) not play any role. Table 1: If targeted cuts have been made, what influenced the size of the cuts?

Not at

all 2 3 4 5 6

To a large extent

Political priorities were cut less 3% 2% 2% 12% 34% 31% 16 %

Better performing areas were cut less

20% 18% 19% 21% 12% 6% 4%

Cuts depended on relations of unit head and budget office

35% 25% 9% 13% 13% 3% 2%

Cuts depended on the communication skills of unit head

25% 21% 12% 11% 18% 9% 4%

The COCOPS questionnaire also included a question whether the role of PI had increased in response to the crisis. As can be seen from Table 2, the perceptions of the ministry officials diverge: about 42% of the officials agree with the statement, whereas about 33% of the respondents disagree and a relatively big share (25%) has a rather neutral position. 18 For an overview of the reform preparation process see Raudla (2013b). 19 The sample size of ministry officials was between from 88 and 123 (depending on the question).

13

Table 2: In response to the crisis, did the role of performance information increase? The relevance of performance information has increased

Not at all 2 3 4 5 6

To a large extent

7% 14% 12% 25% 20% 14% 8% In the COCOPS survey, the public officials were asked about their perception of their role as public sector executive. As can be seen from Table 3, more than 90% of the ministry officials in Estonia perceive their role to be connected to “achieving results” and more than 84% as “ensuring efficient use of resources”. Table 3: Identity and self-understanding of the ministry officials I mainly understand my role as public sector executive as:

Strongly disagree 2 3 4 5 6

Strongly agree

Ensuring efficient use of resources 0% 2% 2% 11% 14% 34% 37%

Achieving results

1% 1% 2% 6% 16% 31% 43%

The COCOPS questionnaire also included questions about the use of different public management instruments like strategic planning, management by objectives, cost-accounting systems, decentralization of financial decisions, performance related pay, and staff performance appraisal, which are likely to have repercussions on the possibilities to utilize performance-based budgeting. As can be seen from Table 4, most of the ministry officials agree that strategic planning (about 79% agreement), staff performance appraisal (about 87% agreement) and to some extent also cost-accounting systems (about 55% agreement) are used in their organizations. Answers diverge with regard to the use of management by objectives: about 49% of the respondents claim it is rather not used, whereas ca 31% claim the opposite. Most of the officials claim that decentralization of financial decisions and performance-related pay are rather not used in their organization (52% and 67% claim rather no usage of these instruments respectively). Table 4: Use of management instruments in ministries

To what extent are the following instruments used in your ministry?

Not at All 2 3 4 5 6

To a large

extent Business/strategic planning 1% 2% 10% 8% 14% 37% 28% Management by objectives and results 12% 19% 18% 20% 15% 11%

5%

Cost accounting systems 7% 8% 13% 17% 22% 21% 12%

Decentralization of financial decisions 12% 18% 22% 22% 14% 6% 6% Performance related pay 39% 17% 11% 7% 16% 7% 3% Staff appraisal talks / performance appraisal 1% 5% 2% 5% 9% 29% 49%

When asked about the reform trend of focusing on outcomes and results, about 65% of the ministry officials answered that it is important (and 14% that it is not important, with 21% of the answers falling in between).

14

Finally, the COCOPS survey also addressed the role of PI in public sector organizations. As can be seen from Table 5, the ministry officials’ claim to measure mostly outputs and outcomes (about 53% agreement), and not to measure inputs and processes (about 51% disagreement). With regard to feeling rewarded for achieving their goals, the responses diverge: ca 36% agree and ca 43% disagree with the statement. When turning to sanctions for not achieving the goals, most of the officials (ca 70%) feel they do not face sanctions for not achieving the goals. Similarly, most of the officials (ca 63%) feel that political leaders do not use indicators to monitor their performance. Table 5: Role of PI in ministries

To what extent do the following statements apply to your organization?

Strongly disagree 2 3 4 5 6

Strongly agree

We mainly measure inputs and processes

6% 17% 29% 21% 16% 8% 3%

We mainly measure outputs and outcomes

5% 2% 12% 27% 32% 16% 6%

We are rewarded for achieving our goals

10% 18% 15% 21% 20% 14% 2%

We face clear sanctions for not achieving our goals

14% 27% 29% 21% 6% 2% 1%

Politicians use indicators to monitor our performance

17% 22% 24% 16% 12% 6% 3%

In sum, the results of the COCOPS survey present a somewhat puzzling picture of the role of PI in the Estonian ministries. On the one hand, the ministry officials in Estonia perceive their role to be primarily about achieving results and ensuring cost-efficient use of resources; also some instruments related to performance budgeting (e.g. strategic planning, cost-accounting, and staff appraisal) are claimed to be used in the ministries; the reform trend of focusing on results is considered to have increasing importance in their organizations; and the ministries focus rather on outputs and outcomes than on inputs and processes. On the other hand, management by objectives as an instrument is claimed not to be used extensively; the officials feel that they do not face sanctions or rewards for achieving the goals and that politicians do not use indicators to monitor their performance. According to the top officials, during the crisis, better performing areas were not cut less, but political priorities were cut less and the assessments of whether the role of PI has increased as a result of the crisis diverge. Such ambiguous results certainly warrant a closer examination of the dynamics in the Estonian ministries via a qualitative study. 4.4. Findings of the Qualitative Studies 4.4.1. Use of Performance Information during the Fiscal Crisis in 2008-2009 Ministry of Finance The five interviewed officials from the MoF all concurred that PI was not taken into account in the cutback decisions for the austerity packages; instead, all sectors and ministries faced a (more or less) equal share of cutbacks. As several interviewees emphasized, in an atmosphere of

15

“emergency”, there was simply no time for conducting any analyses that would allow utilizing PI for the cutback decisions (e.g., Interviews E, F). Some of the interviewees also admitted that it would have been difficult to achieve political consensus and agreements in the cabinet on the cutbacks if a more differentiated approach to making the cutbacks had been chosen; no line ministry would have “volunteered” to implement larger cuts than the others; therefore, the equal share approach was the only one that was politically feasible (interviews E; F). One of the interviewees also noted that if the government had set out to make cuts on the basis of PI, it might not have yielded sufficiently large cuts (Interview B). At the same time, the interviewees agreed that some highly prioritized areas (like education) were less hit by cuts. Further, one of the interviewees claimed that although there were no discussions on how cutbacks would influence the achievement of specific targets in sectoral strategic plans or organizational development plans, the MoF officials undertook at least some “impact assessment” (i.e. impacts on the citizens, on the general state of economy, on inflation) before the cutback decisions were taken (Interview E) and they were more hesitant to impose cutbacks in those areas of which they had less knowledge (Jõgiste et al. 2012). Line ministries All the interviewed line ministry officials agreed that in deciding on the expenditure cuts during the heights of the crisis in 2009, PI was not taken into account. Many of the interviewees noted that because the cutback decisions in 2009 had to be taken very fast, there was simply no time to analyze PI and adopting across-the-board cuts was the fastest option. If a more “differentiated” approach implementing the spending cuts was at all undertaken by a ministry, then instead of analysing PI, cutback decisions followed other principles or rules of thumb. Most interviewees noted that the ministries cut those expenditures that were “cuttable”. The main principles guiding the cutbacks were the following: first of all, planned, but not yet implemented activities were postponed; areas not directly related to the provision of public services were addressed and lastly expenditures related to the “staff” rather than the “line” were cut (Interviews O, P). In some cases, some additional criteria were used. For example, in the Ministry of Agriculture, the size and the existing level of resources of the organizations were taken into account (e.g. museums, which were small and were already struggling with resources, faced a smaller cut than larger organizations) (Interview H). Another interviewee emphasized that the utmost aim was to “leave” the cuts in the organization and spare the citizens (avoid cuts in benefits and support measures) (Interview P). Many cuts on operational costs were also symbolic – “no colour-printing”, “no free coffee” etc – in order to pass on the general mentality of austerity (Interviews O, R). 4.4.2. Use of PI after the Fiscal Crisis: 2010 and Onwards Ministry of Finance In their answers given to the NAO during the audit conducted in 2011, the Ministry of Finance officials maintained that the MoF is using PI in budgetary negotiations and decision-making (NAO 2012). The analysis of the documents undertaken by the NAO in the course of the audit, however, revealed that, for most part, PI does not constitute part of the budgetary discussions and decision-making within the MoF or in the negotiations between the line ministries or the MoF

16

(NAO 2012). As the audit report points out, the MoF officials do indeed analyse the development plans submitted by the line ministries during the budget process, but they concentrate on checking whether the plans comply with the requirements of the strategic planning regulation (and on how well-phrased the goals and indicators are) rather than on the substance of the PI provided. As the audit report notes, the main focus of the MoF in budget negotiations with the line ministries is to “maintain mathematically calculated limits” (p. 2) and to make sure that the budget requests of the line ministries remain within the ceilings imposed on them by the MoF. In our interviews (conducted for academic research), the MoF officials were somewhat more open about the actual non-use of the PI in the budget process. The interviewed ministry of finance officials presented somewhat diverging assessments about the extent to which PI has been raised in budgetary discussions and taken into account in budgetary decision-making – depending on how they defined “performance information” and “taking it into account”. One of the interviewees, for example, noted that while during the crisis, PI was largely left out from the SBS, after the acute phase of the crisis was over there was again more “space” to include it (interview E). She added that in contrast to the budget discussions in the cabinet in 2009 (when no PI was discussed), in 2011 the cabinet ministers raised questions based on certain performance indicators when discussing the SBS. Another official argued that if under “taking PI into account” one understands “having PI as necessary background information for budgetary discussions”, it is “kind of taken into account”. As he put it, “a ministry cannot just submit a request for a certain amount of money”, they have to explain what they are going to do with that money, what kind of activities they are going to undertake and this takes the form of PI (interview F). When asked about explicit links between PI and resource allocation, however, most of the interviewed MoF officials agreed that in specific budgetary decisions PI is largely not taken into account. Interestingly, one of the officials even openly admitted that the answers given to the NAO overstated the actual use of PI in MoF. The officials cited different reasons, however, for why PI is not taken into account in the budget process. Some of them argued that “the amount of PI submitted is too large and we don’t have the capacity to analyse it all” (e.g., interview F). Others noted that “we simply don’t know how to take it into account in budgetary decision-making” or “we don’t have a methodology for it” (e.g., interview D). Yet others blamed the low quality of the PI submitted by the line ministries, rendering it un-usable (e.g., interview B). All interviewees agreed that establishing direct links between PI and funding decisions is challenging because of attribution problems. It was also pointed out that a large portion of the expenditures in the budget is fixed (i.e. mandated by laws) and hence the leeway to adjust resource allocations according to PI is limited. Some also mentioned that the root of the problem of non-use of PI is the administrative culture and the attitudes of the public officials (interviews B, D). Some argued that a change of budget format (from input-based to results-based) would facilitate the linking of PI and budgeting (B, D); others emphasized that utilizing PI in budgetary decisions does not mean that the budget format would have to be changed (interview E).

17

Ministries According to the audit report of the NAO (2012), although the line ministries provide documents containing extensive amounts of PI, in actual budget negotiations with the MoF they do not, for the most part, use arguments referring to PI to justify budget requests. As the audit report points out, an important reason for that is that the line ministries, despite putting together lengthy performance documents, do not have the resources or capacity to analyse specific links between resources-activities-outputs-outcomes. In addition, the line ministry officials feel that using PI in budget requests would not convince the MoF. As was stated by an official from the Ministry of Economic Affairs and Communications “If we tried to tell the Ministry of Finance in budget negotiations that specific activities and monetary contributions would influence the achievement of indicators, the ministry would not believe this, because the connection remains too abstract in the case of major indicators, which means that making budget decisions on the basis of this would be impossible.” (NAO, p. 24) With regard to the internal budget process within the audited ministries, the auditor’s analysis of the documents indicated that the focus of budgetary discussions within the ministries tends to be on specific activities rather than on the “performance documents” (i.e., the development plan, action plan and action plan report) (NAO 2012). The audit report draws attention to one exception, however – the homeland security area in the Ministry of Interior, where expected output results have been used in making allocation decisions. Among the reasons why PI is not used for making allocation decisions and why the year-to-year changes in allocations are largely incremental, the officials of the audited ministries pointed to the following issues. First, it was noted that since a large portion of the budget expenditures is fixed by laws (around 75%), there isn’t much leeway to make re-allocations in the first place. The re-allocations within the remaining 25% are usually based on other considerations than PI (e.g. international obligations or changes in the administrative division of tasks) (p. 24). Second, it was pointed out that identifying clear links between resources and outcomes is difficult. While some of the officials argued that “they just don’t have a proper methodology” for it yet (NAO 2012, p. 15) or that they don’t have enough resources to conduct or purchase the analyses (p. 16), others pointed to attribution problems in identifying clear links between activities and outcomes, which arise from the nature of the public sector and complexity of the phenomena involved (p. 16). In the interviews conducted by the academic researchers, the answers of the line ministry officials to a large extent corroborated the findings of the audit report: most of the interviewees note that PI is not used for budgetary decision-making, although some exceptions can be observed. In the Ministry of Social Affairs, for example, spending reviews are undertaken for the non-mandated programmes and if the programmes are not deemed to be effective, their funding is discontinued (Interview J). A number of interviewees claimed that in the internal budget process of the line ministries, the heads of departments sometimes do try to use PI for advocacy purposes (in order to explain additional funding needs), but that often the success of such attempts depends more on the individual persuasion powers and personal relations of the individuals involved than the objective merits of the PI used (Interview M). Also in the budgetary negotiations between line ministries and the MoF such dynamics of informal

18

communications were observed; as one of the interviewees put it, “When it comes to making budgetary decisions, then if we need anything our general secretary calls the general secretary of the MoF − this is how it gets decided.” (Interview A). In addition to the reasons indicated in the audit report (NAO 2012) for why PI is not used in budgetary negotiations or decision-making within the line ministries, the interviews pointed to the following factors. A number of interviewees emphasized the political nature of budgetary decision-making, which challenges the use of PI in allocation decisions; they noted that if budgetary re-allocations in (the generally incremental) budgeting do take place, these are often driven by the preferences of politicians in power. For example, in the Ministry of Justice, the new incoming minister decided to make criminal policy a priority and hence allocated additional resources to that sector (Interview M). Several officials argued that the budget process is highly time-constrained, which leaves limited time for analysing PI. It was also noted, that the context of chronic scarcity is making it difficult to provide space for discussing PI in the budgetary negotiations (Interview G). 5. Concluding Remarks As the Estonian case demonstrates, in a situation of fiscal crisis, when the decision-makers are intent on adopting sizable fiscal consolidation packages, the incrementalist prediction – that PI would not be used in budgetary decision-making – is corroborated (both by the interviews and survey results). The main reasons for the non-use of PI in devising austerity measures during fiscal crisis in Estonia were the extreme time pressure for adopting the austerity packages, the political nature of the budgetary process, and limited analytical capacities for gathering and interpreting PI both in the line ministries and the ministry of finance. In the aftermath of a period of fiscal crisis, however, when fiscal stress continues (but is not so extreme anymore), the Estonian case shows that the understanding provided by interactive-dialogue theory of PI is the most accurate: PI is “used” in budgetary negotiations, broadly speaking, but different actors perceive its role and impact differently. Some MoF officials consider the submission of PI as background information in the budget process already as evidence of its “use”. Some line ministry officials also agree that, very generally speaking, PI “somehow” plays a role in the budget process. When probed for more specific examples, however, the evidence for utilization becomes rarer and from all ministries, only few examples where PI has directly influenced allocation decisions can be found. For most part, as the evidence gathered from interviews shows, budgetary decision-making is driven by other concerns (like political preferences, legal mandates and international obligations). One could even argue that the interactive-dialogue theory of PI use would also be helpful in explaining why different methodological approaches to investigate the use of PI lead to diverging results. When public officials are asked about whether they use PI, the answers appear to depend significantly on who asks, in what context and for what purpose. In a more decontextualized setting of survey research, the officials would be more likely to respond on the basis of what they “would like to believe” (or think they “should believe”). The Estonian case demonstrates that according to the survey results, the role-perception of the public officials can be strongly oriented to “achieving results” and “ensuring efficient use of resources”, but these

19

attitudes are not necessarily translated into using PI in making budgetary decisions (as shown by interviews). The Estonian case also showed that in the context of official audit, the normative pressures to confirm the use of PI in budgetary decision-making may be especially strong, especially by MoF officials who are “expected to be using PI” (since the MoF requires the line ministries to submit PI). In interviews conducted by independent academic researchers, however, interviewees tend to be more open about whether and how they actually use PI in budget process. In future research, the theoretical framework outlined in this paper could be further elaborated upon. In particular, additional dimensions – e.g. the distinction between the ministry and the agency level, differentiation between various sectors, and demarcation of different management functions (e.g. budgeting vs other functions) – could be added in order to develop more specific hypotheses about the use of PI in various types of fiscal environments. These hypotheses could then be tested with the cross-national database put together from the results of the COCOPS survey. Acknowledgements: The research leading to these results has received funding from the European Union’s Seventh Framework Programme under grant agreement No. 266887 (Project COCOPS), Socio-economic Sciences & Humanities. References Askim, J. (2008). “Determinants of Performance Information in Political Decision Making.” In S. Van de Walle and W. Van Dooren (eds), Performance Information in the Public Sector: How it is Used? Palgrave Macmillan, pp. 125-39. Askim, J. (2007). “How Do Politicians Use Performance Information? An Analysis of the Norwegian Local Government Experience.” International Review of Administrative Sciences 3(3): 453-472. Behn, R. D. (1980). “Leadership for Cut-back Management: The Use of Corporate Strategy.” Public Administration Review 40(6): 613-620. Brumby, J. (1999). “Budgeting reforms in OECD member countries.” In S. Schiavo-Campo and D. Tommasi (eds) Managing Government Expenditure. Manila: Asian Development Bank. pp. 349-62. Brumby, J., and M. Cangiano. (2001). “Public Expenditure Management Reform and Fiscal Consolidation in OECD Countries.” Paper presented at the Fifth International Conference Institutions in Transition in Otocec, Slovenia. http://www.umar.gov.si/fileadmin/user_upload/konference/06/02_cangiano.pdf Burke, B.F., and B.C. Costello (2005). “The Human Side of Managing for Results.” The American Review of Public Administration 35(3): 270-286. Caiden, N. (1998). “Public Service Professionalism for Performance Measurement and Evaluation.” Public Budgeting & Finance 18(2): 35-52. Carlin, T.M. & Guthrie, J. (2003). “Accrual Output Based Budgeting Systems in Australia.” Public Management Review 5(2): 145-162. Connelly, M. and G.L. Tompkins (1988). “Does Performance Matter? A Study of State Budgeting.” Review of Policy Research 8(2): 288-299.

20

Cothran, D.A. (1993). “Entrepreneurial Budgeting: An Emerging Reform?” Public Administration Review 53: 445-454. Curristine, T. (2005a). “Government Performance: Lessons and Challenges.” OECD Journal on Budgeting 5(1): 127-151. Curristine, T. (2005b). “Performance Information in the Budget Process: Results of the OECD 2005 Questionnaire.” OECD Journal on Budgeting 5(2): 87-131. De Lancer Julnes, P.D.L., and M. Holzer (2001). “Promoting the Utilization of Performance Measures in Public Organizations: An Empirical Study of Factors Affecting Adoption and Implementation.” Public Administration Review 61(6): 693-708 Downs, G.W., and D.M. Rocke. (1984). “Theories of Budgetary Decision-making and Revenue Decline. Policy Sciences 16(4): 329-347. Dunsire, A. and C. Hood (1989). Cutback Management in Public Bureaucracies. Popular Theories and Observed Outcomes in Whitehall. Cambridge: Cambridge University Press. Frank, H.A., and J. D’Souza (2004). “Twelve Years into the Performance Measurement Revolution: Where We Need to Go in Implementation Research.” International Journal of Public Administration 27(8-9): 701-718. Gilmour, J.B. and D.E. Lewis, D. E. (2006). “Does Performance Budgeting Work? An Examination of the Office of Management and Budget’s PART Scores.” Public Administration Review 66(5): 742-752. Glassberg, A. (1978). “Organizational responses to municipal budget decreases.” Public Administration Review 38(4): 325-332. Good, D. A. (2011). “Still Budgeting by Muddling Through: Why Disjointed Incrementalism Lasts.” Policy and Society 30(1): 41-51. Halachmi, A. (2005). “Performance Measurement: Test the Water Before you Dive in.” International Review of Administrative Sciences 71(2): 255-266. Hammerschmid, G., A. Oprisor and V. Stimac. (2013). “COCOPS Executive Survey on Public Sector Reform in Europe Research Report.” Available at: http://www.cocops.eu/wp-content/uploads/2013/06/COCOPS-WP3-Research-Report.pdf Hatry, H.P. 2002. “Performance Measurement: Fashions and Fallacies.” Public Performance and Management Review 25(4): 352–58. Ho, A.T.K. (2011). “PBB in American Local Governments: It’s More than a Management Tool.” Public Administration Review 71(3): 391-401. Hou, Y., R.S. Lunsford; K.C. Sides and K.A. Jones (2011). “State Performance‐Based Budgeting in Boom and Bust Years: An Analytical Framework and Survey of the States.” Public Administration Review 71(3): 370-388. Jimenez, Benedict S., 2013a. “Performance Management and Deficit Adjustment in U.S. Cities during the Recent Fiscal Crisis.” Available at http://dx.doi.org/10.2139/ssrn.2218856 Jimenez, B. (2013b). “Smart Cuts?: Strategic Planning, Performance Management and Budget Cutting in US Cities During the Great Recession.” Forthcoming in the Journal of Public Budgeting, Accounting & Financial Management. Jõgiste, K., P. Peda and G. Grossi (2012). “Budgeting in a Time of Austerity: The Case of the Estonian Central Government.” Public Administration and Development 32(2): 181-195.

21

Jordan, M. M., and M. M. Hackbart (1999). “Performance Budgeting and Performance Funding in the States: A Status Assessment.” Public Budgeting & Finance 19(1): 68-88. Joyce, P. G. (2008). “Does More (or Even Better) Information Lead to Better Budgeting? A New Perspective.” Journal of Policy Analysis and Management 27(4): 945-960. Joyce, P. G. (1997). “Using Performance Measures for Budgeting: A New Beat, or is it the Same Old Tune?” New Directions for Evaluation 1997(75): 45-61. Joyce, P. G. (1996). “Appraising Budget Appraisal: Can You Take Politics Out of Budgeting?” Public Budgeting & Finance 16(4): 21-25. Joyce, P. G. (1993). “Using Performance Measures for Federal Budgeting: Proposals and Prospects.” Public Budgeting & Finance 13(4): 3-17. Kelly, J., and J. Wanna. 2004. “Crashing through with Accrual-Output Price Budgeting in Australia: Technical Adjustment or a New Way of Doing Business?” The American Review of Public Administration 34(1): 94-111. Kong, D. (2005). “Performance-based Budgeting: The US Experience.” Public Organization Review 5(2): 91-107. Kraan, D-J., J. Wehner and K. Richter. (2008). “Budgeting in Estonia.” OECD Journal on Budgeting 8(2): 1-40. Lauth, T. P. (1985). “Performance Evaluation in the Georgia Budgetary Process.” Public Budgeting & Finance 5(1): 67-82. Levine, C.H. (1979). “More on Cutback Management. Hard Questions for Hard Times.” Public Administration Review 39(2): 179-183. Levine, C.H. (1978). “Organizational Decline and Cutback Management.” Public Administration Review 38(4): 316-325. Levine, C.H., I. Rubin and G.G. Wolohojian (1981). “Resource Scarcity and the Reform Model. The Management of Retrenchment in Cincinnati and Oakland.” Public Administration Review 41(6): 619-628. MacManus, S.A. (1984). “Coping with Retrenchment: Why Local Governments Need to Restructure their Budget Document Formats.” Public Budgeting & Finance 4(3): 58-66. Marcel, M. 2013. “Budgeting for Fiscal Space and Government Performance Beyond the Great Recession.” Paper presented at the fifth annual meeting of OECD parliamentary budget officials and independent fiscal institutions in Ottawa, Canada, February 21-22, 2013. Massey, J., and J.D. Straussman (1981). “Budget Control Is Alive and Well: Case Study of County Government.” Public Budgeting & Finance 1(4): 3-11. Melkers, J., and K. Willoughby (2005). “Models of Performance‐Measurement Use in Local Governments: Understanding Budgeting, Communication, and Lasting Effects.” Public Administration Review 65(2): 180-190. Melkers, J. E., & Willoughby, K. G. (2001). “Budgeters’ Views of State Performance‐Budgeting Systems: Distinctions across Branches.” Public Administration Review 61(1): 54-64. McTighe, J.J. (1979). “Management Strategies to Deal with Shrinking Resources.” Public Administration Review 39:86-90. Moynihan, D. P. (2008a). The Dynamics of Performance Management: Constructing Information and Reform. Georgetown University Press.

22

Moynihan, D.P. (2008b). “Advocacy and Learning: An Interactive-dialogue Approach to Performance Information use.” In W. Van Dooren and S. Van de Walle (eds.), Performance Information in the Public Sector: How It Is Used. Basingstoke: Palgrave Mc Millan. pp. 24-41 Moynihan, D.P. (2006). “What Do We Talk about When We Talk about Performance? Dialogue Theory and Performance Budgeting.” Journal of Public Administration Research and Theory 16(2): 151-168. Murray, L. W., and A.M. Efendioglu. (2009). “State Governments Face Cash Crunch: It’s Time for Performance-Based Budgeting.” Proceedings of the 17th Annual International Conference. Quebec: The Association on Employment Practices and Principles. National Audit Office of Estonia (NAO) (2012). “Activities of Government of the Republic in Assessing Impact of its Work and Performance Reporting.” Audit report. Accessible at: http://www.riigikontroll.ee/tabid/206/Audit/2265/Area/1/language/et-EE/Default.aspx OECD (2012). “Estonia.” In Restoring Public Finances. OECD Publishing. Olden, B., D. Last, S. Yläoutinen, S., and C. Sateriale (2012). “Fiscal Consolidation in Southeastern European Countries: The Role of Budget Institutions.” IMF Working Paper WP/12/113. Accessible at http://www.iadb.org/intal/intalcdi/PE/2012/11092.pdf Pitsvada, B., and F. LoStracco (2002). “Performance Budgeting-the Next Budgetary Answer. But What Is the Question?” Journal of Public Budgeting, Accounting and Financial Management 14: 53-74. Plant, J. F. and L.G. White (1982). “The Politics of Cutback Budgeting: An Alliance Building Perspective.” Public Budgeting & Finance 2(1): 65-71 Pollitt, C. (2010). “Cuts and Reforms. Public Services As We Move into a New Era.” Society and Economy 32(1): 17-31. Pollitt, C. (2006). “Performance Information for Democracy The Missing Link?” Evaluation 12(1): 38-55. Pollitt, C. (2001). “Integrating Financial Management and Performance Management.” OECD Journal on Budgeting 1(2): 7-37. Pollitt, C. and G. Bouckaert (2011). Public Management Reform: A Comparative Analysis: New Public Management, Governance, and the Neo-Weberian State. Oxford: Oxford University Press. Raudla, R. (2013a). “Fiscal Retrenchment in Estonia during the Crisis: The Role of Institutional Factors.” Public Administration 91(1): 32-50. Raudla, R. (2013b). “Pitfalls of Contracting for Policy Advice: Preparing Performance Budgeting Reform in Estonia.” Forthcoming in Governance: An International Journal of Policy, Administration, and Institutions. DOI: 10.1111/gove.12006. Raudla, R. (2012). “The Use of Performance Information in Budgetary Decision-Making by Legislators: Is Estonia Any Different?” Public Administration 90(4): 1000-1015. Raudla, R. and R. Kattel (2013). “Fiscal Stress Management During the Financial and Economic Crisis: The Case of the Baltic Countries.” International Journal of Public Administration, 36(10): 732-742. Raudla, R. and R. Kattel (2011). “Why Did Estonia Choose Fiscal Retrenchment after the 2008 Crisis?” Journal of Public Policy 31(2): 163-186.

23

Raudla, R, R. Savi and T. Randma-Liiv (2013). “Literature Review on Cutback Management.” Accessible at: http://www.cocops.eu/wp-content/uploads/2013/03/COCOPS_Deliverable_7_1.pdf Robinson, M. and J. Brumby. (2005). “Does Performance Budgeting Work?: An Analytical Review of the Empirical Literature.” IMF Working Paper. Available at SSRN: http://ssrn.com/abstract=888079 Rubin, I. (1993). The Politics of Public Budgeting: Getting and Spending, Borrowing and Balancing. Chatham, NJ: Chatham House. Schick, A. (1990). “Budgeting for Results: Recent Developments in Five Industrialized Countries.” Public Administration Review 50: 26-34. Schick, A. (1988). “Micro-Budgetary Adaptations to Fiscal Stress in Industrialized Democracies.” Public Administration Review 48(1): 523-533. Schick, A. (1986). “Macro-Budgetary Adaptations to Fiscal Stress in Industrialized Democracies.” Public Administration Review 46(2): 124-134. Schick, A. (1983). “Incremental Budgeting in a Decremental Age.” Policy Sciences 16(1): 1-25. Schick, A. (1980). “Budgetary Adaptations to Resource Scarcity.” In C.H. Levine, and I. Rubin (eds) Fiscal Stress and Public Policy. London: Sage Publications, pp.113-134. Stiefel, L., R. Rubenstein and A.E. Schwartz. (1999). “Using Adjusted Performance Measures for Evaluating Resource Use.” Public Budgeting & Finance 19(3): 67-87. Straussman, J. D. (1979). “A Typology of Budgetary Environments: Notes on the Prospects for Reform.” Administration & Society 11(2): 216-226. Swain, J.W. and C.J. Hartley, Jr. (2001). “Incrementalism: Old but Good?” In J.R. Bartle (ed.) Evolving Theories of Public Budgeting. Amsterdam: JAI Press, Amsterdam. Troupin, S., J. Stroobants and T. Steen. (2013). “The Impact of the Fiscal Crisis on Belgian Federal Government: Changes in the Budget Decision Making Process and Intra-Governmental Relations.” Paper presented at the Administrative Culture conference on the Impact of the Fiscal Crisis on Public Administration, 3-4 May 2013, Tallinn, Estonia. Available at: https://lirias.kuleuven.be/bitstream/123456789/398559/1/COCOPS-WP7_Federal+Belgium.pdf Van Dooren, W. (2011). “Better Performance Management.” Public Performance & Management Review 34(3): 420-433. Van Dooren, W. (2005). “What Makes Organisations Measure? Hypotheses on the Causes and Conditions for Performance Measurement.” Financial Accountability & Management 21(3): 363-383. White, J. (2012). “Playing the Wrong PART: The Program Assessment Rating Tool and the Functions of the President's Budget.” Public Administration Review 72(1): 112-121. Wildavsky, A. 1978. “A Budget for All Seasons? Why The Traditional Budget Lasts.” Public Administration Review 38(6): 501-509. Wildavsky, A. 1969. “Rescuing Policy Analysis from PPBS.” Public Administration Review 29(2): 189-202. Wildavsky, A. 1966. “The Political Economy of Efficiency: Cost-Benefit Analysis, Systems Analysis, and Program Budgeting.” Public Administration Review 26(4): 292-310.

24

Wildavsky, A. B., and N. Caiden (1988). The New Politics of the Budgetary Process. Glenview, IL: Scott, Foresman. Willoughby, K. G. (2004). “Performance Measurement and Budget Balancing: State Government Perspective.” Public Budgeting & Finance 24(2): 21-39. Willoughby, K. G., and J.E. Melkers (2000). “Implementing PBB: Conflicting Views of Success.” Public Budgeting & Finance 20(1): 85-120. Zaltsman, A. (2009). “The Effects of Performance Information on Public Resource Allocations: A Study of Chile's Performance-Based Budgeting System.” International Public Management Journal 12(4): 450-483. Appendix I

Interviews

Interview A: official of the Ministry of Culture, 12 June 2010

Interview B: official of the Ministry of Finance, 21 April 2011

Interview C: official of the Ministry of Finance, 2 May 2011

Interview D: official of the Ministry of Finance, 4 May 2011

Interview E: official of the Ministry of Finance, 6 May 2011

Interview F: official of the Ministry of Finance, 11 May 2011

Interview G: official of Ministry of Culture, 3 April 2013

Interview H: three officials of the Ministry of Agriculture, 18 April 2013

Interview I: official of the Ministry of Interior 21 April 2013.

Interview J: official of the Ministry of Social Affairs, 25 April 2013.

Interview K: official of the Ministry of Culture, 26 April 2013

Interview L: two officials of the Ministry of Social Affairs, 26 April 2013

Interview M: official of the Ministry of Justice, 26 April 2013

Interview N: official of the Ministry of Justice 29 April 2013

Interview O: official of the Ministry of Economic Affairs and Communications, 6 March 2012

Interview P: official of the Ministry of Social Affairs, 13 March 2012

Interview R: official of the Ministry of Finance, 27 February 2012