How can the information content of neutral business survey responses be exploited for forecasting purposes?

1

3rd CESIfo Conference on Survey Data in Economics –

Methodology and Applications

Munich, 6 November 2009

http://www.cesifo-group.de/ifoHome/events/Archive/conferences/2009/11/2009-11-06-event-

ConfIfo.html

How can the information content of neutral business

survey responses be exploited for forecasting

purposes?

Oscar Claveria† and Anna Stangl‡ [email protected] ; [email protected]

† Research Institute of Applied Economics (IREA), University of Barcelona, Barcelona (Spain)

‡ Ifo Institute for Economic Research, Munich (Germany)

Abstract

Business surveys are an essential tool for gathering information about the

development of the economy. Survey results are presented as weighted percentages of

respondents expecting a particular variable to rise, fall or remain unchanged. The aim of

the paper is to analyze whether taking into account the percentage of respondents

expecting a variable to remain constant helps to improve the forecasting performance of

survey results. With this objective we present a variation of the balance statistic

(weighted balance).

By means of a simulation experiment we test whether a variation of the balance

statistic outperforms the balance statistic in order to track the evolution of agents’

expectations and produce more accurate forecasts of the generated quantitative variable

used as a benchmark. We then use the information from a Business Survey in the

German manufacturing sector in autoregressions and Markov switching regime models

to analyze the predictive performance of the proposed methods in tracking the

Production Index.

The results of the forecasting evaluation permit us to conclude that taking into

account the fraction of respondents expecting a variable to remain constant improves the

forecasting performance of business survey results.

Keywords: Business surveys; Quantification; Expectations; Forecasting

JEL classification: C42, C51, C53, C82, D12, D84

http://www.cesifo-group.de/ifoHome/events/Archive/conferences/2009/11/2009-11-06-event-ConfIfo.html

http://www.cesifo-group.de/ifoHome/events/Archive/conferences/2009/11/2009-11-06-event-ConfIfo.html

mailto:[email protected]

mailto:[email protected]

2

1. INTRODUCTION

Business surveys are an essential tool for gathering information about the

development of the economy. As a rule, survey results are presented as percentages of

respondents expecting a particular variable to rise, remain constant or fall.1 The aim of

this paper is to analyse whether taking into account the percentage of respondents

expecting a variable to remain constant helps to improve the usefulness of survey results

in forecasting macroeconomic variables. With this objective we evaluate the

performance of an alternative way of exploiting the information content of survey

responses for forecasting purposes: a variation of the balance statistic, known as

weighted balance (Claveria, 2010).

Balance statistics (fraction of positive responses minus the fraction of negative

responses) is the most popular quantification method of business expectations applied in

practice and used by Eurostat to track the official data on economic growth, such as the

production index. The balance statistic ignores the fraction of neutral responses and

considers the proportion of negative and the proportion of positive responses only. This

procedure is problematic as neutral responses exhibit the highest frequency of

occurrence in surveys. In business surveys their proportion not seldom lies at 60-70

percent. Ignoring neutral responses consequently results in vast information loss.

The information content of neutral responses is not explicit. Neutral responses

may reflect: (1) a true view of a respondent about a variable remaining constant, (2)

positive or negative responses, which have not reached a particular threshold, and/or (3)

“epistemic uncertainty”. Bruine de Bruin et al. (2000) argue that heaping at the middle

of a scale reflects uncertainty, and only seldom represents the true view of the

respondent. The neutral category subsequently functions as a surrogate for a “don’t

know” or “unsure about the future development” option (even if a “don’t know” option

is available within the question).

The role of uncertainty in influencing economic behavior of agents, such as

investment decisions or monetary-policy, has been recognized early (Knight, 1921):

Economic agents may delay investment and consumption decisions if they are uncertain

1 The measurement procedure of business surveys on economic expectations in the European Union has

been harmonized by the European Commission to three-category rating questions.

3

about the future economic development. Empirical findings confirm that uncertainty has

cyclical properties and is higher around cyclical turning points (Ramey and Ramey,

1995, Mitchell et al., 2005, Doepke and Fritsche, 2006, Guiso and Parigi, 1999). As the

information content of neutral responses encompasses an uncertainty component, we

use the fraction of neutral responses to weight the balance statistics (weighted balance)

and examine their properties by a simulation study and in an empirical analysis.

The simulation experiment is designed to test whether the proposed variation of

the balance statistic (weighted balance) outperforms the balance statistic in order to

track the evolution of agents’ expectations and produces more accurate forecasts of the

quantitative variable generated that is used as a benchmark. Survey results are as a rule

quantified making use of official data. The differences between the actual values of a

variable and quantified expectations may arise from three different sources (Lee, 1994):

measurement or conversion error due to the use of quantification methods, expectational

error due to the agents’ limited ability to predict the movements of the actual variable,

and sampling errors. Since survey data are approximations of unobservable

expectations, they inevitably entail a measurement error. Monte Carlo simulations allow

us to distinguish between these sources of error.

We also use survey results of a business survey in the German manufacturing

sector in autoregressions and Markov switching regime models to analyze the predictive

performance of the various proposed methods of assessing business surveys results to

track the evolution of the German Production Index.

The paper is organized as follows. The second section describes the variation of

the balance statistic. Section three presents the simulation experiment and analyzes the

relative forecasting performance of the variation of the balance statistic. Section four

presents the empirical data and the models and evaluates the relative forecasting

performance of the various methods of assessing business surveys results described in

section two. Section five concludes.

4

2. THE WEIGHTED BALANCE STATISTIC

Unlike other statistical series, survey results are weighted percentages of

respondents expecting an economic variable to increase, decrease or remain constant.

As a result, tendency surveys contain two pieces of independent information at time t ,

tR and tF , denoting the percentage of respondents at time 1t expecting an economic

variable to rise or fall at time t . The information therefore refers to the direction of

change but not to its magnitude.

A variety of quantification methods have been proposed in the literature in order

to convert qualitative data on the direction of change into a quantitative measure of

agents’ expectations (see Claveria et al, 2006). The output of these quantification

procedures (estimated expectations) can be regarded as one period ahead forecasts of

the quantitative variable under consideration. In this paper we use agents’ expectations

about the future (prospective information) and compare the balance statistic to the

weighted balance.

The first attempt to quantify survey results is due to Anderson (1951). Assuming

that the expected percentage change in a variable remains constant over time for agents

reporting an increase and for those reporting a decrease, Anderson (1952) defined the

balance statistic ( tt FR ) as a measure of the average changes expected in the variable.

The balance statistic ( tB ) does not take into account the percentage of

respondents expecting a variable to remain constant ( tC ). As tC usually shows the

highest proportions and high levels of dispersion, we propose a non-linear variation of

the balance statistic ( tWB , weighted balance) that accounts for this percentage of

respondents:

t

t

tt

tt

tC

B

FR

FRWB

1 (1)

Weighting the balance statistic by the proportion of respondents expecting a

variable to rise or fall allows discriminating between two equal values of the balance

statistic depending on the percentage of respondents expecting a variable to remain

constant.

5

3. THE SIMULATION EXPERIMENT

By Monte Carlo simulations we compare the forecasting performance of the two

quantification methods: the balance and the weighted balance. The experiment is

designed in five consecutive steps:

(i) The simulation begins by generating a series of actual changes of a variable. We

consider 500 agents and 250 time periods. Let ity indicate the percentage change of

variable itY for agent i from time 1t to time t . Additionally we suppose that the true

process behind the movement of ity is given by:

ititit εdy (2)

500,,1i , 250,,1t and 1, tiit yφμd , where itd is the deterministic

component. The initial value, 85.00 iy , is assumed to be equal for all agents2. itε is an

identical and independent normally distributed random variable with mean zero and

variance 2

εσ =0.5, 5, 50 . The average rate of change, ty , is given by i iti yy

5001 .

The same weight is given to all agents. We assume different values of μ and φ .

(ii) Secondly, we generate a series of agents’ expectations about ty under the

assumption that individuals are rational in Muth’s sense3:

itit

e

it ζdy 2,0~ ζit σNζ (3)

where e

ity has the same deterministic part as ity but a different stochastic term itζ . We

derive i

e

it

e

i yy 5001 . Additionally, we assume that 122 ζζi σσ . All the values

given to 2

εσ and 2

ζσ , and to the indifference interval are set to simulate actual business

survey series.

2 To check the robustness of the results, we chose different values for the autoregressive

parameter, ranging from 0 to 1 with an increase of 0.1 each time. As the final results did not vary

significantly from one specification to the other we presented the results for 0, 0.3, 0.6 and 0.9. 3 Muth (1961) assumed that rationality implied that expectations had to be generated by the same

stochastic process that generates the variable to be predicted.

6

(iii) The third step consists of constructing the answers to the business surveys. The

answers are given in terms of the direction of change, i.e., if the variable is expected to

increase, decrease or remain equal. We assume that agents’ answers deal with the next

period and that all agents have the same constant indifference interval ba, with

5 ba . If 5.4e

ity , agent i answers that itY will increase; if 5.4e

ity , i expects

itY to decrease; while the agent will report no change if 5.45.4 e

ity . With these

answers, qualitative variables itR and itF can be constructed. itR ( itF ) takes the value 1

(0) whenever the agent expects an increase (decrease) in itY . tR and tF are then

constructed by aggregation.

(iv) The fourth step of the simulation experiment consists of using the two different

quantification methods to trace back the series of actual changes of the generated

quantitative variable, ty , from the qualitative variables. We will refer to these

expectations as estimated expectations in order to distinguish them from the

unobservable ones. With the aim of analysing the performance of the different proxy

series, we use the last 100 generated observations. Keeping the series of actual changes

fixed, the experiment of generating the rational expectations series as well as the proxy

series is replicated 1500 times4.

(v) To test the robustness of the results, we repeat the simulation experiment for

different values of μ , therefore assuming ity is generated by a random walk and by an

autoregressive process with different drifts.

In order to evaluate the relative performance and the forecasting accuracy of the

different quantification procedures, we keep the series of actual changes fixed and we

replicate the experiment of generating the rational expectations series as well as the

qualitative variables tR and tF 1500 times. The specification of the quantification

procedures is based on information up to the first 150 periods; models are then re-

estimated each period and forecasts are computed in a recursive way. In each

simulation, forecast errors for all methods are obtained for the last 100 periods.

4 All simulations are performed with Gauss for Windows 6.0.

7

In order to summarize this information, we calculate the Root Mean Squared

Error (RMSE), the Mean Error (ME), the Theil Coefficient (TC) and the three

components of the Mean Square Error (MSE): the bias proportion of the MSE (U1), the

variance proportion (U2) and the covariance proportion (U3). With the aim of testing

whether the reduction in RMSE when comparing both methods is statistically

significant, we calculate the measure of predictive accuracy proposed by Diebold-

Mariano (1995). Given these two competing forecasts and the series of actual changes

of the generated quantitative variable, we have calculated the DM measure which

compares the mean difference between a loss criteria (in this case, the root of the mean

squared error) for the two predictions using a long-run estimate of the variance of the

difference series.

Table 3.1 shows the results of an off-sample evaluation for the last 100 periods

when 0μ . Table 3.2 and Table 3.3 show the results of an off-sample evaluation for

the last 100 periods when 1μ and 1μ respectively.

Table 3.1 shows that the weighted balance (WB) shows lower RMSE, ME and

TC in all cases. Although the proportion of systematic error (U1) is not very different,

the balance shows higher proportions of regression error (U2). As 2

εσ increases,

forecasting results tend to worsen for both methods. Nevertheless if we look at the

results of the DM test, we can see that there is no significant difference between both

methods.

However, in Table 3.2 and Table 3.3, when 0μ , the difference is significant

and is always in favor of the weighted balance, with the exception of the scenario in

which 9.0φ and 502 εσ . Comparison of the results in Table 1 with those in Table 2

and Table 3 highlights several differences, in particular regarding the forecasting results

as the value of φ increases from 0 to 0.9: while they worsen when 0μ , this effect is

not clear when 0μ . Another difference is that when 0μ , as 2

εσ increases the

forecasting results improve for both methods.

8

Table 3.1. Forecasts evaluation ( 0μ )

0μ 5.02 εσ 52 εσ 502 εσ

0φ B WB B WB B WB

RMSE a 3.60 2.35 3.56 2.43 3.11 2.80

ME b 0.02 0.01 0.00 0.00 0.18 0.15

% U1 c 1.0 1.0 0.9 0.9 0.6 0.6

% U2 c 98.9 98.7 97.8 96.3 92.6 90.8

% U3 c 0.1 0.3 1.3 2.8 6.8 8.6

TC d 13.15 5.64 12.97 6.13 8.94 7.06

DM e 1.04 0.02 0.94

3.0φ B WB B WB B WB

RMSE a 3.61 2.36 3.65 2.50 2.68 2.46

ME b -0.04 -0.02 0.06 0.04 0.11 0.09

% U1 c 1.1 1.1 0.8 0.8 0.5 0.5

% U2 c 98.8 98.5 97.6 96.0 86.5 83.7

% U3 c 0.2 0.4 1.5 3.2 13.0 15.7

TC d 13.22 5.67 13.70 6.48 6.57 5.42

DM e -1.57 0.29 0.99


RMSE a 3.61 2.36 3.66 2.54 2.68 2.49

ME b 0.06 0.04 0.09 0.06 -0.16 -0.17

% U1 c 1.0 1.0 0.9 0.9 0.8 0.9

% U2 c 98.8 98.5 97.4 95.7 84.5 81.3

% U3 c 0.2 0.5 1.6 3.4 14.7 17.8

TC d 13.23 5.68 13.77 6.72 5.98 4.96

DM e 2.31 0.34 0.42


RMSE a 3.61 2.36 3.80 2.85 3.96 3.97 ME b -0.14 -0.09 0.70 0.50 0.15 0.12

% U1 c 1.2 1.1 3.9 3.5 1.2 0.9

% U2 c 98.6 98.2 94.4 93.3 3.7 7.5

% U3 c 0.3 0.7 1.8 3.2 95.1 91.7

TC d 13.24 5.71 14.81 8.36 2.61 2.70

DM e 4.11 1.74 -1.19

Notes: a RMSE = root mean square error b ME = mean error c Decomposition of the mean square error:

(i) %U1 = percentage of mean error (bias proportion of the MSE).

(ii) %U2 = percentage of regression error (variance proportion of the MSE).

(iii) %U3 = percentage of disturbance error (covariance proportion of the MSE). d TC = Theil coefficient. e DM = results of the Diebold-Mariano test. Statistic uses a NW estimator. Null hypothesis: the

difference between the two competing series is non-significant. A positive sign of the statistic

implies that the Balance has bigger errors, and is worse. When that t-stat is significant, the second

model is statistically better. * Significant at the 5% level.

9


1μ 5.02 εσ 52 εσ 502 εσ

0φ B WB B WB B WB

RMSE a 7.17 4.41 6.67 4.33 2.69 2.44

ME b -6.21 -3.72 -5.62 -3.56 -0.78 -0.66

% U1 c 74.74 70.93 70.68 66.93 10.38 9.47

% U2 c 25.22 28.96 28.91 32.09 74.84 71.63

% U3 c 0.04 0.11 0.41 0.98 14.78 18.90

TC d 51.67 19.58 44.84 18.99 6.13 4.79

DM e 361.33 42.50 4.14


RMSE a 9.51 5.81 8.79 5.69 2.70 2.44

ME b -8.81 -5.30 -8.02 -5.12 -0.84 -0.68

% U1 c 85.79 83.12 83.04 80.37 12.07 10.05

% U2 c 14.19 16.82 16.71 19.04 73.07 70.87

% U3 c 0.02 0.06 0.25 0.59 14.86 19.07

TC d 90.68 33.86 77.65 32.62 6.07 4.74

DM e 440.73 50.71 6.07


RMSE a 15.70 9.63 13.32 8.77 2.89 2.62

ME b -15.30 -9.31 -12.84 -8.40 -1.09 -0.86

% U1 c 94.85 93.48 92.75 91.35 17.55 13.92

% U2 c 5.14 6.49 7.12 8.37 69.96 70.07

% U3 c 0.01 0.03 0.12 0.28 12.49 16.00

TC d 246.87 92.83 177.82 77.25 6.91 5.40

DM e 614.10 68.99 7.50


RMSE a 53.30 39.60 36.39 27.69 4.49 4.64

ME b -53.22 -39.49 -36.26 -27.55 3.34 3.56

% U1 c 99.73 99.44 99.29 99.00 85.35 87.43

% U2 c 0.27 0.56 0.69 0.96 1.56 0.70

% U3 c 0.00 0.00 0.02 0.04 13.09 11.88

TC d 2840.68 1568.23 1324.37 766.85 13.11 14.47

DM e 4070.45 153.88 -9.38








10


1μ 5.02 εσ 52 εσ 502 εσ

0φ B WB B WB B WB

RMSE a 7.15 4.40 6.70 4.35 2.99 2.71

ME b 6.19 3.71 5.64 3.57 0.56 0.45

% U1 c 74.82 71.02 70.24 66.57 4.08 3.38

% U2 c 25.14 28.86 29.33 32.41 86.38 84.70

% U3 c 0.05 0.12 0.43 1.02 9.54 11.93

TC d 51.35 19.46 45.34 19.20 8.22 6.60

DM e 400.45 40.47 4.16


RMSE a 9.55 5.83 8.45 5.47 2.87 2.58

ME b 8.84 5.32 7.67 4.89 1.09 0.87

% U1 c 85.68 82.98 82.00 79.20 17.54 14.62

% U2 c 14.29 16.95 17.74 20.17 69.51 68.55

% U3 c 0.03 0.07 0.26 0.62 12.95 16.83

TC d 91.39 34.13 71.82 30.23 6.93 5.34

DM e 427.04 59.15 8.00


RMSE a 15.70 9.63 13.70 9.02 2.63 2.42

ME b 15.30 9.31 13.24 8.65 0.55 0.41

% U1 c 94.87 93.51 93.23 91.87 5.96 4.35

% U2 c 5.12 6.47 6.66 7.87 77.37 74.73

% U3 c 0.01 0.03 0.11 0.26 16.68 20.91

TC d 246.91 92.79 187.99 81.58 5.38 4.30

DM e 773.48 69.98 4.25


RMSE a 53.12 39.42 38.25 29.27 5.04 5.24

ME b 53.05 39.31 38.13 29.14 -3.64 -3.92

% U1 c 99.73 99.43 99.37 99.07 83.28 85.47

% U2 c 0.27 0.56 0.61 0.89 0.66 0.29

% U3 c 0.00 0.00 0.02 0.04 16.06 14.24

TC d 2822.47 1554.43 1463.55 857.01 15.96 18.00

DM e 3429.23 262.86 -17.21








11

Although it is impossible to completely eliminate the measurement error

introduced when converting qualitative data on the direction of change into quantitative

estimations of agents’ expectations, the weighted balance shows lower measurement

errors and better forecasts. These results suggest that taking into account the percentage

of respondents expecting a variable to remain constant may improve the use of the

balance statistic with forecasting purposes.

4. EMPIRICAL ANALYSIS

In order to evaluate the relative forecasting accuracy of the business tendency

survey measures we use monthly data from the German Manufacturing Business Survey

to track the evolution of the Production Index. We incorporate business survey results in

autoregressive (AR) and Markov switching regime (MKTAR) models, and we compare

the results to those obtained without using business survey information (benchmark

models).

All models are estimated from 1993.01 until 2004.12 and forecasts for 1, 2, 3, 6

and 12 months ahead are computed5. The specification of the models is based on

information up to that date, and then, forecasts are computed up to 2008.12. Forecasts

errors are computed in a recursive way. In order to summarize this information, the root

mean square error (RMSE) is computed.

4.1. Data

Every month the Ifo Institute for Economic Research asks within its Business

Tendency Survey (BTS) a sample of companies in the German manufacturing sector for

their views on some key business variables, such as business expectations and present

situation. The panel involves business officials with a range of specializations within

their companies in management, finance, and other strategic business functions. The

survey participation is absolutely voluntary and derives entirely from the interest in the

5 All calculations are performed with Gauss for Windows 6.0.

12

survey results, as no other compensation is offered. Becker and Wohlrabe (2008)

provide a detailed description of the Ifo Business Survey micro data.

The companies covered by the sample include small, medium and big-size

companies. As BTS are designed to forecast the business cycle in a sector with respect

to output growth, particularly big economic players are involved in the survey. In the

December 2006 survey the overall sample contained 2,622 respondents, which is

approximately 1% of German enterprises in the manufacturing sector.

The dataset of individual responses covers 192 survey waves, beginning with the

January 1993 survey and continuing to the December 2008 survey. The data of the

study comprise a conventional incomplete panel dataset, as the sample changes from

time to time and not every respondent provides an expertise every months. However,

the estimated overlap of respondents who participate in two consecutive survey rounds

lies at around 80% or higher, indicating that the panel is sufficiently stable.

The performance of the manufacturing industry is proxied by the monthly year-

on-year growth rate of the production index (year 2000=100) for the German

manufacturing sector. As business survey respondents are asked to respond to the

questions on business expectations without taking into account differences in the length

of months or seasonal fluctuations, value adjusted for working day variations and

seasonally adjusted production index growth (PI) is used in the paper. Growth rates of

the production index are calculated as percentage change of the index compared to the

same month of the previous year. The German Statistical Office continuously revises

the production indices backward. The data used in the study were retrieved in August

2009.

4.2. Methodology

4.2.1. Benchmark models

In this work two different types of models (AR and MKTAR models) have been

proposed to obtain forecasts for the quantitative variable (Production Index of the

German manufacturing sector) expressed as year-on-year growth rates. As there are few

attempts in the literature to incorporate qualitative information in quantitative

forecasting models, both types of models have also been applied including qualitative

survey data.

13

4.2.1.1. Autoregressions (AR)

Autoregressions explain the behaviour of the endogenous variable as a linear

combination of its own past values:

tptpttt xxxx ...2211 (4)

In order to determine the number of lags that should be included in the model,

we have selected the model with the lowest value of the Akaike Information Criteria

(AIC) considering models with a minimum number of 1 lag up to a maximum of 24

(including all the intermediate lags).

4.2.1.4. Markov switching regime models (MKTAR)

Time series regime-switching models assume that the distribution of the variable

is known conditional on a particular regime or state occurring. Hamilton (1989)

presented the Markov regime-switching model in which the unobserved regime evolves

over time as a first order Markov process.

In this analysis, we use a Markov-switching threshold autoregressive model

(MKTAR) allowing for different regime-dependent intercepts, autoregressive

parameters, and variances. Once we have estimated the probabilities of expansion and

recession using the Hamilton filter together with the smoothing filter of Kim (1994), we

construct the following model for the time series tx using the estimated probabilities of

changing regime:

tt uxLB )·( if PxExpansionP kt / (5)

tt vxL )·( if PxExpansionP kt / (6)

where, tu and tv are white noises, )(LB and )(L are autoregressive polynomials, k

is the value minimizing the sum of squared errors among 1 and 12 and the value P ,

known as threshold, is given by the variation of the probability.

4.2.2. Models where business survey information is incorporated

One way to use the qualitative information of survey data on the direction of

change in order to improve the forecasts of the quantitative variables consists in

introducing qualitative information from business surveys as explanatory variables in

autoregressions. We have followed the same approach by incorporating the judgement

14

about the present business situation and expectations by the end of the next 6 months to

autoregressive (AR) and Markov switching regime (MKTAR) models in the form of

balances (B) and weighted balances (WB).

4.3. Results

The results of our forecasting competition are shown in Tables 4.1 and 4.2. The

tables present the values of the Root of the Mean Squared Error (RMSE) obtained from

recursive forecasts for 1, 2, 3, 6 and 12 months during the period 2004.01-2008.12 for

both, the benchmark models and the models including information from business survey

results6. Table 4.1 shows the results for the present business situation and Table 4.2 for

business expectations.

As expected, forecasts errors increase for longer horizons in all cases. Regarding

the forecast accuracy of the benchmark models compared to the models with business

survey information, results differ depending on whether one incorporates judgements

about the present business situation or expectations for 6 months ahead. In the former

case, while AR models with survey data about the present business situation show lower

RMSE than AR models just for longer horizons (Table 4.1), AR models where business

expectations are incorporated (Table 4.2) outperform benchmark models for all

forecasting horizons.

Table 4.1. Average RMSE - Recursive forecasts from January 2004 to December 2008.

Production Index. Year-on-year growth rate of seasonally adjusted series.

Survey results - Present business situation

Models without survey information 1 month 2 months 3 months 6 months 12 months

AR 2.06 2.54 2.87 3.97 5.63

MKTAR - - - - -

Models with business survey information 1 month 2 months 3 months 6 months 12 months

AR_B 2.29 2.71 3.08 3.92 4.50

AR_WB 2.22 2.69 3.02 3.88 4.19

MKTAR_B - - - - -

MKTAR_WB - - - - -

Notes: a AR_B refers to the AR model with the balance; AR_WB refers to the AR model with the

weighted balance; MKTAR_B refers to the MKTAR model with the balance; MKTAR_WB refers to

the MKTAR model with the weighted balance. b MKTAR models did not converge. c Bold indicates best mode.

6 To check the consistency of the obtained results, we have chosen different time periods for the

forecasting evaluation, obtaining very similar results.

15

Table 4.2. Average RMSE - Recursive forecasts from January 2004 to December 2008.

Production Index. Year-on-year growth rate of seasonally adjusted series.

Survey results – Business expectations

Models without survey information 1 month 2 months 3 months 6 months 12 months

AR 2.06 2.54 2.87 3.97 5.63

MKTAR - - - - -

Models with business survey information 1 month 2 months 3 months 6 months 12 months

AR_B 2.05 2.33 2.53 3.02 3.58

AR_WB 2.15 2.43 2.64 3.00 3.44

MKTAR_B - - - - -

MKTAR_WB 2.35 2.88 3.29 3.86 5.06

Notes: a AR_B refers to the AR model with the balance; AR_WB refers to the AR model with the

weighted balance; MKTAR_B refers to the MKTAR model with the balance; MKTAR_WB refers to

the MKTAR model with the weighted balance. b MKTAR models without business survey information did not converge. Neither did MKTAR models

with the Balance statistic. c Bold indicates best model.

This difference in the results between the two different questions is also

observed when comparing models where business survey information is incorporated.

Thus, in the case of judgements about the present business situation AR models with the

balance statistic (AR_B) are outperformed by those incorporating the weighted balance

(AR_WB) for all forecasting horizons. For business expectations this only happens for

longer horizons (6 and 12 months).

The obtained results permit to conclude that incorporating business survey

information and taking into account the percentage of respondents expecting a variable

to remain constant when assessing survey results improve the forecasting accuracy,

specially for longer horizons (6 and 12 months).

5. CONCLUDING REMARKS

In this work we have presented a different way to assess survey results: a

variation of the balance statistic (weighted balance).This measure allow us to take into

account the percentage of respondents expecting no change in the evolution of an

economic variable.

By means of a simulation experiment we have tested whether this variation of

the balance statistic outperforms the balance statistic in order to track the evolution of

16

agents’ expectations and produces more accurate forecasts of the quantitative variable

generated used as a benchmark. In all cases, the weighted balance outperformed the

balance statistic and provided more accurate forecasts of the quantitative variable

generated as a benchmark.

In order to evaluate the relative forecasting accuracy of the different methods

proposed to assess survey results we have also used the information provided by the Ifo

Institute for Economic Research’s Manufacturing Business Survey to track the

evolution of the Production Index. We have incorporated business survey results in

autoregressive and Markov switching regime models, and we have compared the results

to those obtained without using business survey information (benchmark models). We

have found that models with business survey information outperform benchmark

models.

When comparing the relative forecasting accuracy of the different methods

proposed to assess survey results we have found that taking into account the percentage

of respondents expecting a variable to remain constant improves the forecasting

accuracy of business expectations.

REFERENCES

Anderson, O. (1951). Konjunkturtest und Statistik. Allgemeines Statistical Archives, 35, 209–

220.

Anderson, O. (1952). The business test of the IFO-Institute for economic research, Munich, and

its theoretical model. Revue de l’Institut International de Statistique, 20, 1–17.

Becker, S. O. and Wohlrabe, K. (2008). Micro data at the Ifo Institute for Economic Research -

the “Ifo Business Survey”, usage and access. Journal of Applied Social Science

Studies/Schmollers Jahrbuch, 128, 487–496.

Bruine de Bruin, W., Fischhoff, B., Halpern-Felsher, B. and Millstein, S. (2000). Expressing

epistemic uncertainty: It's a fifty-fifty chance. Organizational Behaviour and Human

Decision Processes, 81, 115–131.

Claveria, O. (2010). Qualitative survey data on expectations. Is there an alternative to the

balance statistic? In A. T. Molnar (Ed.), Economic Forecasting (pp. 181–190).

Hauppauge, NY: Nova Science Publishers.

Claveria, O., Pons, E. and Ramos, R. (2007). Business and consumer expectations and

macroeconomic forecasts. International Journal of Forecasting, 23, 47–69.

Claveria, O., Pons, E. and Suriñach, J. (2006). Quantification of expectations. Are they useful

for forecasting inflation? Economic Issues, 11, 19–38.

Diebold, F. X. and Mariano, R. (1995). Comparing Predictive Accuracy. Journal of Business

and Economic Statistics, 13, 253–263.

Doepke, J. and Fritsche, U. (2006). When do forecasters disagree? An assessment of German

growth and inflation forecast dispersion. International Journal of Forecasting, 22, 125–

135.

17

Guiso, L. and Parigi, G. (1999). Investment and Demand Uncertainty. The Quarterly Journal of

Economics, 114, 185–227.

Hamilton, J. (1989). A new approach to the economic analysis of nonstationary time series and

the business cycle. Econometrica, 57, 357-384.

Kim, C. J. (1994). Dynamic linear models with Markov-Switching. Journal of Econometrics,

60, 1–22.

Knight, F.H. (1921): Risk, Uncertainty, and Profit. Boston, MA: Hart, Schaffner & Marx;

Houghton Mifflin Company.

Lee, K.C. (1994). Formation of price and cost inflation expectations in British manufacturing

industries: a multi-sectoral analysis. Economic Journal, 104, 372–385.

Mitchell, J., Mouratidis, K. and Weale, M. (2005). Uncertainty in UK manufacturing: evidence

from qualitative survey data. Economics Letters, 94, 245–252.

Muth, J. (1961). Rational expectations and the theory of price movements. Econometrica, 29,

315–335.

Ramey, G. and Ramey, V. A. (1995). Cross-country evidence on the link between volatility and

growth. American Economic Review, 85, 1138–1151.

Documents

How can the information content of neutral business survey responses be exploited for forecasting purposes?