71

Carl F. Christ

Carl F. Christ is a professor of economics at The Johns Hop-kins University. Helpful comments on an earlier draft weremade by Jonathan Ahlbrecht, Stephen Slough, Pedro de Limaand William Zame. Any remaining shortcomings are myresponsibility.

i Assessing Applied EconometricResults

T IS A GREAT HONOR to be asked to participatein this conference to celebrate the work of TedBalbach, who has long upheld the standard ofrelevant, independent, intelligible economicstudies at the Federal Reserve Bank of St. Louis.

Mv invitation to this conference asked for aphilosophical paper about good econometricpractice. I have organized my views as follows.Part I of the paper defines the concept of anideal econometric model and argues that to tellwhether a model is ideal, we must test it againstnew data—data that were not available when themodel was formulated. Such testing suggests thateconometric models are not ideal, hut are approxi-mations to a changing realit . Part I closes witha list of desirable properties that we can realisti-calls’ seek in econometric models. Part TI is aloosely connected set of comments and criticismsabout several econometric techniques. Part HIdiscusses methods of evaluating econometricmodels hr means of their forecasts and suinma-rizes sonic results of such evaluations, as proposedin part I. Part IV resurrects an old, plain-vanillaequation relating monetary velocity to an interestrate and tests it with more recent data. The ratherremarkable result is that it still does about aswell today as it did nearly 40 years ago. Part Vis a brief conclusion.

HOW TO RECOGNIZE AN IDEALMODEL IF YOU MEET ONE

The Goal of Research and theConcept of an Ideal Model

The goal of economic research is to improveknowledge and understanding of the economy,either for their own sake, or for practical use.We want to know how to control what is con-trollable, how to adapt to what is uncontrollable,and how to tell which is which. ‘i’he goal ofeconomic research is analogous to the prayer ofAlcoholics Anonymous (I do not suggest thateconomics is exactly like alcoholism)—”God grantme the serenity to accept the things I cannotchange; the courage to change the things I can;and the wisdom to know the difference.”

The goal of applied econometrics is quantitativeknowledge expressed in the form of mathemati-cal equations.

1 invite you to think of an ideal econometricmodel, by which I mean a set of equations, com-plete or incomplete, with numerically estimatedparameters, that describes some interesting set ofpast data, closely hut not perfectly, and that

MARCH/APRIL 1993

72

Figure 1Three Methods of Formulating and Estimating a Model and Checking ItsCorrespondence with 1950-1991 Data

Method 1

Method 2

Method 3

Date of model’s formulation

I_______ Period of data available when model wasformulated

_______ Period of data not yet available whenmodel was formulated

will continue to describe all future data of thattype.

The Need for Testing AgainstNew Data

how can we tell whether we have found anideal econometric model? We czm certainly telllion’ well a model describes a given set of pastdata. (We will discuss what is meant by a gooddescription later). Suppose we have a model in1 992, with estimated pan’meters, that closelydescribes past data for 1950—91 - To tell whether itis the ideal model we seek, we must try it withfuture data. Suppose that after three year’s n’ery the model with data for 1992—94, and it

describes them closely also. Still, in 1995 all wewill he sure of is that it describes data closelyfor a past periocL this time from 1950 through 1994.In principle we can never he sure we have foundan ideal model because there will always bemore future data to come, so we will never heable to say that a mutiel is ideal. ‘the longer thestring of future data that a model describesclosely, however, the more confidence we havein it.

Is t Ii is only a matter of the amount of data thatthe model describes, or is there something elseinvolved? I argue that something else is involved.

Suppose again that ill 1992 we have a modelthat closely describes an interesting data set forthe past period 1950—91. Consider the followingthree methods, shown in figure 1, by which thismodel might have been obtained and by which itsability to describe data for 1950 through 1991 mighthave been assessed:

It was formulated in 1992, and fit ted to datafor the entire period 1950—91 -

2. It was formulated in 1992, fitted to data forthes nb-period 1950—7], and used to predictdata from 1972 through 1991.

3. It was formulated in 197.2, fitted to data forthe. sub-period 1950—71, and used to predictdata from 1972 through 1991.

Methods 1 and 2 differ in that metl iod I fits themodel to a/I the available cia Ia, whereas method 2

fits it to the first part only and uses the resultto predict the second part, from 1 972 onward.1972 is not a randomly chosen date. It was theyea!- before the first oil crisis. Method 3 differs inthat the model builder did not vet know about theoil crisis when formulating the model.

Now consider the following question: Given thegoodness of fit of this model to data for the whole

period 1950—91, does your confidence in the modeldepend on which of these three methods was

Estimation period

1950 1991 1992

Date of model’s formulation

I Estimation period Prediction period1950 1971 1972 1991 1992

Estimation period

Date of model’s formulation

1950 1971 1972 1991—

‘1992

FEDERAL RESERVE BANK OF ST. LOUtS

73

used to obtain it? I argue that it should. In par-ticular, I argue that aci equal ion obtained by amethod similar to method 3, which involvestesting against data that were not available tothe model builder when the model was formulated,desert’es more confidence than the same equationobtained by either of the other two methods.

The argi.mient has to do with the goal of aneconometric model—to describe not only pastdata, but also future data. It is easy to formu-late a model thai. can describe a given set ofpast data perfectly but cannot describe futureobservations at all. Of course, such a researchstrategy should he avoided.

here is a simple example. Imagine a pail- of vari-ables whose relationship we want to describe.Suppose we have two observations on the pail’of variables. Then a line, whose ccl uation is linear,will fit the data perfectly. Now suppose we obtaina third observation. It will almost certainly not lieon the line determined by the first two observations.But a parabola, whose equation is quadra tic (ofdegree 2), will fit the three observations perfectly.Now suppose a lour’th observation becomesavailable. It will almost certainly not lie on the

parabola. But a sort of S-curye, whose equationis cutnc (of degree 3), will fit the four observa-tions perfectly. And so on. In general, a poly-nomial equation of degree ii will fit a set ofn + 1 observations on two variables perfectly,hut a polynomial of higher degree will herequired if the number of observations isincreased. Methods of this type can describeany set of past data perfectlv but almost cer-ainlv cannot describe any tnt w’e data.

If a model is to describe future data, it needsto capture the enduring systematic features ofthe phenomena that are being modeled and itshould amid conforming to accidental lea t ui-esthat will not endure. The trouble wit Ii the exact-fitting polynomial approach just discussed is thatit tloes not try tn distinguish between the enduringsystema tic and the temporary accidental featuresof reality. In the process of fitting past data per-fectlv, this approach neglects to fit endu ringsystematic features even approximately.

This relates to the cli oice among methods 1, 2and 3 for linding a model that describes a bodyof data. When for iuulating a model, resear-cherstypically pay attention to the behavior of avail-able data, which perforce are past data. One tries

different equation forms and different ~.‘ariablesto seewhich formulation best describes the data. This pro-cess has been called data mining. As a method offormulating tentative hypotheses, data mining isfine. l3ut it inyolyes the risk of being too clever,of fitting the a~’ailabledata too well and hence ofchoosing a hypothesis that conforms too muchto the temporary accidental and too little to theenduring systematic features of the observeddata. In this respect it is similar to the exact-fitting polynomial approach described earlier,though not as bad -

the best protection against having done toogood a job of making a model describe p~istdatais to test the model against new data that werenot available when the model was formulated.‘Ibis is what method 3 does, and that is why amodel obtained hr method 3 merits more confi-dence, other things equal.

‘Frygve tIaa~’elmoonce said to me, not enti relyin jest, that what we economists should do isformulate our models, then go fishing [or 50 earsand let new data accumulate, and finally comeback and confront our models with the new data.

Wesley Mitchell put the matter very wellwhen he wrote the following:’

The pi-oposilion mar lie ventured that a competentstatistician, with sufficient clerical assistance aridtime at his comma rid, can take almost any pailof time series for a girt-rn penod and work theminto forms which will \‘ield coefficients of cor-relation exr.eeding ±.9. tt has tong teen knownthat a mathemancian can fit a cur-ye to any timeseries which will pass through evers’ point oftI ie data. t’ei-for ii ances of the latter- so!-t hayeno significance, however, unless the mathe-niaticallr computed rune continues to agree withthe dl at a when pr-oiectecl herorid the pe i’i nd frn’which it is fitted. So wor’k of the sor’t whichMi’. Karsten and Professoi- Fisher hare shown howto do must he judged. not by the coefficients ofcor’r’elatiou obtained within the periods for, whichthr~y Iiaye nian i11u lated the cIa t a, I) (It hr t tie co-efficients which they get in earlier ni later pe,-iodsto which their formulas may be applied.

h:Iilton Friedman, in his review of Jan ‘I’inbergen’spioneering model of the U.S. economy, referredto Mitchell’s coni client and expressed a similar-idea somewhat differently:2

‘rinher—gens results cannot be judged hv ordinarytests of statistical significance. The r—eason is that

‘See Mitchell (1927). 2See Friedman (1940) and Tinbergen (1939).

MARCH/APRIL 1993

74

the variables with which lie winds up. the parti-culam- series iii ciast iri rig these Va m-iab tt,s, the leadsand lags, and various other aspectsof the eqtianonsbesides the particular values of the parameters(which alone can be testedl by the usual stati-stical technique) have been selected aft er an exten-sire pr-ocess of trial and error because theyyield high coefficients of cor-relation. tinbergenis selcIon, satisfiecl wit Ii a corr-ela tion coefficient lessthan .98. t3u t these atti-active con-elation coeffi-cienIs ci ea te rio presuiup tiom i that the i-eta tionshi psthey describe will hold in the future, the multi-

ple regr’ession equations which yield tlieni ar’esimply tautological reformulations of se/criedeconomic data, ‘I’aken at face value, Tinher-gen’swork ‘‘explains” the er-rors in his data no lesshan their real oioyemen ts.

That last statement can be strengthened. Tinher-gen’s method, which has been the method ofmost model builders ever since, explains what-evet- temporary accidental components thei-emay be in the data (regardless of whether they aremeasurement errors), as well as the enduringcocii ponents -

Most macr’oeconometric models for’mulatedbefore the 1973 oil crisis had rio variables repre-senting the prices and quantities of oil and energy.Most of these models were surprised by the oilcrisis and its aftermath, and most of them made stib-stantial forecast errors thereafter. Many modelsformulated after 1973 pay special attention tooil and energy. Of cour’se many of those modelsprovide better explanations of the post-oil-crisisdata than do models that ignore oil and energy.But my point is different. A model that was for-mulated after the oil crisis ~~as.~specificallydesigned to conform to data during arid aftei thecrisis, and if there are lemporai-y accidental var-iations, the model will conform to them just asmuch as to the systematic variations. Hence thetask of explaining data between the onset of the1972 oil crisis and 1992 is easier for a model thatwas formulated in 1992 than for a model thatwas formulated before the crisis. Therefore ifboth models do equally well at describing data from1950 to 1991, the one formulated before the crisishas passed a stricter test and nierits more con-fidence.

What about the relative merits of methodsI arid 2? Sometimes method 2 is recommended;that is, it is recommended that iesearchers esti-matea model using only the earlier part of thea~’ailabIedata and use the later part as a test ofthe model’s forecasting ability. When thinkingabout this proposal, consider a model that has

been formulated with access to all of the data.It does not make much difference whether partof the data is excluded fr’om the estimation pro-cess and used as a test of that model, as inmethod 2, or whether it is included, as inmethod 1. Either way, we draw the same con-clusions. If the model with a set of constantcoefficients describes both pam-ts of the datawell, method 1 will yield a good fit for thewhole period and method 2 will yield a good fitfor the estimation period and small errors forthe forecast period. if the model with a set ofconstant coefficients does not desci-ibe both

parts of the data well, in method I the residuals,if examined carefully, will reveal the flaws, andin method 2 the residuals, the forecast errorsor both will reyeal the flaws. And with bothmethods 1 and 2 we have a risk that the modelwas formulated to conform too much to thetemporary accidental features of the available data.

One noteworthy difference between methods Iand 2 is that if the model’s specification is correct,method 1 will yield more accurate estimates ofthe parameters because it uses a larger sampleand thus has a smaller sampling error.

Econometric Models AreApproximations

When I began work in econometrics, I believeda premise that undem-hies much econometric work—namely, that a true model that governs thebehavior of the economy actually exists, withboth systematic and random components andwith true parameter values. And I believed thatultimately it would he possible to discover thattrue model and estimate its parameter values.My hope was first to find several models thatcould tentatively be accepted as ideal and even-tually to find more general models that wotildinclude particular ideal models as special cases.(One way to top your colleagues is to show thattheix- models are special cases of yours. Nowa-days tins is called “encompassing.’)

Experience suggests that we cannot expect to findideal models of the sort just described. When anestimated econometric model that describes pastdata is extrapolated into the future for’ morethan a year or two, it typically does not hold up well.‘I’o try to understand how tins might happen, letus temporarily adopt the premise that there is atrue model. Of course, we do not know the formor parameters of this true model. They may ormay not he changing, but if they are changingaccording to some rule, then in principle it is

FEDERAL RESERVE BANK OF ST. LOUIS

75

possible to incorpom-ate that male into a moregeneral unchanging true model.

Suppose that an economist has specified a model,which may or may not lie the sameas the truemodel. If the form and pam-ameters of theeconomists model are changing according tosonic rule (not miecessarth’ the samne as the rulegovei-mnng the true model), again in principle itis possible to incorpom’ate that rule into a moregeneral unchanging model.

Now considem- the following possiile ways inwhich the economist’s model might describe pastdata quite well but fail to describe future data:

1. ‘l’he form and parameter values of theeconomist’s model may be correct for boththe past period arid the future period, but asthe forecast hom-izon is lengthened, the fore-casts get worse because the variance of the fore-cast is an increasing function of the length ofthe horizon. This will be discussed later.

2. The for-mn of the economist’s model may he cor-i-eel for- both the past period and the futureperiod, but some or all of the true paramne-ters may change during the future period.

3. The fom-ni of the economist’s niodel may hecorrect for- the past period hut not for thefuture period because of a change in theform of the true model that is not matchedin the economist’s model.

4. The form of the economist’s model may beincorrect for both periods but more nearlycoitect for the past period.

The last possibility is the most likely of thefoum- in view of the fact that the ecomiomy hasmillions of different goods and services producedand consumed by millions of individuals, eachwith distinct cham-acter- tm-ails, desires, knowledgeand beliefs.

These considerations lead to the conjecture thatthe afor-enientioned premise underlying econo-metrics is wrong—that there is no unchanging tiuemodel with true pam-ameter values that governstue behavior of the economy now amid in thefutur-e. Instead, every estimated econometricmodel is at best an appm-oximation of a cbangingeconomy—an appr-oximation that becomes worseas it is applied to events that occur further intothe future from the period in which the modelwas for-mulated. In tIns case we should not besurprised at our failure to find an ideal generalmodel as defined earlier. Instead, we should be

content with models that have at best only atempoi-an’ and approximate validity that deteri-orates with time. We should sometimes also be con-tent with models that describe only a restrictedrange of events—for example, events in a parti-culai- country, industry or- population group.

Desiderata for an EconometricModel

If no ideal model exists, what characteristicscan we r-ealistically strive for in econometricmodels regarded as scientific hypotheses? Thefollowing set of desiderata are withimi meacli:

- ‘l’he estimated model should provide a gooddescription of some interesting set of past data.This means it should have small i-esiduals rela-tit’e to the variation of its variables—that is,high correlation coefficients. The standarderi-ors of its pam-ameter estimates should hesmall relative to those estimates, that is, its1-ratios should be large. If it is estimated for sep-am-ate subsets of the available data, all those esti-mates should agree with each other. Finally,its residuals should appear random. (tf theresiduals appeal- to behave systeruatically, itis desirable to try to find variables to explainthem.)

2. The model should be testable against data thatwere not used to estimate it and against datathat wer-e riot a~’ailablewbemi it was specified.

3. ‘l’he estimated mnodel should be able to describeevent.s occurring after- it was formulated andestimated, at least for a few quarters or years.

4. The model should make sense in the light ofout knowledge of the economy. This meansin part that it should not generate negativevalues for variables that must he non-negative(sur.h as interest rates) amid that it should beconsistent with theoretical propositiomis aboutthe economy that we think are correct.

5. Othem’ things equal, a simple model is prefer-able to a complex one.

7. Othem- things equal, a model that incorpom-atesother useful models as special cases is prefer-able to one that does not. (This is almost thesame point as the previous one.)

6. Other thingswide varietyexplains only

equal, a model that explains aof data is prefenthilc~to one thata narrow range of data.

MARCH/APRIL 1993

76

In offering these desiderata, I assume that thepuipose of a model is to state a hypothesis thatdescribes ati interesting set of available data andthat may possibly dlesctihe new data as well. Ofcourse, if the purpose is to test a theory that weare not sure about, the model should be constructedin such a way that estimates of its par’ameterswill tell us something ahotmt the validity of thattheory. The failure of such a model to satisfythese desiderata may tell us that the theory itembodies is false. This too is useful knowledge.

COMMENTS AND CRITICISMSABOUT ECONOMETRIC

TECHNIQUES

Theory vs. Empiricism

Two general approaches to formulating a modelexist. One is to consult economic theory. The otheris to look for regularities in the data. Either canbe usedh ~rsa staitimig point, but a comnbinatiomiof both is best - A model derived fmoni eleganteconomic theory may be appealing, hut unlessat least sonic of its compomients or imnphicationsare consistent with real data, it is not a reliablehypothesis. A model obtained by pure data mtiimi-imig may he consistent with the body of datathat was mined to get it, but it is not a m-eliablelwpothesis if it is not consistent with at least somneother data (recall what was said about this earlier),and it will not he understood if no them-v toexplain it exists.

The VAR Approach

Vector autoregression (VAR) is one way oflooking for regularities in data. In VAR, a set ofobsem-vahie yariables is chosen, a maximum laglength is chosen, and the current value of cactiyariable is regressed on the lagged values ofthat van-iable and all other ~‘am-iables.No exogenousvariables exist; all obseryable variables aretm-eated as emidogenous. Except for that, a ~‘ARmodel is sinnlar to the unrestricted reducedformn of a conventional econometric model. Eachequation contains only one current endogenousvariable, each equation is just identified, and nouse is made of any possible theoretical infon-rnatiomiabout possible simultaneous structural equationsthat might conitain more than one currentendogemious variable. In fact, no use is made ofany theoretical informnation at all, except in thechoice of the list of variables to be included amidthe length of the lags. In macroeconomics it is

not practical to use mans’ variables and lags in aVAR because the number of coefficients td) heestimated in each equation is the product of thenumnber of variables times the number of lagsand because one cannot estimate an equationthat has more coefficients than there are obser-yations in the sample.

The ARIMA Approach

The Box-Jenkins type of time-series analysis isanother way to seek regularities in data. Hereeach observable variable is expressed in terms ofpurely random disturbamices. This can he done withone y~u-iableat a timne or in a niultivariate fashion.In the univai-iate case an expression involvingcurrent and lagged i’alues of an ohsem-~’ahIevariable is equated to an expressiomi involvingcurmenit and lagged values of an unobservablewhite-noise distur’hance; that is, a serially inde-pendent random disturbance that has a meamiof zero and constant variance. Such a formulatiomiis called an autoregressive integrated moving aver-age (ARIMA) process. The autoregressive partexpresses the current value of the variable as afunction of its lagged values. The integrated par-trefers to the possibility that the first (or higher-order) differences of the variable, rather thamiits levels, may he govet-ned by the equation.‘I’hen the variable’s levels can he obtained fromits differences b~’undoing the difleremicimigoperation—that is, by integrating first differ-ences once, integrating second differencestwice, and 50 on. (If no integm-atiomi is inyolved,the process is called ARMA instead of ARIMA.)The moving average part expresse.s the equa-tion’s disturbance as a mnovimig average of cur-t-ent and lagged vah.mes of a wbnte-noise disturbance.To express a variable in ARIMA form, it isnecessary to choose three integers to character-ize the process. One gi~’esthie order of the auto-regression (that is, the numnher of lags to beincluded for the observable yariable); one givesthe order of the moving average (that is, thenumber of lags included foi the white-noise dis-turbance); and one gives the order of integration(that is, the niumher of timnes the highest-orderdifferences of the observable variable must heintegrated to obtain its levels). The choice of thethree integers (some of which may he zero) ismnade by examining the time series of data forthe ohsemvahle variable to see what choice bestconforms to the data. After that choice has beenmade, the coefficients in the autoregression andmoving average are estimated. The multivariateform of ARIMA modeling is a generalization of the

FEDERAL RESERVE BANK OF ST. LOUIS

77

univariate form. And, of course, VAR modelingis a special case of multivariate ARhMA modeling.

VAR amid ARIMA models can be useful if theylead to the discovery of regularities in the data.If enduring regularities in the data are discovered,we haye something interesting to try to under-stand and explain. In my view, howeyer, onedisadvamitage of both approaches is that theymake almost no use of any knowledge of thesubject matter being dealt with. To use univari-ate AR1MA on an economnic variable, one needknow nothing about economics. I think ofurnvariate ARIMA as mindless data unning. Touse nnultivamiate ARIMA, one need only make alist of variables to he included and choose therequired tuiree imitegers. To use VAR, one ricedonly make a list of the variables to be includedand choose a maximum lag length. Knowledgeof thie subject the equations deal with can enterinto thie choice of variables to he included.

It may seem that the ARIMA approach and theconventional econiomnetmic model approach areantithetical and inconisistent with each other.Zellner amid Palm-n (1974), howevem-, have pointedout that if a conventional model’s exogemiousyamiables am-c generated by an ARIMA pmocess,the model’s endogenous variables am-c genem-atedthe same way.

General-to-Spee jfic Modeling

Genen-al-to-specific modeling stamts with ani esti-mated equatiomi that contains many yam-iahlesand many lagged values of each. Its appn-oach is topare this genen-al forni dowmi to a more specific fominby omitting lags andl variables that do not con-ti-ihute to the explanatory power of the equation.Much can be said for this technique, hut ofcourse it will not lead to a correct result if thegenera I formn one starts with does not contain

the variables amid the lags that belong in amiequation that is approximately comTect.

The Error Correction Mechanism

The error correction mnechanismn (ECM) providesa way of expressing the rate at wInch a variableni~es towam-d its desired or equilibrium valuewhen it is away from that value. Ecomioniic theoryis at its best whemi derivimig desired or equihibimiumvalues of variables, cithem- static positions ordynamic paths. EcU has so far not been goodat deriving thie path followed by ani ecomiomy thatis out of equilibrium. Error correctioni mnodels areappealinig because the~’permit the natum-e of theequmlibriuni to be specified with the aid of the-

ory hut pemmit the adjustment path to be dIeter-mined largely by data.

Testing Residuals for Randomness

I have already discussed testing residuals forramidomness. If an equation’s residuals appear to fol-low any regtmlar or systematic pattern, this is a sig-mial that there may be some regular or systematicfactor that has not been captured by the formand variables chosen for the equation. In such acase it is desirable to try to modify thie equation’sspecification, either- by including additional vari-ables, by changing the form of the equation, orboth, until the residuals lose their regular orsystemnatic character and appear to be random.

Stationarity

It is often said that the residual of a properlyspecified equation should be stationary, that is,that its mean, vamiance and autocovariancesshould be constant through time. Jiowevem-, foran equation whose variables are growing overtime, such as an aggregate consumption or mnomiey-demnand equation, it would be unreasonable toexpect the vat-iamice of the residual to he constant.That would meami that the con-elation coefficientsfor the equation in successive decadles (or othertime intervals) would approach one. It would bemore reasonable to expect the standard deviationof the residual to gm-ow moughly in pmoportion tothie dependent variable, to one of the indepen-dent variables, or to some combination of thiemn.

The Lucas Critique

Robert Lucas (1976) warned that whemi anestimnated econometm-ic model is used to pm-edictthe effects of changes in government policy yam-iables, the estimated coefficients may ttmmri outwrong arid hence the predictions may’ also turnout wrong. Under ~~‘hatconditions can this beexpected to occur? Lucas says that thus occurswhem’m polic~makersfollow one polic~’ruleduring the estimation period and begin to followa different polic~’male during the prediction pemiod.The reasomi for this, he argues, is that in manycases the parameters that were estimated at-cnot constants that represent invariant economicrelationships, hut imistead am-c yam-iables thatchamige imi m-esponse to changes in policy rules.Thus is because they depend hiotbi omi constammIparamneters and on varyimig expectations thatprivate agenits formulate by observing polici-nuakers andi trying to discover- wImat policy maleis being followed. Jacob Marschak (1953) fore-shadowed tIns idea ivhen he cautiomied that

MARCH/APRIL 1993

78

predictions madle froni an estimated economnetnicmodel will not be valid if the structure of themodel (that is, its ma thiernatical form amid itsparameter values) changes between time estimationpemiod arid the predictiomi penio~1-‘therefore, tomnake successful predictiomis after a structuralchange, one must discover the nature of the5 tm-uct umal change and allow for it.

t take this warmung senioush . It nieed riot con-cem-n us whemi policy variations whose effectswe wamit to predhict are similar to variations thatoccurred during the estimniation Iieniod. But whena change mi the policy rule occurs, pm-ivate agemitswill eventually discover that tbieir pm-e~’uJu~expectation fommationprocess is no longer valid amidwill adopt a new one as quickl~’as they cami. As the~’do so, sonic of the estimated parameters willchange amid make the previously obtained esti-niates unirehiahile.

Goodhart’s Law

Lucas’ warning is melated to Goodbian-t’s Law,winch states that as soon as policyrnakem-s hegimito act as if sonic pm-eviouslv observed relation-ship is rehiahile, it will no longer be reliable andwill change.i A striking example is the short-run, dlownward.sloping Pbullips cut-ye.

Are Policy Variables Exogenous?

Most economnet i-ic models treat at least somepolicy variables as exogetious. Btmt puhhic policym-esponds to events. Pohic~’variables are notexogenous. ‘The field of public choice studiesthe actions of policymakers, treatimig tbienn asmaximizers of their owmi utility subject to the con-stm-ainits they face, Ecomiometnic model hiuilders haveso far niot made much use of public choice eco-nomnics.

BY THEIR FORECASTS YE SHALLKNOW THEM (MODELS, THAT IS)

Methods of Evaluating Models’Forecasts

A conventiomiah econonietric mnodhel comitains dis-tum-hiamices andl eridogenous and exogenous ~‘ania-bles. ‘1 ‘vpically sot-ne of tbie endogenonms variablesappear witbi a lag. Consider an annual model withdata for- all variables up to amid including 1 992.

Suppose thnat at the end of t992 we wish toforecast thie endogenious variables for 1993, one

year ahead. ‘Ihis is ami cx ante forecast. For thuswe riced estimates of the model’s parameters,which can he computed fm-omn our availabledata. In addition, we need 1993 values for thelagged emidogenonms variables. Thiese we almeadybiaye hiecause we have values for the years 1992and earlier. Further, we need pnedlictedl 1993values for the disturbances. We usually use zeroshere because disturhamices are asstmmedl to beserially independent with zero means. (Sonicmnodelers, biowevem-, would use values related totIne residuals for 1992 and possibly earlier yearsif the disturbances were thought to he seriallycorrelatedl.) Finally, we need! predicted 1993values for’ the exogenous variables. These pm-c-dictions must he obtained from some sourceoutside the modlel.

Our pn-edlictions of the endogenous van-iabhesfor 1993 will he conditional on our estimatedlmodel and on out- pmedictions of the disturbamicesamid exogenous variables. If we make errors imifom-ecasting the endlogenous variables, it may hebecause our estimated mnodel is wm-ong, becauseour’ pm-edictiom’ns of the disturbances om exogenom.msvariables are wm-omig, or because of sot-ne combi-nation of these.

It is possible—amid desirable—to test the fore-castinig ability’ of an estimated! mnodlel independentlyof tIme model user’s ability to forecast exogenousvaniabiles. ‘[bus is clone with an cx post forecast.An cx post forecast for’ one pemiotl ahead, sayfor 1993, is mad. as follows: Wait until actual1993 data for thie exogenous variables are avail-able, use them-n instead of predhictedl values ofthe exogenous variables to compute forecasts ofthe 1993 enidlogenious vam-i~rbles,amid examine theem-m-ors of those fomecasts.

When comparing forecasts froni (imffement mod-els, beam’ in mnimid that die models may differ in their-lists of exogenous vam-iables and that this may affectthe comiapam-ison. I-or examnple, a model that hashard-to-fon-ecasl exogenous variables is not goingto he helpful tom’ practical cx ante forecastimig,even if it makes excellent cx post forecasts.

Ernor’s of cx ante and ey post forecasts tell usuhifferent things. Lx ante forecasting errors tellus about the quality dif tmue forecasts hut do notallow tms to sehiarate the effects of incorrectestmmnated models from the effects of had pr-edhctionsof exogenous variables and dhisturbanices. ~x postforecasting errors tell us how goodl an estiniatedlmodel has been as a scientific hypothesis, which is

‘See Goodhart (1981).

FEDERAL RESERVE BANK OF ST. LOWS

79

distinct from anyone’s ahiilitv to forecast exogenousvariables audI disturhances. If you are initerestedin the qualit~’of pm-actical forecasting, you shouldevaluate cx ante forecasts. If you are interestedin thie quality of a model as a scientific theory,you should evaluate cx post forecasts. fix postforecasts are usually more accurate thiarm cx antefon-ecasts because the pn-edhictionis of the exogenousvam-iahles that go into cx ante fon-ecasts areusually at least somewhat wrong.

What if we want to make forecasts two yearsahead, for 1994, based on dhata up to and including1992? We need 1993 values fom- the endogenousvaniahles to use as lagged endogemious values fom our1994 forecast; however, we do not have actual 1993data. 1-lence we must make a one-year--ahead fome-cast for 1993 as hefore. Then we can make our1 994 forecast using our 1993 forecasts as thelagged values of the eridhogemiotis ~‘amiahles fon1994. ‘I’hus the errors of onmr 1994 forecast willdepend partly on the em’roms of oum 1993 fore-cast and partly on the values we use for the 1994exogenous variables and distum-hanices. If we wantto mnake forecasts fom mi years ahead instead oftwo veai-s ahead, the situatiomi is simimilam- excepttI-mat n steps are m-edhuiredl inisteatl of two. We canstill comisidher either cx ante or cx post forecasts.As before, cx jios! forecasts use actual vahnmes ofthe exogenous vaniahiles.

%%‘hieni m-naking cx ante forecasts, the tvpicah

economnetric forecaster does riot automaticalhyadopt the forecasts generated by a model.Iris tead the foreca stem- comupa i-es these fonecastswith his subjective judgnmenit about tbie future ofthe economy, amid if tI-mere are substantial dis-cn-epanicies, lie makes subjective. adljustnienits tohis niodel’s forecasts. This is usually’ done withsuhjective adjustmnenits to the predicted distur-bances. Thus the accumacy of cx ante forecasts

typically depends not only on tI-me adequacy oftbie estimated nimodel, but also on the modelbuilder’s ability to forecast exogenous variablesand to make subjective adjustments to the mod-el’s forecasts. Paul Samuelson once caricaturedthis situation at a meeting some years ago bylikenung the process that pn-oduces cx anteeconomnetnic for-ecasts to a black box insidewhich we find only Lawrence R. Klein!

Errors of Forecasts from SeveralEconometric Models

Most presemitations of forecasting acd:um-acy an-cbased omi cx ante ratbier than cx post forecasts,often with suhjective adjustments, perhaps becausedif the interest mi practical forecasting. I like tolook at cx post forecast errors without adjust-nments because 1 am initem-estedi in economneti-icmodlels as scientific hypotheses.

Fromnm and Klein (1976) and Christ (19751discuss root mean square errors (RIVISEs) ofcx post quarterly fom-ecasts of real GNP, nominialGNP andl the GNP deflator one quan-ter to eightqua rtem-s ahead by eight models with no subjec-tive adjustment by the fom-ecaster. ‘the modelswere formulatedh by Brookings, the 11.5. Bureauof Ecomiomic Analysis, Ray Fair, Leonall Ariden-sen of the federal Reserve Banik of St. Lonmis, T.C. Liu ann] others, the Umnversity of Michigan;tmid tbie Vvhartoni School (two versions). ForGNP they show RMSEs risimig from-n 0.7 pm-centto 2.5 or 4.5 hiercenit of the actual value as thehorizon increases from-n one quarter to eightquarters. F’or the GNP dleflator they showRMSEs mising fromu 0.4 pemtent to 1 .9 percent,as shiowmi in table I -

In a series of papers ovem tI-ne past seven-alyears, Stephemi McNees (1986, 1988 and 1990)bias reported on the accuracy of subjectively

MARCH/APRL 1993

80

adjusted cx arm Ic qua rte rl~’forecasts of severalmacroecormonnetric models, for horizons of omnetd) eight dhuarters ahead, amidh has compam-edthem with two simple mnechamucal fored:astimigmethods. One is the univam-iate AtTIMA methodof Charles Nelsomi (1984), which is called BMABK(for bemmchmimark). ‘lhe ditbier is the Bayesian i’ec-ton auton-egm’essiomi method] of Robert Littermnami(1986), which is called BVAR. The models dns-d:ussed imi mMcNees 11988) am-c those formtmhatedby the U.S. Bureau of Economic Analysis, ChaseEconomet m-ics, Data Resources Inic., GeorgiaState University, Kent Institute, the Universityof Michigan, UCLA and Wharton.

McNees’ resumlts for quarterly forecasts may’ hesummniarizedl in the follonvirig five statements:

1. ‘the mnodlels’ forecast emmors were usuallysmaller than those of BMARK.4

2. The modhehs’ forecast erm-ors were usually slightlysmnaller than those of BVAB for noniiniah GNPanid most other variables and slightly largerthan those of BVAR for real (‘INP. Thus BVARwas usually better than BMARK for real GNP.’

3. Forecast errom-s for the levels of variablesbecamne worse as the forecast horizon lemigthi-enedl fmomn one dluantem- to eigbit diumarters,roughly qtmadruphimig fom- most yam-iahles andimicneasing tenfold for prices. However, fore-cast errors for the growth rates of nman vari-ables (hut not for price vam-iahles) impm-ovedas the horizon lengthemmedh. In other words,for mmianv vam-iahles, the forecasts fom- gmowtbirates averaged over several quarters werebetter tI-ian the forecasts for short-termin fhuc-tuatiorns.

4 Meami absolute errors (MAEs) of the models’

fomecasts of tIme level of nominal GNP n-em-cusimallv about 0.8 percent of the true level forforecasts one qnmartem- ahead and1 increasedhgm-aduahlv to about 2.2 percemit for’ fomecastsone y’eam- abiead and about 4 percemit fom- fore-casts two years ahead. Real GNP forecastem-rors were sonuewhat smuialler. Errors forothem vam-iables ivem-e comparable. Price-levelforecast em-mrs n-en-c smaller fom- thirn one-quarter horizomi hut grew faster andl werelamger for the two-mean- horizon.’

5. When sub jectivelv adhjustedh forecasts wereconipared with umiadjusted forecasts, theanljustmenits were helpful in muost cases,though sometimes they madhe the forecastworse. Usually’ the adjustmnents were largerthan op tinial.s

One-year-ahead annual forecasts of n-cal GNPby tine Umiiversity of Michigami’s Research Centerin Ouanititative bconomuics, by tbie Councih ofEcomidimic Advisers and liv private forecasterscovered hi’ the ASA/NBER survey all had MAEsof about 0.9 percent to 1.1 perd:ent of the trumelevel, arid RMSEs of about t.2 percent to 1.5

percent of tIme true level.’ (The relative sizes ofthe MAEs amid RMSEs are rougb’nly consistentwith the fact tbiat for a nom-mal distrihutiomi, theRMSE is about 1.25 times the MAE.)

Implications of Worsening Ex PostForecast Errors

Because the root mean sqtmane em’mor of anecomiometnic model’s cx post forecasts roughlyquadruples when the hom-izoni increases fnonione quarter to eight quarters as in table 1, canwe conclude thiat the model is no lonigen correctfor the forecast pemiod? The answem- is possibly,hut riot certainly’.

F’or a static model we could comicludhe this becausethe error of eacbi forecast would inivolve distur-bances only for the period tieinig forecast, riotfor periods in tbie earlier part of the horizon.Hemice theme is no reason to expect great cbiarigesinn the size of the forecasting error for a staticmodel as the homizon imicreases. Small increaseswill occnnr because of ern-ors in the estimates ofthe models paramneters if the values of the mod-el’s independent vamiabiles move furtbien away’fn’omnn their estimation-period meanis as the horn-zomi lengthens. Tbus is because any em-rors in theestimates of equationis’ slopes will gemieratelarger effects as the distance over which theslopes are projected imncreases.

But m-nost econonietm-ic fomecasting mnodlels con-tain lagged endogenous variables. ‘I’herefome, asnoted previously, to forecast n periods ahead,we nnmst first fom-ecast the laggedh endlogenoims-vaniahle values tbiat are neededh for the ni-pem-inds-

4See McNees (1988 and 1990).

5See McNees (1990).

8See MeNees (1988).

‘See McNees (1988).tSee McNees (1990).

tmSee McNees (1988).

FEDERAL RESERVE BANK OF ST. LOWS

81

ahieanh forecast. Thus imivolves a drain of ni steps.‘l’he finst step is a forecast one pem-iod ahead],whose err-on- involves disturbances only fm-omnmthe first lieriodh in the n-per-iod horizon. Thesecondl step is a forecast two pe.riods ahead,wI-nose emmnim involves distum’banices fm-omn the sec-ond period in the horizomi and also disturbancesfmoni the first period because they’ affect theone-pem-iod-ahead forecast, whicbi inn tummi affectsthe two-pem-iods-ahead forecast. And Sn) on, untilthe nth step, whose forecast error imivolves dis-turbances fmoni all periods in the horizon fmomnomie thn-ough mi. Thmus, ton- a dy’mnamic modlel, tbievariance of a forecast n pen-iods ahead will dhependonm the variances and covarrances of disturbancesin all n pem-iods of the hon-izon, amid except invery special circumstanices, it will increase asthe horizon inceases.

To decide whether the evidence in table I showsthat the estimated models it nlescm-ihies are incor-rect for the forecast horizon of eight quarters,we need to kniow whether the RMSEs of a correctmodel would quadhruple as the fom-ecast homizonincreases from-n one quarter to eight quartem-s.If they’ would, then the quadrupling observed inthe table is not evidlence of imicomrectness of theestimnated models. If they would not, then cvi-nlence of immconrectness exists, We do not haveeniougbi imiforniation about tbie models underly-ing tbie table to settle this issue dlefimutively, hutsome sinnphe examples wihl illustrate the principleinvolved.

Suppose the model is hinear anidl pem-fectly cor-rect, and suppose it comitains lags of omme dluarteron- more (as most models do). ‘then the varianceof the error of an ni-pen-kids-ahead fom-ecast willhe a linear combination of the variances and]covariances of the distum-banices in all periods of thehorizon. In the simple case dif a single-eqimatiomimodel, if the disturbances an-c serially indepen-dhent amid if the coefficients mi the linear combi-niationi of chistummhances are all equal to one, thevam-iance dif the linear combinationi of distur-bannces frir a hiomizomi of eight quarters will beeight times tbiat of one quarter. So the RMSE ofcx post forecast emrors from a correct modelwill increase by’ a factor of the square m-oot ofeight (about 2.8) as the hiomizomi goes fmonu omiequamter to eight quarters. If the coefficients mithe linear combination are less tI-man one, as inthe case of a stable model with only one-periodlags, the vam-iamice of the hineam- comhination for

eight quarters will lie less than eight times thatfor’ one quarter. Hence the RMSE of e,x postforecast em-ron-s frdmm a correct model willincrease by’ less thani a factor of the square rootof eight as tbie hom’izori goes from one quarterto eight quarters. In such a case, if the observedRMSEs appm-oximatelv d~uadlmupled,it would castsome douhit on the validity’ of the mnodlel.

Cdinsider a single-equationm model with a singlelag, and no exogenous vam-iahles as follows:

= a + Py,1 + 5

where s is a serially indepenidenit nhisturbance withzem-o mean and constant variance a’. Suppose thatthe values of a and 13 are known and thus rio fon’e-cast errom is attributable to incorrect estimates ofthese coefficients. Then the variance of the errorof a orme-pem-iod-ahead forecast is a’, that of a two-periods-ahead for-ecast is (1 + 13’) a’, that of athree-periods-ahead forecast is (1 + if’ + 135a, andso on. The variance of an mi-periods-ahead forecastis ~ 13” a’, which is equal to (t — ff) a’/tl — 1~’)

Table 2 shows how the stanidandh dleviatiomi ofsuch a forecast error increases as the horizonincreases from one quarter to eight quarters forsevemal values of the parameter /3. Table 2 sug-gests that if the RMSE of a model’s forecastsquadruples as the horizon increases from-n onequarter to eight quamtens, either- J3 (the rate ofapproach of the modhel to equihibniunn) must belarge or chose to one, or the model is inade-quate as a description of the fonecast period.

Corresponding expressions can he derived fommnulti-equation models with many lags ann seriallycdirm-elated distum-bances, but they am-c rathercunihen-some -

AN OLD, PLAIN-VANILLAEQUATION THAT STILL WORKS,

ROUGHLY

Nean-hy 40 yean’s ago Hemiry Allen Latan~pub-hshed a short paper in which he reported thatfor 1919—52 the inverse of the GNP velocity ofMl is described by a sinnple least squan-esm-egnession on the inverse of a lonig-term, high-gn-ade homid rate RL as follows:”

(I) M1/GNP = .100 + .795/RL, F’ = .75(1-ratio) (10)

1mSee Latané (1954).

MARCH/APRIL 1993

82

Here and in what follows, I have expressed inter-est rates in units of percent per year, so a 5 per-cent rate is entered as 5, not as 0.05, and its inverse0.20, riot 20. The Appendix gives the definitionsand data sources for variables in this and sub-sequent equations. Latan~showed the unad-justed correlation coefficient r, but showed nei-ther the standard deviation niom the t-ratio ofthe slope. I calculated the adjusted F’ and [he t-ratio. The latter is the square root of r’ (di) 1(1 —r’),where df, the number of degrees of freedom,equals 32.

This specificationi bias son-ne of the pm-operties ofa theoretical nnorme dlemandl equation—nanuely, apositive income elasticity (restn-icted to he con-stant and equal to omie by’ cdinnstruction) amid anegative interest elasticit~’(restricted to have anabsolute value less than one and ndit constant).But its least-squares estimate \x’onmldl almost cer-taim’mly be biased or inconisistent, even if tIme forrnrof the equatidimi were con-rect, hecause the homid]rate is almost certainly niot exogenous and] henicenot independent of the equation’s disturbances.

Nevem-theless, thus specification has continuedto work fairhy’ well for other periods. Nearly 30years ago Ml/GNP was descrihiedl for- 1892—1959hi’ a siminilar regression on tine inivemse of Moody’sAaa honid rate with alnuost the same coefficients,as follows:’’

(2) M1/GNP = .131 + .7I6iRAaa, F’ = .76(I—ratio) (14)

(3) MI/GNP = .085 + .774/HAaa, F’ = .90(t-ratio) (13) (17)

If GNP in equation (3) is replaced by the newoutput variable GDP for 1959—91, the result isalmost identical, as follows:

(4) MI/GoP = .086 + .771/RAaa, F’ = .91([‘ratio) (13) (18)

David Dickeys discussion is biasedl on the1959—91 dlata that underlie equation (3).

For 1892—1991 a similar result is againobtainenl, as follows:

(5) M1/GNP = .083 + .874/RAaa, F’ = .89(t.ratio) (ii) (28)

Table 3 shows the estimatedl equations Ii) —(5)and] seven-al other estimated equations that willhe described soon. Equations (1’) and (2’) areattempts to duplicate the results in equations (1)and (2) using tbie sam-ne diata base that is used miequations (3), (5) amid later equations. The i\~ipen-dix gives ntata soum’ces.

Figure 2 shows the graphs of M1/GNP and1/RAaa over time. Figures 3 and 4 show thescat ter diagrams for equations (3) and (5),respectively’. (I should add that, of the four

~See Christ (1963).

For 1959—91 the same specification describesthe ratio of Ml to GM’ with almost the samecoefficients, as follows:

FEDERAL RESERVE BANK OF ST. LOUIS

83

~‘ ,~1’ ~N,, , >‘‘~‘,~““N~/N

- ‘-~4~ ~ ‘~ K ‘N’ ~ A’-‘K ,‘ N ‘ N N- N-’’* K”

N “ ‘A N N/ NN N N N

fiSsions *1 MVbNP oê%Sflt$! dwiMAas~idOtheYNVadabIes~N ‘N N

‘.N‘N’ N N ‘, ‘N ‘N EW(S*t~1t~$tan444a$$’ / ~ / N

Nq c~Ø’K WA ‘N ~ ~“N

~ ~‘N ~,, N,,,” / ‘N N’N4K/ ~ ii NON- N A/NA/K ‘7ø5*~/’ N~ ,N’~ y’, ,K’’/,~,;c’N~N’ ;K’ ;A~’ :.

4 N ‘4 / ‘N “AN V ‘N,, N 1~ ~I6’‘N $~ NN N’, ‘N ‘‘N ,, /N,~ ‘NN A A NN

SN ‘tSSGt / A ~ ,,‘,N~ o / / ‘N‘N ‘N ‘N — ~,‘. ~‘~“ c ‘N~N ,,KN ‘N

K~4c,,,*”/~$Str~’’,AW~%, ;, ,:N’ K~’ ~

/ ,,N’N,, ,< ‘N — ,,‘N K - N~ A N

‘N A ‘N~ S6b~4 ‘N ‘N — ‘N “N NN’$3~

( NA ,}14~~4~ 23s/N “N A N ‘SW N NA ~‘“$ØS’ A> ~ ‘N-c’:’ AA,N’J6’ ~

A / ‘N — A_N / ‘N” ‘N A /N”~’N N N ,:‘ ‘N ‘N N

(IØ$041 ,,‘.:&, ‘,>~“ ),, ,/N / “= 2t,~ ,“,. - / , ‘1N ‘N~ N ‘N N N “‘N ‘N Y. /NA

N’. ~ ~‘2f~r~”N’Y’~- ---e’t”-t N

Nfl~fl~C N

;~a DcS~/ , N , ~ ,,, MW”N nprnSc~a1ØSm~4 ttdt K •~ h~A ‘~ N

‘N ,KA’ \“ N ‘N~,,’

N ~ ~ 6’ ~ffiStN ~ N K N N ‘N N

“N:’ - ,/‘‘N , N “N A>’ ‘ ‘NN,N’ ‘‘ ,,K’ ‘N’ - “‘N’, ‘N’

equations that can be obtained by regressinigeither the velocity of Ml om its inverse on eitherRAaa or its invem-se, the fomm that is presented]here fits the best.)

It is rather remarkable that this plain-vanillaspecification continues to descmibe the relationbetween Ml’s velocity and] time long-term Aaahonnd mate with’m such similar- regm’essidin amid cor-relation cdiefficienits for the four pen-iodls, espe-cially’ in view of the changes in interest-rateregulation and in the defimiition of Ml that haveoccurred over the last century. However, thedifferences amiiong the foum- estimated versionsare not negligible, as seeni mi a comparisom’r ofthe computed values of M1/GNP that they yield.For 1959—91 these computed values are shownin figur(, S together with the actual values ofM1/GNP. Note that those computed from equa-tions (1) and (2) using 1919—52 amid 1892—1959data am-c ex post fdimecasts, whereas those fromequations (3) and IS) using 1959—91 and 1892—1991data are within-sample calculated values. Figure 6shows the vahnmes of l~1I IGNP obtainiedl whenequation (3) based omi 1959—91 data is used tohackcast M1/GNP for 1892—1958, and it also

shows the actual values arid tine calculatedvalues from-n equation (5) using 1892—1991 data.Time fom-ecastimig and hackcasting errors are byno means negligible, hut the general pattern ofbehavior of M1/GNP is reproduced.

The estimates of the plain-vanilla equation aren-ather stable, acr-oss time, as indicated by fig-umres 7 amid 8 which show the behavior of theslope as the sample lieriod is gradually length-ened by adding one year at a time. In figure 7the sample period starts with 1959—63 and isextended a year at a tim-ne to 1959—91. In figure8 the sample period starts with 1892—97 and isgradually extended to 1892—1991. In each figurethe slope settles down quickly after- jumnpingam-oundh at fin-st and varies little as the sample isextended thereaftem-.

However, this simple specification does not byann’ means satisfy all of the desiderata listedpreviously’. In particular, the 1959—91 Durbini-Watson statistic is a minuscule 0.38, and the1892—1991 Durhiini-Watsomi statistic of 0.48, is riotmuch better, which suggests that the residualshave a strong positive serial correlation. This byitself would not create bias in the estimates if

MARCH/APRIL 1993

Figure 2M1/GNP and 1/RAaa, 1892-91

0.6

0.5

0.4

0.3

0.2

0.1

0

Figure 3

-‘[__

0.05 0.1 0.15 0.21/RAaa

0.25

84

1900 10 20 30 40 50 60 70 80 1990

Regression of M1/GNP on 1/RAaa, 1959-91M1/GNP

0.3

025

0.2

0.15

0.1

FEDERAL RESERVE BANK OF ST. LOUIS

85

Figure 4Regression of M1/GNP on 1/RAaa, 1892-91

Ml /GNP0.6

0.5

0.4

03

0.2

0.1

1/AMa

Actual, Computed and Forecast Values of1/RAaa for Four Periods

0.35

0.30 -

0.25 -

0.20-

0.15-

0.10 I I I1959 61 63 65

M1/GNP from Regressions on

0 0.1 02 0.3

Figure 5

0.4

Actual

1959-91

I I I I I I I I I I I I I67 69 71 73 75 77 79 81 83 85 87 89 1991

MARCH/APRIL 1993

86

Figure 6Actual, Computed and Backcast Values of M1/GNP from Regressions on 1/RAaa forTwo Periods

0.3-

Figure 7

1959-91

Estimates of Slope in Regression of M1/GNP on 1/RAaa for SamplesStarting in 1959 and Ending in 1963.~1991

7.5-

5—

23-

0-

-2.5-

-5—

-7.5 -

—10— I I I I I I

1963 65 67 69 71 73 75“1 I I ~F~” ‘ I77 79 81 83 85

I I I87 89 1991

0,5—

0.4-

0.2-

1892-91

‘/

0.1— I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I

18929598 1 4 7 101316 1922252831 3437404346 4952555861 6467707376 798285881991

+ 2 Standard Error

/ -2 Standard Error

Estimates of Slope

FEDERAL RESERVE BANK OF ST. LOUIS

87

Figure 8Estimates of Slope in Regression of M1/GNP on 1/RAaa for SamplesStarting in 1892 and Ending in 1897...1 991

2

1.5

-0.5-

—1 —

I I I I I I I I I I1900 10 20 30 40 50 60 70 80 90

the edluation form were correct and if the dis-turbanice were independent of the inten-est rateand had zero mean and constant variance. Butit certainl suggests strongly that the equationhas not captured all its relevanit systematic fac-tors. The graph of the residuals of the 1959—91equation (3) against tin-ne is illuminating. It showsan almost perfect 12-year cycle of diminishingamplitude witl’n peaks (positive residuals) in 1959(om possibly earlier), 1970 and 1982 and tnoughs(negative residuals) mi 1965, 1977 and 1990. It alsosuggests a negative time tnenid. The residuals ofthe 1892—1991 equation (5) show a roughly sinn-ilzu’ pattern. (See figures 9 anid 10.)

The very low Durhin-Watson statistics suggestthat the equation should lie estimated eitherusing the first differenices of its variables, orbetter, using the levels of its variables with afirst-order autonegressive IAB(1)] comm-ectionapplied to its m-esiduals. Estimation mi levels withan AR(1) correction would he appropriate if thedisturbance u in the original equation wereequal to its own lagged value times a constant,p, plus a serially independent distum-bance, E,

with constant variance, as follows:

(6) u = pu, +

In this case, if the original equation is

(7)y, = a + fix + o, = a + fix, + pu, +

the AR(1) correctioni subtracts p times thelagged version of equation (7) fnom equatiomi (7)itself and produces the followinig equation:

(8)y, = py, + (1 — p) a + (ix — (ipx, , +

This equation is nonlinear in the parametersbecause the coefficient of lagged x, —fip, is thenegative of the pmxiduct of the coefficients of xand lagged y. If that m-estriction is ignored andthe coefficient of lagged x is denoted by y, theequation becomes as follows:

(9) y, = py,~, + (1 — P) a + fix, + yx +

‘Ihis equation can lie given the following errorcorrection interpretation. Suppose that the equil-ilirium value y* of a dependent variable y is linearin an independent variable x, as follows:

(10) y~ = a + fix,

and that the change in y depends on both thechange in the equilibrium value and an error

1—

0.5-

0-

Standard Error

(/:2 S;andard Erwr S Estima;es of Slope

•1.5

MARCH/APRIL 1993

88

Figure 9Residuals from Regression of M1/GNP on 1/RAaa, 1959-91

0.04

0.03

0.02

00l

-0.01

-0.02

-0.03

Figure 10Residuals from Regression of M1/GNP on 1/RAaa, 1892-1 991

1959 61 63 65 67 69 71 73 75 77 79 81 83 85 87 89 1991

1900 10 20 30 40 50 60 70 80 90

FEDERAL RESERVE BANK OF ST. LOUIS

89

correction term proportiomial to the gap betweenthe lagged equilibrium and the lagged actualvalues, as follows:

(11) ày, = OAy,* + (1 — p) (y* — y,_,) + ,

Substitution from edluation (10) into edluation(It) implies ani equation with the same variablesas the ABA 1) equation (8) hut with some differentparanietems, as follows:

(12) ~ = py, + (1 — p)a + Ofix, + (1 — p — 0)/h,, +

If the adjustment parameter 0 in equation (12)were equal to one, then equation (12) wouldbecome the same equatidin as (8).

Estimates in first differences would he appro-priate if the value of p in equation (6), (7) and(8) were one. In this case, edluation (8) becomesa first-dlifference equationi, as follows:

(13) ày, = flAx, +

The least-squares estimate of equation (8) milevels with the AR(1) correction for 1960—91 isas follows:

(14) M1/GNP — .896(MVGNP) -

= (1 — .896)126 + .267(1!RAaa — .896/RAaa,)(t-natio) (8) (2.8) (26)

with an adliusted I-I squamecl of .98 and 13Wequal to 1.82. This is equivalent to the followingedluatiomi:

(15) M1/GNP = .896(M1/GNP)

+ .013 + .267/RAaa — .239IRAaa,

Theme is no evidence of a trenid.

The least-squares estimate in levels wit Ii theAWl) correction for 1893—1991 is as follows:

(16) MIIGNP — .831(M1/GNPL,=0 — .831)117 + .7t1(1/IlAaa — .831!RAaa_,)

ft-rat io) (5) (7) (12)

with ami adjusted R squared of .95 and OWequal to 1.60. This is equivalent to the followingequation:

(17) M1/CNP = .831(MI/GNP)+ .020 + .711/RAaa — .591/FtAaa_,

There is again no evidence of a tn-end.

Least-squares estimation of the ECM equation(12) for 1960—91 (without restricting 0 to be onie)yields the following equation:

(18) M1/GNP = .857(M1/GNP),

(t-ratio) (1 1)+ .016 + .275!RAaa — .212/Ri~aa,

(2.2) (2.8) (—2.2)

with an adjusted R squared of .98 and 13Wequal to 1.78. This is quite close to the AR(t)nesult in equation (15), nTlnch suggests that theadjustment coefficient 0 in equatiomi (12) is notvery different froni one. The hypothesis that inequation (18) the coefficient of lagged l/RAaa isequal to the negative of the product of thecoefficients of 1/RAaa amid lagged M1/GNP, asnequired by equation (8) and as satisfied byedluation (15), is strongly accepted by a Waldtest (the p-value is .59).

Least-squares estimation of equation (12) for1893—1991 (again without restricting 0 to he one)yields the following equation:

(19) M1/GNP = .807(M1,’GNI’)(t-ratio) (12)+ .016 + .593/RAaa — .428/RAaa,

(2.1) (5) (—3.6)

with an adjusted H squared of .95 and OWequal to 1.59. This is quite close to the AR(1)result in equation (17), which again suggeststhat the adjustment coefficient 0 in edluation(12) is not very diffem-ent from one. The hypothesisthat mi equation (19) the coefficient of laggedl1/RAaa is equal to the negative of the productof the coefficiemits of 1/HAaa and lagged M1/GNP,as required by equation (8) and as satisfied byequatiomi (17), is accepted by a Wald test (thep-value is .11).

Equations (iS), (17), (18) and (19) are betterthan the plain-vanilla equations (3) anid (5) insome nespects, anidi worse in others. The~’havesubstanitially higher ad)justed H-squared values,nnnch less serial correlation in their residltlals,no evidence of a tim-ne trend, and significantcoefficients. The ECM equations (18) and (19),however, are vem-y unstable (ivem’ time. Imi equa-tion (18) the coefficient of 1/BAaa vamies fromabout .6 for 1960—70, to .05 for 1960—78 and1960—81, to .3 for 1960—86 and 1960—91. Inequatiomi (19) the coefficient of 1/RAaa variesalmost as much hut remains at about .7 or .6 forsamples that include at least the years 1893—1950.

MARCH/APRIL 1993

90

I comijecture that in the AWl) edluations (15) and(17) the coefficient of 1/RAaa is also unstalileacross time because the AHO) anid ECM equa-tiomi estimates are quite similam-.

By comparing equations (12) and (18), one cansolve for the 1960—91 estimates of the fourparameters p, a, (I and 0, in that order, toobtain:

(20)~=.857,=.l12,/3=.441andO=.624

This imnplies that the equilibrium relation inequation (10) embedlded mi the ECM is astollows:

(21) (Mi/GNP)* = .112 + .441/RAaa

Similarly, by coniparing equations (12) and (19)one can solve for the 1893—1991 estimates ofthe fonnr parameters as follows:

(22) ~ = .807, & = .083, /1 = .855 and S = .694

This implies that the equilibrium relation inedluation (10) ennhedded mi the ECM is asfdillows:

(23) (Mi/GNp)* = .083 + .855/HAaa

The two equilibrium relations in equations (21)and (23) for the two periods 1960—91 and1893—1991 are quite different, which is consis-tent with the instability of the ECM specificatidinacross time.

Now let us return to the first-differenice equation(13). The least-squares estimate for 1960—91 isas follows:

(24) AC 1/GNP) = .380A(1/HAaa), = .05(t-ratio) (3.6)

(25) A(M1/GNP) = .494A(1!RAaa), i2 = .15(1-ratio) (4.1)

iyith OW = 1.76. Table 4 shows the estimatedequations (24) and (25). The estimnates of thisfirst-difference specification are not quite asstable acm-oss tinne as those of the specificationiin levels of the variables. This can be seen bycomparing equations (24) and (25) and also fronnfigures 11 and 12, which show the values mif theestinnates as the sample is increased one year- ata time, starting respectively with 1960 anid 1893.In each figure the estimates stabilize, after aninitial period of instability, but the values atwhich they settle differ by a factor of about .75.

If a constant term is included mi equationi (24),which implies a trend term in equation (3), theconstant is small hut significantly negative, theslope falls to about .3,and the adjusted H-squared and OW values improve slightly. ‘I’heestimated slope, however, heconnes wildly unsta-ble across tinie. If a trend variable is includedin equation (3), its coefficient is smnall but signifi-canitly negative, the interest-rate coefficient fallsto .49 and remains highly significanit, theadljusted H-squaned and the OW values riseslightly, and againi the estimated slope is wildlyunstable, across time.

If a constant tem-m is included mi equation (25),it is small amid insignificantly negative, the restof the equation is almost unchanged, and theslope becomes quite unstable through time,varying from .6 to zero and back to .6 again.if a trend is included in equation (5), its coeffi-cient is small hut significantly negative, theinterest-rate coefficient is almost unchanged at.81, the adijusted H-squared value rises a hit, theOW value rises a hit, and the coefficient isagain wildly unstable across time.

with OW = 1.23. For 1893—1991 it is as follows:

FEDERAL RESERVE BANK OF ST. LOUIS

91

Figure 11Estimates of Slope in Regression of A(M1/GNP) on A(1/RAaa) forSamples Starting in 1960 and Ending in 1962...1991

10-

5- n,.

- \+2 Standard Error

0- ‘ -~‘ •

- 2 Standard Error

-5-

-10— I I I I I I I I I I I I1962 64 66 68 70 72 74 76 78 80 82 84 86 88 1990

Figure 12Estimates of Slope in Regression of A(M1/GNP) on A(1/RAaa) forSamples Starting in 1893 and Ending in 1896...1991

3—

2—+2 Standard Error

1— a...

Estimates of Slope • - -

-1- -2 Standard Error

-2—

I I I I I I I I1900 10 20 30 40 50 60 70 80 90

MARCH/APRIL 1993

92

On the whole, the first-difference specificationdoes not stand up well.

Where do mnatten-s stand? On the one hand,we liax’e the plain-~’amiillaequatiomi such as equa-tion (3), whichi fits only mnoderately well and hassevere serial correlation in its r-esiduals hut hasan estimated slope that is rather stable acrosstime. On the other hand, we have more compli-cated) dvnamnic equations such as the ECM edjua-tion (18), which fit much better and have niceDurbin-Watson statistics hut have estimatedcoefficients that vary greatly across time. Nei-ther is quite satisfactory, hut if the aim is tofind an estimated equatidin that will describe thefuture as well as it does the past, I think t wouldnow bet oni the plaimi-vanilla specification, eventhough the relation of its estimated coefficientsto structural parameters is umiclear.

CONCLUSION

Econometrics has given us somne results thatappear to stand up well over time. The priceandi incdimne elasticities of demanid for farm

lirodurts are less than one. The income elastic-its’ of household demnand for food is less thanone. iIouthakker (1957), in a paper com-memorating the 100th anniversary of Engel’slaw, reports that for 17 countries and severaldilferent periods these imiconne elasticities rangehetweeni .43 and .73. Rapid inflation is associatedwith a high growth rate of the money stock.Sdime shom-t-term macroeconometric forecasts,especially those of ttie Michigan model, at-cquite good.

Rut there have also been some nasty surprisesabout which ecomiornetrncs gave us little om miowarning imi advance. The stiort-rumi downward-sloping Phillips curve met its dlemise in the1970s. (Milton F’riedlman [1968] and EdmundPhelps 119681 predicted that it wdiuldl.) The oilembargo of 1973 and its aftermath threw mostmodels off. The slowdowmi of productivitygm-owth hieginning in the 1970s was unfom-eseen.The money demand equation, which appearedto fit well amid he quite stable until the 1970s,has niot fit so well simice then.

How then shouldi we approach econometrics,fom- sciemice amid for policy, in the future? As forscrenice, we should formulate and estimatemodels as we usually dlo, relying both on economictheory and) on ideas suggested by regnmlaritiesobserved in past data. But we should not fail totest those estimated models againist new data

that were not available to influence the processof formulating them. As for policy, we shouldbe cautious about using research findings topredict the effects of any large policy change ofa type that has not been tmied before.

REFERENCES

Christ, Carl F. ‘Interest Rates and ‘Portfolio Selection’among Liquid Assets in the U.S.,” in Christ et al., Meas-urement in Economics: Studies in Mathematical Economicsand Econometrics in Memory of Yehuda Grunteld (StanfordUniversity Press, 1963).

“Judging the Pertormance of Econometric Modelsof the U.S. Economy,” International Economic Review(February 1975), pp. 54—74.

Friedman, Milton. Review ot “Business Cycles in the UnitedStates of America, 1919—1932” by Jan Tinbergen, AmericanEconomic Review (September 1940), pp. 657—60.

Kendrick, John. Productivity Trends in the Un/ted States(Princeton University Press, 1961).

Latane, Henry Allen. “Cash Balances and the Interest Rate—A Pragmatic Approach,” Review of Economics and Statistics(November 1954). pp. 456—60.

Litterman, Robert B. “Forecasting with Bayesian VectorAutoregressions—Five Years of Experience,” Journal ofBusiness end Economic Statistics (January 1986),pp. 25—38.

Lucas, Robert E. Jr. “Econometric Policy Evaluation: A Critique,”The Phillips Curve and Labor Markets, Carnegie-RochesterConference Series on Public Policy, vol. 1, (North-Holland,1976), pp. 19—46.

Marschak, Jacob. “Economic Measurements for Policy andPrediction,” in William C. Hood and Tjalling C. Koopmans,eds., Studies in Econometric Method, Cowles CommissionMonograph No. 14 (Wiley, 1953), pp. 1—26.

McNees, Stephen K. “The Accuracy of Two ForecastingTechniques: Some Evidence and an Interpretation,” NewEngland Economic Review (March/April 1986), pp. 20—31.

_______ “How Accurate Are Macroeconomic Forecasts?”New England Economic Review (July/August 1988),pp. 15—36.

,~ “Man vs. Model? The Role of Judgment in Fore-casting,” New England Economic Review (July/August 1990),pp. 41—52.

Mitchell, Wesley C. Business Cycles: The Problem and Its Set-ting (National Bureau of Economic Research, 1927).

Nelson, Charles R. “A Benchmark for the Accuracy ofEconometric Forecasts of GNP,” Business Economics(April 1984), pp. 52—58.

_______- “The Role of Monetary Policy,” American Eco-nomic Review (March 196B), pp. 1—17.

Fromm, Gary, and Lawrence R. Klein. “The NBER/NSF ModelComparison Seminar: An Analysis of Results,” Anna/s otEconomic and Social Measurement (Winter 1976), pp. 1—28.

Goodhart, Charles. “Problems of Monetary Management:The U.K. Experience,’ in A. S. Courakis, ed., lntlation,Depression, and Economic Policy in the West (Barnes andNoble Books, 1981).

Houthakker, Hendrik. “An International Comparison ofHousehold Expenditure Patterns, Commemorating theCentenary of Engel’s Law,” Econometrica (October 1957),pp. 532—51.

FEDERAL RESERVE BANK OF ST. LOUIS

Phelps, Edmund. “Money-Wage Dynamics and Labor-MarketEquilibrium,” Joumalof Political Economy (Part II, July/August1968), pp. 678—711.

Tinbergen, Jan. Business Cycles in the United States ofAmerica, 1919—1932, Statistical Testing of Business Cycle

Appendix

93

Theories, vol. 2, (League of Nations, 1939).

Zellner, Arnold, and Franz Palm. “Time Series Analysis andSimultaneous Equation Econometric Models,” Journal ofEconometrics (May 1974), pp. 17—54.

On Data For Tables 3 and 4

A. Datafor equations (1’), (2’), (3), (5), (14—19),and (24—25):

Ml = currency plus checkable deposits, bil-lions of dollars

1892—1956, June 30 data: U.S. Bureauof the Census. Historical Statisticsof the US. from Colonial Times to1957 (Government Printing Office,1960), p. 646, series X-267.

1957—58, June 30 data: Economic Reportof the President, 1959, p. 186.

1959—91, averages of daily data forDecember, seasonally adjusted: Eco-nomic Report of the President, 1992,p. 373.

Note: December data, seasonallyadjusted, are close to June 30 data.

GNP = gross national product, billions of dollarsper year

1892—1928: Kendrick (1961), pp. 296—7.1929—59: Economic Report of the

President, 1961, p. 127.1960—88: Economic Report of the

President, 1992, p. 320.1989—91: Survey of Current Business,

July 1992, p. 52.

RAaa = long-term high-grade bond rate, percentper year

1892—1918: Macaulay’s unadjustedrailroad bond rate, U.S. Bureau ofthe Census. Historical Statistics ofthe United States from Colonial Timesto 1957 (Governiment Printing Office,1960), p. 656, series X-332.

1919—91: Moody’s Aaa corporate bondrate:1919—38: U.S. Bureau of the Census.

Historical Statistics of the UnitedStales from Colonial Times to

1957 (Government PrintingOffice, 1960), p. 656, seriesX-333.

1939—91: Economic Report of thePresident, 1992, p. 378.

Note: For pre-1959 data I used sourcesthat were available in 1960, in anattempt to make equation 2’ reproducethe 1892—1959 equation 2, whichoriginally appeared in Christ (1963).These same sources also yield equation1’, which is an approximate reproduc-tion of the 1919—52 equation 1, fromLatané (1.954).

B. Data for 1959—91 for equation (4):

Ml = currency plus checkable deposits, billionsof dollars: same as above.

GDP = gross domestic product, billions of dol-lars per year: Economic Report of thePresidenl, 1992, pp. 298 or 320.

RAaa = Moody’s Aaa corporate bond rate, per-cent per year: same as above.

C. Data for 1919—52 for equation (1), asdescribed in Latani (1954), p. 457:’

Ml: “demand deposits adjusted plus cur-rency in circulation on the mid-yearcall date, (Federal Reserve Board Data).”

U.S. Bureau of the Census. HistoricalStatistics of the United States fromColonial Times to 1957 (GovernmentPrinting Office, 1960). Series X-267

GNP: “Department of Commerce series from1929 to date; 1919—28 Federal ReserveBoard estimates on the same basis(National Industrial Conference Board,Economic Almanac, 1952, p. 201).”

‘Though Latané’s work was published in 1954, researchanalysts at the Federal Reserve Bank of St. Louis usedmore recent data to replicate his work.

MARCH/APRIL 1993

94

RAaa: ‘‘interest rate on high-grade long-termcorporate obligations.’l’he tJ. S. Treasuryseries giving the yields on corporatehigh-grade bonds as r-epoi-ted in theJ”ederal Reserve Bulletin is used from 1936to date. Before 1936 we use annual aver-ages of Macaulav’s high-grade railroadbond yields given in column 5, TablelC), of his Bond Yields, Interest Rates,Stock Prices,” pp. A157—A161. Macaulay,Frederick R. Bond Yields, Interest Rates,Stock Prices (National Bureau of EconomicResearch, 1938).

I). Data for 1892—1959 for equation (2), asdescribed in Christ (1963), pp. 21 7_18:2

Ml: “currency outside banks” plus “demanddeposits adjusted’’, “billions of dollars asof June 30.”

[iS. Bureau of the Census. Historical Statis-tics ofthe United Statesfrom Colonial Timesto 1957 (Government Printing Office, 1960).Series X-267

U.S. Bureau of the Census. !IistoricalStatis-tics ofthe United Slales from Colonial Timesto 1957; (ontinuation to 1962 and Revisions(Government Printing Office, 1965).Series X-267

RAaa:” lonig-tet-m interest rate (Moody’s Aaacorporate bond rate, extrapolatedbefore 1919 via Macaulay’s railroadbond yield index)”, “pet-cent pet- year.”

GNP: “gross national product, billions of dollarsper year.”

‘Though Christ’s work was published in 1963, researchanalysts at the Federal Reserve Bank of St. Louis usedmore recent data to replicate his work.

FEDERAL RESERVE BANK OF ST. LOUIS