When Do the USDA Forecasters Make Mistakes?...forecasters fail to efficiently take into account macroeconomic data and tend to show mistakes during certain economic conditions, which

When Do the USDA Forecasters Make Mistakes?

by

Olga Isengildina-Massa, Berna Karali, and Scott H. Irwin1

May 2013

Forthcoming in Applied Economics

1 Olga Isengildina-Massa is an Associate Clinical Professor at the University of Texas at Arlington. Berna Karali is an Associate Professor in Department of Agricultural and Applied Economics at the University of Georgia. Scott H. Irwin is the Laurence J. Norton Chair of Agricultural Marketing in the Department of Agricultural and Consumer Economics at the University of Illinois at Urbana-Champaign.

1

When Do the USDA Forecasters Make Mistakes?

I. Introduction

‘The U.S. Department of Agriculture’s failure to accurately predict the size of the current corn‐crop harvest has undermined confidence of some investors in the agency’s forecasting, which has for years been held as gospel.’

(Wall Street Journal, October 22, 2010)

The US Department of Agriculture (USDA) ‘…is widely considered unmatched in its data

gathering resources… The National Agricultural Statistics Service, part of the USDA, has a

$156.8m budget for approximately 500 reports each year and 1,050 employees’ (Meyer, 2011).

Due to the unmatched resources used and the generally high quality of information provided,

USDA forecasts function as the “benchmark” to which other private and public forecasts are

compared. The dominant role of USDA forecasts is not surprising given the classic public goods

problem of private underinvestment in information and the critical role that public information

plays in coordinating the beliefs of market participants. The data collection and forecasting role

of the USDA is particularly important in an environment where G-20 countries are developing an

agricultural market information system (AMIS) in response to recent food price volatility. This

system will improve access to data on production and stocks of most food commodities (Boschat

and Moffett, 2011).

It has been widely documented that the release of many USDA reports moves the

markets. For example, Isengildina-Massa et al. (2008) showed that volatility in corn and

soybean futures markets increases about seven times on report release days. The release of the

reports is usually widely watched by traders and other market participants. The market impact of

the reports implies that the errors in USDA forecasts may have large consequences. For

example, a Wall Street Journal article (Pleven and McGinty, 2011) recently indicated that

‘government reports about the U.S. corn crop have become increasingly unreliable of late,

2

contributing to wild swings in corn prices… Corn (CBOT futures) prices have hit the basic limit

20 times since the start of 2009, with eight of those instances, or 40% of them, coming on the

day of a USDA report… Between 1996 and 2008, only 20% of such limit moves came on report

days.’ The article further stated that ‘…over the past two years, the Department of Agriculture’s

monthly forecasts of how much farmers will harvest have been off the mark to a greater degree

than any other two consecutive years in the last 15.’ In addition to market effects, errors in

USDA price estimates may lead to large changes in payments to producers, since some

government payments are calculated based on these estimates.

Evaluation of USDA forecasts in the previous literature has largely focused on

production and price forecasts. For example, Bailey and Brorsen (1998) demonstrated that

USDA forecasts underestimated beef and pork production in the 1980s, but the bias has

disappeared since then and the variance of the forecasts has also declined. Sanders and

Manfredo (2002) found that USDA forecasts of pork, beef, and broiler production were unbiased

but not efficient with respect to past forecasts and they did not encompass all the information

contained in simple time-series models. Sanders and Manfredo (2003) investigated one-quarter

ahead forecasts of quarterly live cattle, live hog, and broiler prices and found that broiler price

forecasts were biased and all forecast series had a tendency to repeat errors. While the USDA

correctly identified the direction of price change in at least 70% of its forecasts, only 48% of

actual prices fell inside a forecasted range for broilers and 35% for hogs. Isengildina et al.

(2004) found that corn price forecast intervals contained the actual prices 36%-82% of the time,

while soybean intervals did so 59%-86% of the time. Isengildina et al. (2006) argued that

revisions to USDA corn and soybean production forecasts were ‘smoothed,’ meaning that not all

3

information available at the time the forecasts were made was incorporated in the forecasts and

some was carried into the next forecast, which could cause substantial loss in accuracy.

The goal of this article is to analyse USDA forecast errors in an attempt to identify

patterns and better understand when the USDA forecasters make mistakes. We hypothesize that

errors in USDA forecasts may stem from two general sources. The first one deals with

forecasters’ abilities and reflects behavioural characteristics and tendencies as described by

Kahneman and Tversky (1973). Thus, forecasting behaviour may be different with respect to

predicting positive versus negative change and extreme versus moderate rate of change (Denrell

and Fang, 2010). The second source reflects data that are not (efficiently) included in the

forecasts. Previous studies (Sanders and Manfredo, 2002, 2003) showed that USDA forecasts do

not encompass simple time-series models. In our study, we investigate whether USDA

forecasters fail to efficiently take into account macroeconomic data and tend to show mistakes

during certain economic conditions, which has not been analysed in previous studies of these

forecasts. Specifically, we investigate how these factors affect the size and the direction of the

forecast errors. The findings of this study will help forecast providers to improve their

procedures and assist forecast users with interpretation and utilization of information contained

in the USDA forecasts.

This study focuses on forecasts for all US corn, soybeans, and wheat categories published

within the World Agricultural Supply and Demand Estimates (WASDE ) reports over the

1987/88 through 2009/10 marketing years. These forecasts are of particular interest because

corn, soybeans, and wheat account for over 90% of total US grain and oilseed production. While

some characteristics of corn and soybean production and price forecasts have been examined

before (Isengildina et al., 2004, 2006), the accuracy of most other USDA forecasts describing

4

supply and demand conditions has been largely overlooked. To the best of our knowledge, only

a few previous studies analyzed a broad range of WASDE production and consumption forecasts

for corn and soybeans (Botto et al., 2006), cotton (Isengildina et al., 2012), and sugar (Lewis and

Manfredo, 2012). Knowledge of supply and demand forecast accuracy is important because

these categories serve as building blocks for price forecasts. Furthermore, supply and demand

estimates are published within a set of other forecasts in WASDE reports that have been shown

to affect the markets.

II. WASDE Forecasts

WASDE reports are typically released by the USDA between the 9th and the 12th of each month

and contain forecasts of supply and demand for most major crops. WASDE forecasts follow a

balance sheet approach, accounting for each component of supply and utilization (see Vogel and

Bange, 1999 for a detailed description of the USDA crop forecast generation process). The

supply side of the balance sheet consists of beginning stocks, production, and imports.

Utilization includes domestic use (or consumption), exports, and ending stocks. Domestic use is

often further subdivided into commodity specific use categories. The demand side of the balance

sheet also includes a “residual” category to account for usage that cannot be measured directly.

The ending stocks for year become beginning stocks for year 1. The balance sheet

approach requires that ‘…total supply must equal domestic use plus exports and ending stocks.

Prices tie both sides of the balance sheet together by rationing available supplies between

competing uses’ (Vogel and Bange, 1999, p. 10). Unlike all other estimates, price forecasts are

published in interval form. Therefore, to be consistent with the other forecasts, midpoints of the

published intervals were considered.

5

Several agencies within USDA are involved in preparing these forecasts: the National

Agricultural Statistics Service (NASS) collects data on US crop production and stocks; the Farm

Service Agency (FSA) describes current policy environment and farmers’ reaction to current

policies; the Agricultural Marketing Service (AMS) provides current price and marketing

reports; the Foreign Agriculture Service (FAS) provides information regarding foreign

production, use and trade; the Economic Research Service (ERS) identifies important economic

effects and implications for prices, quantities supplied and demanded; and the World

Agricultural Outlook Board (WAOB), with separate leaders for each commodity, coordinates the

interagency process used to produce WASDE estimates. This process ensures that the best

available data are used and the estimates are consistent across all USDA publications.

All estimates are forecast on a marketing year basis (September through August for corn

and soybeans and June through May for wheat), thus forecasted values represent marketing year

averages. Fig. 1 demonstrates marketing years and forecast months for the three commodities

studied. The first forecast for a marketing year is released in May preceding the US marketing

year. Production and beginning stocks forecasts are generally finalized by October (for wheat)

and January (for corn and soybeans) of the marketing year. All other estimates are typically

finalized by next September (for wheat) and November (for corn and soybeans). Thus, there is a

9 (6) month forecasting cycle for production and beginning stocks forecasts, and a 19 (17) month

cycle for all other categories for corn and soybeans (wheat).

Within this forecasting cycle, forecasts differ substantially with respect to the amount of

available information. For corn and soybeans, May through August forecasts are provided

during the growing season and reflect new information about the development of the crop.

September through November forecasts represent harvest-time forecasts and production

6

uncertainty is limited to the challenges of the harvesting process. December through August

forecasts generally are considered post-harvest with uncertainty driven mainly by the speed of

consumption. Post-August estimates reflect revisions of the observed marketing year data.

USDA wheat forecasts combine information for both winter wheat and spring wheat. For winter

wheat, the growing season is May, the harvest season is June-July, and the post-harvest season is

August-May. For spring wheat, the growing season is May-July, the harvest season is August-

September, and the post-harvest season is October-May. Combining this information, the

forecasts may be largely differentiated into highly uncertain stage 1 forecasts during May-August

(May-June), less uncertain stage 2 forecasts during September-November (July-August), stage 3

forecasts during December-August (September-May) and stage 4 estimates during September-

November (June-September) for corn and soybeans (wheat), as shown in Fig. 1.

[INSERT FIGURE 1 HERE]

USDA’s WASDE forecasts are considered fixed-event forecasts because the series of

forecasts is related to the same terminal event , where is the release month of the final

estimate for marketing year . The forecasted value published in month j of marketing year is

denoted as jty , where 1,… , , and 19 for corn and soybeans and 17 for wheat. Thus,

each subsequent forecast is an update of the previous forecast describing the same terminal

event. The terminal event for each category describes a marketing year total (for production,

consumption, and stocks categories) or average (for price) values. Based on our definition of a

19 (17) month forecasting cycle, WASDE generates 18 (16) updates for each US variable

forecasted except production and beginning stocks2 (eight updates for corn and soybeans and

2 Because the beginning stocks category is the same as previous year’s endings stocks, it is not evaluated as a separate category in this study.

7

five updates for wheat) within each marketing year. The marketing years covered in this study

are 1(1987/88), …, (2009/10), and 23.

To avoid the impacts of changing forecast levels over the study period on forecast

evaluation and analysis, forecasts jtf were defined as percent changes in forecasted values from

the previous year’s values:

100 ∗ , 1,… ,23; 1, … , 6 for corn and soybeans; 1,… ,4 for wheat (1)

100 ∗ , 1,… ,23; 7, … , for corn and soybeans; 5,… , for wheat (2)

Note that the denominator in Equation 1 is labelled 12 rather than since, for USDA’s year

forecasts during 7 for corn and soybeans and 5 for wheat, are not yet known

because the previous forecasting cycle is not finished by the time the new forecasting cycle

starts. For example, corn and soybean forecasts for 13,… , 19, 13,… ,17forwheat

for marketing year 1 and for 1,… , 7 1,… , 5forwheat for marketing year are

released in the same report. Thus, a first forecast for 2009/10 marketing year would be

calculated as a percent change from the 13th forecast for 2008/09 marketing year (released in the

same report), since the final value will not become known for a few months.

Since forecasts are measured in this study as percent changes from the previous year to

control for scale, percentage forecast error is calculated as3:

, 1, … , 1; 1, … ,23 (3)

3 In this notation is identical to traditional percent forecast error when the previous year’s values are finalized:

100 ∗ 100 ∗ 100 ∗ 100 ∗ 100 ∗ .

8

The absolute value of this measure reflects the magnitude of the forecast error and its average

describes forecast bias.

The means and standard deviations of the final forecast levels ( ) shown in Table 1

demonstrate the average magnitude and variability of the forecast supply and demand categories.

Since most of the analysis in this article is conducted using percent changes in variables and

percent errors, unit levels are presented here as a reference point for the relative size of the

various categories. On the unit level ( ), food, seed, and industrial (FSI hereafter) is a highly

variable category, which likely reflects sharp changes associated with the use of corn for ethanol

production, driven both by policy changes and the price of gasoline. Percent forecast changes

( ) provide another indication of relative volatility in forecasted categories, highlighting

variability in year-to-year changes in all ending stocks and wheat feed and residual forecasts.

Since wheat is relatively expensive, its use for feed is residual to that of corn, thus making its

feed and residual use a relatively small and highly variable category.

[INSERT TABLE 1 HERE]

Across all commodities, ending stocks is a highly variable category. Given the balance

sheet nature of WASDE forecasts, ending stocks reflect the difference between all supply and

demand categories, which means that forecast errors in other categories would be carried over to

ending stocks errors. Fig. 2 illustrates the correlations between errors in endings stocks and

errors in supply and demand categories. Since ending stocks are calculated as supply less total

use, we observe a positive correlation between ending stocks errors and production errors and

mostly negative correlation between ending stocks errors and use category errors. The

magnitude of the correlations suggests that across all three commodities, production forecast

errors are the main sources of error in ending stocks in the beginning of the forecasting cycle.

9

For wheat, errors in seed and feed and residual use are also highly correlated with ending stocks

forecast errors early in the forecasting cycle. As the forecasting cycle progresses and production

uncertainty becomes resolved, other categories become important sources of error in ending

stocks, such as exports and feed and residual for corn, exports and crushings for soybeans, and

exports and feed and residual for wheat. These illustrations suggest that better production and

export forecasts would help improving ending stocks forecasts for these commodities.


Unlike the above statistics that focused on the final values ( 19), error statistics shown

in the last three columns of Table 1 are calculated for the first 18 forecasts ( 1,… ,18). Mean

absolute percent errors (MAPE) illustrate the size of forecast errors, reflecting large errors for

highly variable categories described above: all ending stocks, soybean seed and residual, and

wheat feed and residual forecasts. Mean percent errors (MPE) illustrate the direction of errors

and the presence of bias in the forecasts. Based on our forecast error calculation in Equation 3,

positive error indicates underestimation and negative error reflects overestimation in the USDA

forecasts. Significant underestimation is detected in corn price, soybean crushings and price, and

wheat exports and price, while overestimation is present in soybean seed and residual and ending

stocks forecasts and wheat seed and feed and residual forecasts.

Fig. 3 illustrates how these forecasts biases changed over time. For example,

overestimation in soybean ending stocks forecasts was prevalent from 1994/95 through 2004/05

marketing years and less common in previous marketing years. This figure also shows lack of

trend in errors over time, suggesting that forecasts did not become better or worse. However,

there is some evidence of larger price forecast errors across three commodities as well as

production and export forecast errors in wheat, export and ending stocks errors in soybeans, and

10

ending stocks errors in corn since the mid-2000s. This pattern reflects challenges USDA had

with taking into account structural changes in these markets that occurred around 2006, due to

emerging linkages between crop and energy markets (e.g., Mallory, Irwin, and Hayes, 2012) that

can be traced to high crude oil prices and policy incentives, such as that provided by the 2005

Energy Policy and Renewable Fuel Act (RFA) and 2007 Energy Independence and Security Act

(EISA). In the remainder of this study, we attempt to explain the causes of errors in WASDE

forecasts, focusing on both their size, as measured by the absolute percent error (APE), and

direction, as measured by percent error (PE).


III. Sources of Error

Behavioral sources of error

In this study we hypothesize that errors in USDA forecasts stem from two general sources. The

first source of errors deals with forecasters’ characteristics. Previous studies suggest that

forecasting behaviour may be different with respect to predicting positive versus negative change

(Dreman and Berry, 1995; Ashiya, 2003) and extreme versus moderate rate of change (Denrell

and Fang, 2010). Amir and Ganzach (1998) argue that leniency, representativeness, and

anchoring are the main reasons for forecasters’ bias. Leniency describes a tendency for over-

optimistic predictions, resulting in overestimating positive and underestimating negative changes

as found in earnings forecasts (e.g., Givoly and Lakonishok, 1984; Schipper, 1991; Amir and

Ganzach, 1998). Amir and Ganzach (1998) suggest that one of the possible reasons for leniency

in earnings forecasts is the analysts’ desire ‘…to maintain good relations with management as a

primary source of information.’ In the context of public commodity forecasts, however, this

11

argument does not hold as the information contained in these forecasts has a more ‘two-sided’

effect, i.e., producers favor forecasts that increase prices, while users favor just the opposite.

Thus, an asymmetric reaction to positive versus negative change in WASDE forecasts would

likely be due to mis-calibration of forecasters’ reaction to information and can result in optimism

as well as pessimism. Given the public nature of the forecasts and their ability to affect markets,

these biases should be unintended.

The representativeness heuristic leads to overreaction, causing forecasters to overpredict

change (both positive and negative). Furthermore, Kahneman and Tversky (1973) argue that

when using this heuristic, people choose a prediction value which reflects the extremity of the

predicted information, i.e. larger overestimation is associated with extreme versus moderate rate

of change, as observed by Denrell and Fang (2010). Contrary to representativeness, the

anchoring heuristic results in underreaction. This heuristic explains situations when forecasters

anchor their predictions at a previous value and adjust based on additional information. Since

adjustment is typically insufficient, forecasts based on this heuristic are often too moderate

(underestimation) with respect to both positive and negative changes (Slovic and Lichtenstein,

1971; Kahneman and Tversky, 1973).

These ‘behavioural’ sources of forecast errors are modelled in this study with three

variables: lagged forecast errors ( is the previous year’s error for the same report month),

percent change in forecast level ( ), as described in Equations 1 and 2, and percent change in

forecast level multiplied by a negative change indicator ( 1if 0, 0 otherwise). For the

analysis of the size of errors (APE), representativeness would cause larger errors associated with

larger forecasted change and negative correlation with past errors. Anchoring would result in an

12

opposite pattern. Leniency (and pessimism) would lead to larger errors associated with positive

or negative change (significant coefficient on the negative change indicator).

For the direction of error (PE) analysis, evaluation of the behavioural sources of error is

consistent with traditional efficiency tests that evaluate whether forecast errors are orthogonal to

the forecasts themselves as well as to prior forecast errors (Mincer and Zarnowitz, 1969;

Nordhaus, 1987; Holden and Peel, 1990). If forecasts are efficient, there should be no

relationship between forecast errors and these variables; i.e., the null hypothesis is all

coefficients are zero. Correlation with past errors indicates a systematic component in forecast

errors that can be predicted using lagged errors, i.e. tendency to repeat or overcorrect past errors.

Anchoring would result in forecasters repeating previous errors while representativeness would

lead to overcorrection of errors. Positive correlation with forecast level indicates that the

absolute value of the forecast is smaller than the actual realization, underestimation of change in

either direction, indicative of anchoring. Negative correlation means the change is

overestimated, suggesting representativeness in predictions. When interacted with the negative

change indicator, this variable describes the differences in forecasting positive versus negative

change reflecting leniency or pessimism.

Macroeconomic sources of error

The second source of errors considered in this study is due to macroeconomic information that

has not been fully incorporated in the forecasts. Specifically, we investigate whether USDA

forecasts tend to show mistakes during certain economic conditions. Vogel and Bange (1999,

p.14) note that USDA forecasting process includes ‘…information on such diverse factors as

exchange rates, oil prices, the effects of domestic and foreign agricultural policy and economic

growth.’ These “macroeconomic” sources of forecast errors are represented in this study within

13

five variables described in Table 2. Exchange rates are measured using the trade weighted

exchange index (FX), which represents a weighted average of the foreign exchange value of the

US dollar against the currencies of a broad group of major US trading partners.4 A stronger

dollar (as manifested by an increase in FX index) would cause a drop in exports also affecting

domestic use and price. Oil prices affect multiple levels of input costs, from fertilizer (often

derived from petroleum products) to transportation. Higher input costs would lead to contraction

in production, which in combination with higher transportation costs would put an upward

pressure on prices. Oil prices also affect corn use categories due to the use of corn for ethanol

production. The Spot price of West Texas Intermediate (WTI) crude oil at Cushing, Oklahoma

location is used to measure this effect. The impact of economic growth and business cycles on

WASDE forecasts is measured by Industrial Production Index (IPI, 2007 base) following

Ludvigson and Ng (2005). As increases in industrial production illustrate periods of economic

expansion, we expect consumption or use to increase during these times putting an upward

pressure on prices.5 Inflation affects prices paid and received by farmers as well as expectations

about future prices and storage decisions. Producer Price Index for farm products (PPI, 1982

base) is used to measure the impact of inflation on WASDE forecast errors. Because changes in

ending stocks and prices are affected by all supply and use categories, the macroeconomic

sources of errors described above may also have an effect on stocks and prices. The impact of

structural changes in the grain and oilseed markets that occurred around 2006 (described in the

4 Broad currency index includes the Euro area, Canada, Japan, Mexico, China, United Kingdom, Taiwan, Korea, Singapore, Hong Kong, Malaysia, Brazil, Switzerland, Thailand, Philippines, Australia, Indonesia, India, Israel, Saudi Arabia, Russia, Sweden, Argentina, Venezuela, Chile and Colombia. For more information about trade-weighted indexes see http://www.federalreserve.gov/pubs/bulletin/2005/winter05_index.pdf. 5 Alternative specifications also included short term interest rate, which was dropped from the final estimation due to the high correlation with Industrial Production Index.

14

previous section) is modelled with a post-2006 dummy. Our analysis will reveal whether USDA

forecast errors tend to grow during certain economic conditions and in what direction.


IV. Estimation Method

Consistent with the dual objectives of this study, absolute percent forecast errors are used as

dependent variables in regression analysis to measure the impact of considered factors on the

size of the error and percent errors are used to investigate sources of bias or direction in forecast

errors. The independent variables discussed in the previous section are included in the analysis

of errors of all US corn, soybean, and wheat forecasts from 1987/88 through 2009/10 marketing

years. For each crop for price and ending stocks, the following regressions are estimated:

2006 (4)

and

1 2 3 4

2006 (5)

where are the forecasting cycle stages as defined in the WASDE forecasts section, is the

previous year’s error for the same report month, is percent change in forecast level, as

described in Equations 1 and 2, and is a negative change indicator ( 1 if jtf < 0, 0

otherwise). (Producer Price Index), (Industrial Production Index), (monthly

average cash oil price), and (exchange rate index) are macroeconomic variables and

15

2006 is a structural change dummy variable described in Table 2. Regressions for other

categories have a similar format, but include not all but only relevant macroeconomic variables

as discussed earlier. Because commodity forecasts in this study are measured as percent changes

in forecasted values from the previous year’s values, macroeconomic variables are measured as

percent changes from the previous year as well using:

100 ∗ , 1, … , 23; 1, … , (6)

where is the value of the macro variable in WASDE forecast month 1 for marketing

year and 19 17 for corn and soybeans (wheat). Similarly, represents the value

of that macro variable for the same calendar month but in the previous year. Since macro

variables for the forecast month are not known when those forecasts are made, the values of the

macro variables from the previous forecast month, 1, are used.

All equations are estimated using Ordinary Least Squares (OLS) method. However, as

shown by Beck and Katz (1995) the OLS standard errors of the estimated parameters are

incorrect when the time-series, cross-section data are characterized by having repeated

observations on fixed units. In order to correct for the correlation among the forecast months in

a given marketing year and the correlation between the observations on the same calendar day,

we estimate the variance-covariance matrix as in Karali and Thurman (2009). Specifically, for a

given crop we first run OLS regressions for each balance sheet category and obtain residuals.

For production forecast errors, we only use the first eight report months as they are completed

early during the marketing year. The residuals from the regression of production forecast errors

are correlated across report months due to the fact that each monthly report is forecasting the

same terminal event. We compute variances of residuals for each report month as the sample

mean of the squared residuals across all marketing years as:

16

∑ , 23; 1, … , 8forcornandsoybeans; 1, … , 5forwheat (7)

where is the residual of report month in marketing year . Then, covariances between two

report months are computed as the sample mean of the products of related residuals:

∑ , 23; ; , 1, … , 8forcornandsoybeans; , 1,… , 5forwheat (8)

For all other balance sheet categories, the correlation structure is more complicated. This

is because of the overlapping forecasts released in the same report month. These forecasts are

released on the same calendar day but correspond to two different marketing years since the

forecasting cycle is longer than 12 months. However, because the information available is the

same when these forecasts are made, these observations would be correlated. Thus, in addition

to correlation between report months for a given marketing year, there also exists correlation

between the forecasts of two consecutive marketing years that are released on the same calendar

day. For instance, the 13th forecast figures for the marketing year 1987/1988 are released in May

1988. In the same WASDE report, the first forecast figures for the marketing year 1988/1989 are

also released. Thus, the forecast errors of report months 13 through 18 (16) for marketing year

overlap with the forecast errors of report months 1 through 6 (4) for marketing year 1 for

corn and soybeans (wheat). Hence there are 6 (4) overlapping forecast errors in a marketing year

for corn and soybeans (wheat), resulting in a total of 132 (88) overlapping observations.6 To

correct for this contemporaneous correlation we compute the covariance between the same-day

observations as:

,, 1

, 23; 132; 1, … , 6forcornandsoybeans;

88; 1, … , 4forwheat (9) 6 For marketing year 23, forecast errors do not overlap with the forecast errors in the next marketing year. Thus, we have a total of 22x6=132 overlapping observations for corn and soybeans, and 22x4=88 for wheat.

17

Variances of the residuals for each report month and covariances between the residuals of two

report months in a given marketing year are computed as before:

1,

1, 23; ; , 1, … , 18forcornandsoybeans;

, 1, … ,16forwheat (10)

Using these estimates we construct the variance-covariance matrix of OLS residuals and use it to

compute standard errors of OLS parameter estimates. We present t-statistics computed using

these standard errors along with OLS coefficient estimates in our result tables.

V. Results

Size of errors

Factors affecting the size of errors of WASDE corn, soybean, and wheat forecasts are shown in

Tables 3-5. Results are discussed in both percentage terms and physical units. For all balance

sheet categories, coefficients of stage 1, 2, and 3 variables illustrate the reduction in the size of

forecast errors across the forecasting cycle as more information becomes available. For

example, Table 3 demonstrates that stage 1 corn production forecast errors are on average about

4% (397 million bushels) larger than stage 3 errors. Errors in all other categories are interpreted

relative to stage 4 errors, thus stage 1wheat ending stocks forecast errors are about 25% (161

million bushels) larger than stage 4 errors. The largest reduction in the size of errors during the

forecasting cycle is detected in corn ending stocks and exports forecasts, soybean ending stocks

and seed and residual forecasts, and wheat feed and residual and ending stocks forecasts.


18

The impact of the behavioral sources of error is captured through the size of lagged error,

forecast size and direction. The most common behavioural source affecting the size of errors in

corn forecasts (Table 3) is the positive correlation between the size of the current error and last

year’s error, revealing a pattern of large errors being followed by large errors. This pattern is

consistent with anchoring and may be indicative of challenges with forecasting structural change,

particularly when anchoring one’s forecast on previous information provides consistently

erroneous results. This pattern is detected in corn production and price forecasts. Thus, a 1%

(2.5 cents/bu.) error in the June corn price forecast in year is typically followed by a 0.16%

(0.40 cents/bu.) error in the June forecast the following year. A similar persistence of errors of

even larger magnitude is detected in wheat ending stocks and price forecast errors (Table 5).

Correlation with previous forecast errors in soybeans (Table 4) is only found in one instance,

crushings, and is negative (0.11% or 1.6 million bushels), suggesting that large errors are

typically followed by smaller errors.


The size of soybean forecast errors (Table 4) is most commonly associated with the size

and the direction of the forecast. Thus, the magnitude of forecast error increases by 0.2% (505

thousand bushels) when soybean ending stocks are forecasted to increase and decrease by about

0.12% (0.199 - 0.321 = -0.122) or 309 thousand bushels with each percent decrease in forecasted

stocks. A similar pattern is found in soybean seed and residual forecasts and wheat feed and

residual forecasts (Table 5). The opposite pattern is observed in corn price and soybean

production forecasts. Soybean production forecast error decreases by 0.07% (1.9 million

bushels) when production is forecasted to increase and increases by 0.06% (-0.074 + 0.136 =

0.062) or 1.5 million bushels with each percent decrease in forecasted production. In addition,

19

evidence of smaller errors associated with increasing wheat production and declining ending

stocks is also found. These patterns of asymmetric errors associated with positive versus

negative forecasted change are consistent with both leniency and pessimism. Further

investigation in the direction of errors presented later in this study is required to disentangle

whether leniency or pessimism is causing these errors.


Macroeconomic sources of forecast errors are reflected in the last five variables included

in regression results presented in Tables 3-5. Results demonstrate that errors in price forecasts

across all three commodities increase by about 0.1% (0.29 cents/bu. for corn, 0.70 cents/bu. for

soybeans, and 0.40 cents/bu. for wheat) with each percent growth in inflation (PPI).

Periods of economic growth (as reflected in the increase in IPI) are accompanied by

larger errors in corn feed and residual (0.28% or 15 million bushels) and price (0.18% or 0.45

cents/bu.) forecasts as well as wheat feed and residual forecasts (2.08% or 4.8 million bushels).

On the contrary, errors in soybean ending stocks forecasts decrease by 1.4% (3.5 million

bushels), with each percent increase in IPI. Thus, wheat feed and residual and soybean ending

stocks forecast errors are the most sensitive to changes in economic growth.

Changes in oil prices tend to increase forecast errors mostly during periods of falling oil

prices. During the period of study, each percent decline in oil price is accompanied by an

increase in forecast errors of corn production (0.03% or 2.7 million bushels) and FSI use (0.01%

or 304 thousand bushels) as well as wheat production (0.12% or 2.6 million bushels) and price

(0.04% or 0.15 cents/bu.). Soybean price forecasts also exhibit 0.03% (0.21 cents/bu.) larger

errors as oil price declines, but soybean ending stocks forecast errors actually decline by 0.14%

20

(347 thousand bushels) during these periods. These results demonstrate that the effects of

changes in oil prices on forecast errors are mostly negative and small in magnitude.

Changes in exchange rates are also mostly negatively correlated with the size of forecast

errors, but the magnitude of these effects is substantially larger. The largest effects are observed

in wheat, as wheat exports and feed and residual forecast errors increase by 0.21% (2.3 million

thousand bushels) and 0.94% (2.2 million bushels), respectively, while seed use forecast errors

decrease by 0.13% (112 thousand bushels), with each percent decrease in FX index. Errors of

soybean price forecasts increase by 0.35% (2.25 cent/bu.), as US dollar weakens against other

currencies. Corn FSI forecast errors also decrease by 0.13% (3 million bushels), but corn export

forecast errors increase by 0.30% (5.6 million bushels) with weaker US dollar.

The last macroeconomic variable, a post-2006 dummy, describes changes in forecast

errors that are likely associated with challenges in forecasting structural changes in these markets

that took place in the mid-2000s. The largest increase in errors in post-2006 period is observed

in the residual categories across all three commodities with errors in wheat feed and residual

forecasts growing by 36.3% (84 million bushels), soybean seed and residual by 16.5% (22

million bushels), and corn feed and residual by 2.5% (133 million bushels), making it the largest

source of forecast errors in these categories outside of uncertainty associated with early forecasts.

Wheat production and wheat and corn price forecast errors increased by 6.19% (136 million

bushels), 5.67% (20.83 cents/bu.) and 4.09% (10.29 cents/bu.), respectively, and soybean exports

forecast errors also grew by 2.12% (19 million bushels) during 2006-2010 period. These results

provide quantitative evidence for the trends observed in Fig. 2 and discussed in recent media as

quoted in the introduction.

21

Direction of errors

Evaluation of the impact of considered factors on the direction of forecast errors is consistent

with traditional efficiency tests. In the direction of error results presented in Tables 6-8, stage

coefficients are interpreted as changes in bias across the forecasting cycle and can be compared

to the average bias reported in Table 1. Both Tables 1 and 6 indicate that corn forecasts are

mostly bias free. While underestimation of corn price is detected on average across all stages of

forecasting cycle, it is not statistically different from zero in individual stages. On the other

hand, multiple evidence of bias is detected in soybean forecasts. For example, while Table 1

reports average bias for soybean price forecasts as 2.18%, Table 7 indicates that USDA forecasts

underestimate soybean price by 5.8% (37 cents/bu.) in stage 1 forecasts and 3.4% (22 cents/bu.)

in stage 2. Underestimation in soybean exports and crushings appears most pronounced in stage

2 and 3 forecasts, while overestimation of soybean ending stocks is present across the first three

stages. According to Table 8, stage 2 wheat forecasts are most affected by bias with

underestimation detected in exports and price, and overestimation in feed and residual and seed

forecasts. Other stages are mostly bias free with the exception of exports that are underestimated

in the first stage as well.


With respect to behavioural sources, the direction of error results suggest that USDA corn

and soybean analysts have a tendency to overcorrect previous year’s errors as suggested by the

negative correlation with past errors in corn production, ending stocks, and feed and residual

forecasts and soybean exports and price forecasts. Thus, a 1% (94 million bushels)

overestimation in corn production in year is followed by a 0.28% (26 million bushels)

underestimation in year 1 and 1% (6.4 cents/bu.) overestimation in soybean price is followed

22

by a 2.4% (1.5 cents/bu.) underestimation the following year. This pattern combined with the

evidence of persistent errors found in the case of corn production errors (Table 3) suggests that

while large errors in corn production are followed by large errors, these errors are made in

opposite directions, indicative of overreaction or representativeness heuristic. On the other hand,

persistence of errors in wheat forecasts is consistent across the size of error and direction of error

results (Tables 5 and 8). Positive correlation with past errors is detected in wheat production,

seed, ending stocks, and price forecasts. Thus, a 1% (3.7 cents/bu.) underestimation of wheat

price in year is followed by a 0.63% (2.3 cents/bu.) underestimation in year 1, suggesting a

tendency for repeating past errors, and indicative of anchoring.


The direction of error results allows us to discern whether asymmetric errors associated

with positive versus negative forecasted change are caused by leniency or pessimism. A

tendency to overestimate growth in corn FSI use by 0.41% (9.6 million bushels) and

underestimate contraction by 0.34% (0.755 - 0.413 = 0.342) or 7.9 million bushels is indicative

of leniency. Some evidence of overestimation of growth is also detected in soybean seed and

residual, ending stocks, and price forecasts and in wheat food, feed and residual, and price

forecasts. On the other hand, underestimation of growth by 0.20% (3 million bushels) and

overestimation of contraction by 0.11% (1.6 million bushels) in soybean crushings is indicative

of pessimism. Overestimation of contraction is also found in wheat exports and ending stocks

forecasts. Overall, the patterns of leniency and pessimism are mixed across different categories

and do not exhibit consistent evidence of strategic behaviour that would lead to over- or under-

predicting prices (i.e., optimism with respect to use and pessimism with respect to production

and ending stocks would lead to over-predicting prices and vice versa).

23


With respect to macroeconomic sources of error, forecast rationality implies that if all

relevant information is incorporated in the forecasts, forecast errors should be unpredictable

using the information available at the time the forecasts are made. Thus, if USDA forecasts

efficiently incorporate macroeconomic information, coefficients of the macro variables in

direction of error regressions should be zero.

Our analysis indicates that corn forecasts efficiently incorporate information about

inflation, while soybean and wheat forecasts do not. In soybean forecasts, each percent growth

in inflation is associated with overestimation of 0.42% (1 million bushels) in ending stocks, and

underestimation of 0.13% (84 cents/bu.) in price forecasts. In wheat forecasts, higher inflation is

associated with overestimation of 0.42% (2.8 million bushels) in ending stocks, and

underestimation of 0.16% (0.57 cents/bu.) in price forecasts.

Information about economic growth as reflected in changes in the IPI is associated with

overestimation of 0.25% (13 million bushels) and 0.14% (3.3 million bushels) in corn feed and

residual and FSI forecasts, respectively. On the other hand, each percent growth in IPI leads to

underestimation of 0.79% (1 million bushels), 1.15% (2.9 million bushels) and 0.13% (1 million

bushels) in soybean seed and residual, ending stocks and wheat food use, respectively. Thus, the

impact of economic growth is overstated in corn and understated in soybean and wheat forecasts.

Higher oil prices are associated with underestimation in corn feed and residual (0.04% or

2.2 million bushels), and overestimation in soybean price (0.05% or 0.34 cents/bu.) and wheat

production (0.11% or 2.3 million bushels) forecasts. Consistent with the size of error results, the

magnitude of this inefficiency is very small.

24

A stronger US dollar appears to cause overestimation in price forecasts of 0.31% for corn

(0.77 cents/bu.) and wheat (1.15 cents/bu.) and 0.49% for soybeans (3.15 cents/bu.).

Overestimation during the periods of stronger US dollar is also observed in corn FSI use and

wheat seed and export use. Most other categories including soybean seed and residual, and

crushings as well as wheat food use, feed and residual, and ending stocks exhibit underestimation

as US dollar strengthens.

Consistent with the size of error results, structural changes captured by the post-2006

dummy variable have the largest effect on forecast errors. In corn forecasts, larger errors in post-

2006 period are associated with underestimation in ending stocks by 18.2% (291 million

bushels), FSI by 3.2% (75 million bushels), and overestimation in feed and residual forecasts by

3% (157 million bushels). In soybeans, the post-2006 period is characterized with

underestimation in exports by 6% (54.7 million bushels) and overestimation in seed and residual

use forecasts by 21% (27.5 million bushels). Since 2006, the underestimation in wheat ending

stocks is 9.6% (63 million bushels) and overestimation in wheat feed and residual forecasts is

41% (95.1 million bushels) are observed.

VI. Summary and Conclusions

This article seeks to examine USDA corn, soybean, and wheat forecast errors from 1987/88

through 2009/10 marketing years to better understand when forecasters make mistakes. We

hypothesize that errors may stem from two general sources: behavioural and macroeconomic

factors. We find that the largest increase in the size of USDA forecast errors is associated with

structural changes in commodity markets that took place in the mid-2000s. Post-2006, corn price

and feed and residual forecast errors grew by 4.1% and 2.5%, respectively; soybean seed and

25

residual and export forecast errors grew by 16.5% and 2.1%, respectively; and wheat price,

production, and feed and residual errors grew by 5.7%, 6.2%, and 36.3%, respectively. Corn,

soybean, and wheat forecast errors also grew during the periods of economic growth and with

changes in exchange rates, while inflation and changes in oil price had a much smaller impact.

Changes in economic growth had the largest impact on wheat feed and residual and soybean

ending stocks forecasts. Wheat forecast errors were also the most sensitive to changes in

exchange rates, likely due to substantial export orientation of the wheat market.

With respect to behavioural sources, several forecasts in all three commodities

demonstrated persistence in errors (i.e. large errors are followed by large errors) which is

consistent with anchoring and may be indicative of challenges with forecasting structural change.

Our investigation of the direction of forecast errors revealed that asymmetric errors associated

with positive versus negative forecasted change in corn FSI use is indicative of leniency. On the

other hand, underestimation of growth and overestimation of contraction in soybean crushings

forecasts is indicative of pessimism. Overall, the patterns of leniency and pessimism were

mixed across different categories and do not exhibit consistent evidence of strategic behaviour

that would have led to over- or under- predicting prices.

Multiple cases of inefficiency with respect to macroeconomic factors were also found in

the direction of error results. Our analysis revealed that corn forecasts efficiently incorporated

information about inflation, while soybean and wheat forecasts did not. The impact of economic

growth was overstated in corn and wheat forecasts and understated in soybean forecasts. A

stronger US dollar was associated with overestimation in price forecasts and both over- and

underestimation in consumption categories, while changes in oil prices had a very limited

impact.

26

Fildes and Stekler (2002, p. 454) argue that ‘The use of rationality tests, especially those

that relate the forecast errors to known information, can … be viewed as important diagnostic

checks to determine why the errors occurred and to improve the forecasting process and the

quality of subsequent predictions.’ Our findings can thus be used by USDA forecast providers

to improve their forecasts. For example, if our estimates were used to adjust May 2010 corn

Food, Seed, and Industrial forecast for the 2010/11 marketing year, the forecast adjustment factor

would be calculated by multiplying values of macro and behavioral variables known at the time

by the significant inefficiency coefficients shown in Table 6. Given the values of the variables

(measured as percent change from previous year’s values) known in the beginning of May 2010

of 3.767 for forecast size, 5.628 for IPI, and -7.637 for FX, the adjustment factor is 3.145 -

0.413*3.767 – 0.141*5.628 – 0.126*(-7.637) + 3.234 = 4.995. When this adjustment factor is

added to the published forecast of 3.767 (5,950 million bushels), the adjusted forecast is 8.762

(6,247 million bushels). Since the final estimate of corn Food, Seed, and Industrial forecast for

2010/11 marketing year was 6,428 million bushels, this adjustment reduced the May forecast

error from 478 million bushels to181 million bushels.

The findings of this study can also help market participants to interpret USDA

information. If market participants are fully aware of the flaws and inefficiencies in USDA

forecasts and adjust for them in their decision making process, limited or no economic losses

would result (Orazem and Falk, 1989). The degree to which market participants anticipate and

adjust information contained in USDA forecasts presents an interesting area for future research.

27

References

Amir, E. and Ganzach, Y. (1998) Overreaction and underreaction in analysts’ forecasts, Journal of Economic Behavior and Organization, 37, 333-47.

Ashiya, M. (2003) Testing the rationality of Japanese GDP forecasts: the sign of forecast revision matters, Journal of Economic Behavior and Organization, 50, 263-69.

Bailey, D. V. and Brorsen, B. W.. (1998) Trends in the accuracy of USDA production forecasts for beef and pork, Journal of Agricultural and Resource Economics, 23, 515–26.

Beck, N. and Katz J. N. (1995) What to do (and not to do) with time-series cross-section data, The American Political Science Review, 89, 634–47.

Boschat, N. and Moffett, S. (2011) G-20 Tackles Food Production. Wall Street Journal, June 23, 2011.

Botto, A. C., Isengildina, O., Irwin, S. H., Good, D. L. (2006) Accuracy Trends and Sources of Forecast Errors in WASDE Balance Sheet Categories for Corn and Soybeans. Paper presented at AAEA 2006 Annual Meetings, Long Beach CA, 2006.

Denrell, J. and Fang, C. (2010) Predicting the next big thing: success as a signal of poor judgment, Management Science, 56, 1653-67.

Dreman, D. N. and Berry, M. A. (1995) Analyst forecasting errors and their implications for security analysis, Financial Analysts Journal, 51, 30-41.

Fildes, R. and Stekler, H. (2002) The state of macroeconomic forecasting, Journal of Macroeconomics, 24, 435-68.

Givoly, D. and Lakonishok, J. (1984) Properties of analysts’ forecasts of earnings: a review and analysis of the research, Journal of Accounting Literature, 3, 117-52.

Holden, K. and Peel, D. A. (1990) On testing for unbiasedness and efficiency of forecasts, Manchester School, 58, 120-27.

Isengildina-Massa, O., MacDonald, S., and Xie, R. (2012) Comprehensive evaluation of USDA cotton forecasts, Journal of Agricultural and Resource Economics, 37, 98-113.

Isengildina-Massa, O., Irwin, S. H., Good, D. L., and Gomez, J. K. (2008) The impact of situation and outlook information in corn and soybean futures markets: evidence from WASDE reports, Journal of Agricultural and Applied Economics, 40, 89-104.

Isengildina, O., Irwin, S. H., and Good, D. L. (2004) Evaluation of USDA interval forecasts of corn and soybean prices, American Journal of Agricultural Economics, 86, 990-1004.

Isengildina, O., Irwin, S. H., and Good, D. L. (2006) Are revisions to USDA crop production forecasts smoothed? American Journal of Agricultural Economics, 88, 1091-1104.

28

Kahneman, D. and Tversky, A. (1973) On the psychology of prediction, Psychological Review, 80, 237-51.

Karali, B., and Thurman, W. N. (2009) Announcement effects and the theory of storage: an empirical study of lumber futures, Agricultural Economics, 40, 421–36.

Lewis, K. and Manfredo, M. (2012) An evaluation of the USDA sugar production and consumption forecasts, in Proceedings of the NCCC-134 Conference on Applied Commodity Price Analysis, Forecasting, and Market Risk Management, St. Louis, Missouri.

Ludvigson, S. C. and Ng, S. (2005) Macro factors in bond risk premia, NBER Working Paper 11703.

Mallory, M.L., Irwin, S.H., and Hayes, D.J. How market efficiency and the theory of storage link corn and ethanol markets, Energy Economics, 34, 2157-2166.

Meyer, G. (2011). Reliability of US crop reports questioned, Financial Times, July 12, 2011.

Mincer, J, and Zarnowitz, V. (1969) The evaluation of economic forecasts, in Economic Forecasts and Expectations: Analysis of Forecasting Behavior and Performance, National Bureau of Economic Research, Inc., pp. 1-46.

Nordhaus, W. D. (1987) Forecasting efficiency: concepts and applications, Review of Economics and Statistics, 69, 667-74.

Orazem, P. and Falk, B. (1989) Measuring market responses to error-ridden government announcements, Quarterly Review of Economics and Business, 29, 41-55.

Pleven, L. and T. McGinty (2011) Faulty forecasts roil corn market. Wall Street Journal, December 5, 2011.

Sanders, D. R. and Manfredo, M. R. (2002) USDA production forecasts for pork, beef, and broilers: an evaluation, Journal of Agricultural and Resource Economics, 27, 114-28.

Sanders, D. R. and Manfredo, M. R. (2003) USDA livestock price forecasts: a comprehensive evaluation, Journal of Agricultural and Resource Economics, 28, 316-34.

Schipper, K. (1991) Commentary on analysts’ forecasts, Accounting Horizons, 5, 105-21.

Slovic, P., and Lichtenstein, S. (1971) Comparison between Bayesian and regression approaches to the study of information processing in judgment, Organizational Behavior and Human Decision Processes, 6, 649-744.

Vogel, F. A. and Bange, G. A. (1999) Understanding USDA crop forecasts, Miscellaneous Publication No. 1554, US Department of Agriculture, National Agricultural Statistics Service and Office of the Chief Economist, World Agricultural Outlook Board.

29

Table 1. Descriptive Statistics for WASDE Forecasts of U.S. Corn, Soybeans and Wheat, 1987/88 -2009/10 Marketing Years

Forecast Variable Mean

(y) Standard Dev. (y)

Mean (f)

Standard Dev. (f)

MAPE (│e│)

MPE (e)

Bias (t-test)

Corn Production 9409 2073 2.012 21.673 1.816 -0.153 -0.644 Food, Seed, Industrial 2322 1258 6.985 6.241 2.049 0.137 -0.364 Feed and Residual 5333 582 0.389 9.102 2.949 -0.257 -0.855 Exports 1903 274 1.211 19.219 7.038 0.150 0.413 Ending Stocks 1597 727 -4.566 52.211 14.717 1.353 0.885 Price 2.52 0.67 3.746 18.692 5.305 1.019 2.585 **

Soybeans Production 2512 493 2.387 13.271 2.118 0.311 0.901 Crushings 1491 232 1.722 5.615 2.087 0.811 5.442 *** Seed and Residual 131 38 0.164 25.177 15.570 -2.734 5.189 *** Exports 908 238 2.976 17.291 6.019 2.434 -1.628 Ending Stocks 253 107 -4.610 44.808 22.870 -10.300 -6.421 *** Price 6.41 1.61 3.009 18.426 4.466 2.181 5.580 ***

Wheat Production 2204 268 0.745 18.391 2.763 0.056 -0.647 Food 873 74 1.549 3.439 1.800 -0.133 -0.230 Seed 88 9 -2.111 9.878 3.951 -2.166 -6.670 *** Feed and Residual 231 102 -16.282 59.123 24.275 -12.573 -5.515 *** Exports 1129 181 -0.110 17.534 5.905 1.423 2.651 *** Ending Stocks 655 233 -4.185 44.194 13.043 1.043 0.117 Price 3.67 1.14 10.247 29.309 6.205 2.424 3.217 ***

Notes: y is forecast level measured in million bushels and $/bushel. f is a % change in the forecast from the previous year's forecast. Means and standard deviations of forecast level (y) and forecast change (f) are computed using the final estimates (j=19) only. Mean absolute percent error (MAPE) and mean percent error (MPE) are computed for j=1, ..., 18. One asterisk (*) indicates significance at the 10% level, two asterisks (**) indicate significance at the 5% level, and three asterisks (***) indicate significance at the 1% level.

30

Variable Name Description MeanStandard

Dev.

PPIProducer Price Index for commodities (farm products), 1982 base year

1.747 10.572

IPI Industrial Production Index, 2007 base year 2.042 4.054

Oil Price WTI FOB spot price 6.385 30.605

FXTrade Weighted Exchange Index (Broad), 1997 base year

2.271 5.915

Post-2006 dummy 1 after 2005/06 marketing year, 0 otherwise 0.174 0.379

Table 2. Description and Summary Statistics for Macroeconomic Sources of Error, 1987/88 -2009/10 Marketing Years

31

Dependent Variable

Independent VariablesIntercept 1.090 0.137 0.581 -0.521 0.165 0.257

(1.12) (0.25) (1.33) (-0.43) (0.06) (0.32)Stage 1 (May-Aug) 4.223 *** 4.430 *** 2.837 *** 12.429 *** 25.240 *** 10.192 ***

(4.32) (6.54) (6.50) (8.48) (7.59) (8.29)Stage 2 (Sep-Nov) 1.129 3.035 *** 2.490 *** 10.760 *** 16.842 *** 6.509 ***

(1.13) (5.68) (6.19) (8.70) (5.91) (7.83)Stage 3 (Dec-Aug) 1.396 *** 1.097 *** 3.806 *** 6.191 *** 1.799 ***

(5.95) (3.77) (5.23) (3.70) (2.89)Lagged APE 0.182 ** -0.008 0.144 0.033 0.086 0.158 ***

(2.17) (-0.13) (1.63) (0.50) (1.38) (2.68)Forecast (f ) -0.021 -0.062 -0.021 0.102 -0.054 -0.074 **

(-0.50) (-0.90) (-0.37) (1.46) (-1.04) (-2.08)Negative Forecast -0.041 0.112 -0.136 -0.063 -0.040 0.182 **

(-0.58) (1.09) (-0.63) (-0.52) (-0.52) (2.36)PPI 0.193 0.116 ***

(1.42) (2.97)IPI 0.283 *** -0.060 0.072 0.177 *

(4.20) (-1.22) (0.19) (1.78)Oil Price -0.029 * -0.015 -0.013 ** 0.004 0.005

(-1.91) (-1.56) (-2.11) (0.07) (0.35)FX 0.026 -0.131 *** 0.296 *** -0.129 -0.069

(0.51) (-3.56) (3.15) (-0.50) (-1.01)Post-2006 -1.541 2.489 *** 1.325 -1.462 2.168 4.087 ***

(-1.16) (3.61) (1.48) (-1.04) (0.58) (3.98)N 184 414 414 414 414 414

Table 3. Factors Affecting the Size of Errors in WASDE Corn Forecasts, 1987/88 -2009/10 Marketing Years

t-statistics. One asterisk (*) indicates significance at the 10% level, two asterisks (**) indicate significance at the 5% level, and three asterisks (***) indicate significance at the 1% level.

Notes : Negative forecast is forecast (f ) times a negative change indicator. Numbers in parentheses are

Absolute Percent Error (APE)

ProductionFeed &

ResidualFood, Seed, Industrial

ExportsEnding Stocks

Average Price

32

Dependent Variable

Independent VariablesIntercept 1.800 ** -0.255 -0.142 0.632 1.779 0.948

(1.98) (-0.11) (-0.38) (0.69) (0.71) (0.96)Stage 1 (May-Aug) 3.996 *** 14.850 *** 3.871 *** 9.124 *** 34.476 *** 9.386 ***

(4.66) (4.81) (11.10) (7.74) (9.53) (7.59)Stage 2 (Sep-Nov) 1.124 13.783 *** 3.132 *** 8.289 *** 25.689 *** 7.102 ***

(1.26) (6.30) (9.22) (7.30) (10.34) (6.77)Stage 3 (Dec-Aug) 7.896 *** 1.707 *** 3.009 *** 12.404 *** 2.058 ***

(4.32) (8.64) (4.76) (5.93) (2.62)Lagged APE 0.101 -0.084 -0.106 ** 0.080 -0.067 0.008

(1.58) (-1.09) (-2.09) (1.22) (-1.01) (0.10)Forecast (f ) -0.074 * 0.367 *** 0.039 -0.090 0.199 *** -0.082

(-1.80) (5.54) (0.66) (-1.56) (5.02) (-1.62)Negative Forecast 0.136 * -0.436 *** -0.085 0.054 -0.321 *** 0.065

(1.86) (-3.73) (-0.95) (0.60) (-4.58) (0.74)PPI -0.140 0.110 **

(-1.04) (2.13)IPI 0.214 0.017 -1.401 *** 0.086

(0.79) (0.46) (-4.31) (0.66)Oil Price 0.010 0.137 *** -0.032 *

(1.21) (2.81) (-1.89)FX -0.040 0.011 -0.011 -0.053 -0.351 ***

(-0.23) (0.48) (-0.16) (-0.21) (-3.73)Post-2006 -0.599 16.491 *** 0.155 2.117 * -2.658 1.716

(-0.81) (4.89) (0.37) (1.90) (-0.83) (1.18)N 184 414 414 414 414 414

Table 4. Factors Affecting the Size of Errors in WASDE Soybean Forecasts, 1987/88 -2009/10 Marketing Years

One asterisk (*) indicates significance at the 10% level, two asterisks (**) indicate significance at the 5% level, and three asterisks (***) indicate significance at the 1% level.


Production

Notes : Negative forecast is forecast (f ) times a negative change indicator. Numbers in parentheses are t-statistics.

Seed and Residual

Crushings ExportsEnding Stocks

Average Price

33

Table 5. Factors Affecting the Size of Errors in WASDE Wheat Forecasts, 1987/88 -2009/10 Marketing Years

Dependent Variable

Independent VariablesIntercept 2.686 0.469 -0.398 -8.461 1.012 -1.712 -0.010

(1.37) (1.02) (-0.40) (-1.16) (0.78) (-0.78) (-0.01)Stage 1 (May-Jun) 8.082 *** 2.184 *** 6.521 *** 35.737 *** 12.443 *** 24.582 *** 12.638 ***

(3.61) (4.23) (5.75) (4.59) (8.72) (8.87) (6.58)Stage 2 (Jul-Aug) 6.551 *** 2.178 *** 6.689 *** 35.771 *** 12.640 *** 22.335 *** 10.827 ***

(4.07) (4.76) (6.11) (4.41) (8.06) (8.42) (6.04)Stage 3 (Sep-May) 1.478 *** 2.721 *** 13.278 ** 3.730 *** 5.603 *** 1.064

(4.72) (3.92) (2.05) (4.38) (3.88) (1.07)Lagged APE 0.163 -0.020 0.075 0.055 0.046 0.232 *** 0.406 ***

(1.24) (-0.26) (0.94) (0.69) (0.54) (3.48) (6.30)Forecast (f ) -0.171 * 0.109 0.005 0.269 *** -0.021 0.008 -0.069

(-1.66) (1.35) (0.06) (3.15) (-0.31) (0.21) (-1.58)Negative Forecast 0.214 -0.050 -0.106 -0.390 *** -0.014 -0.112 * 0.058

(1.13) (-0.28) (-0.79) (-2.73) (-0.12) (-1.74) (0.74)PPI 0.057 0.108 *

(0.68) (1.89)IPI -0.044 0.058 2.079 *** -0.146 0.156

(-1.01) (0.63) (2.99) (-0.60) (0.89)Oil Price -0.117 *** -0.042 -0.042 **

(-3.62) (-1.41) (-1.98)FX -0.013 0.127 ** -0.944 ** -0.208 ** 0.219 -0.004

(-0.46) (2.26) (-2.32) (-2.37) (1.30) (-0.03)Post-2006 6.187 ** -0.546 0.941 36.315 *** -1.534 3.971 5.670 ***

(2.52) (-1.07) (0.89) (4.70) (-0.93) (1.36) (2.58)N 115 368 368 368 368 368 368



Notes: Negative forecast is forecast (f ) times a negative change indicator. Numbers in parentheses are t-statistics.

Food SeedFeed and Residual


Average Price

Production

34

Dependent Variable

Independent VariablesIntercept -0.934 0.587 3.145 *** -1.359 -3.533 -0.201

(-0.68) (0.73) (5.04) (-0.57) (-0.90) (-0.12)Stage 1 (May-Aug) -1.159 0.968 0.048 0.251 -3.878 3.018

(-0.81) (1.14) (0.07) (0.10) (-0.76) (1.36)Stage 2 (Sep-Nov) 0.662 0.613 0.121 -0.235 1.078 2.124

(0.50) (0.81) (0.22) (-0.12) (0.26) (1.31)Stage 3 (Dec-Aug) -0.731 * -0.144 0.631 2.531 0.216

(-1.94) (-0.44) (0.53) (0.93) (0.17)Lagged PE -0.278 *** -0.253 *** 0.016 -0.023 -0.253 *** -0.091

(-3.02) (-3.57) (0.16) (-0.27) (-3.14) (-1.08)Forecast (f ) 0.050 -0.018 -0.413 *** 0.094 -0.033 -0.054

(-0.92) (-0.18) (-5.10) (0.65) (-0.46) (-0.72)Negative Forecast -0.062 0.036 0.755 ** -0.077 -0.077 -0.072

(-0.63) (0.24) (2.55) (-0.32) (-0.72) (-0.44)PPI -0.061 0.043

(-0.29) (0.44)IPI -0.246 ** -0.141 ** 0.212 0.303

(-2.52) (-1.99) (0.39) (1.29)Oil Price 0.004 0.042 *** -0.006 -0.031 -0.044

(0.17) (3.33) (-0.65) (-0.45) (-1.49)FX 0.053 -0.126 ** 0.217 -0.315 -0.305 **

(0.73) (-2.32) (1.15) (-0.81) (-1.98)Post-2006 1.834 -2.947 *** 3.234 *** 1.045 18.197 *** 1.087

(0.98) (-2.65) (2.58) (0.35) (3.26) (0.45)N 184 414 414 414 414 414



Percent Error (PE)

Table 6. Factors Affecting the Direction of Errors in WASDE Corn Forecasts, 1987/88 -2009/10 Marketing Years

Average Price

ProductionFeed &

ResidualFood, Seed, Industrial


35

Dependent Variable

Independent VariablesIntercept -0.630 -2.447 -1.212 ** -0.348 -2.605 1.828

(-0.47) (-0.81) (-2.18) (-0.22) (-0.51) (1.32)Stage 1 (May-Aug) -0.188 2.328 0.606 2.473 -13.534 *** 5.796 ***

(-0.13) (0.75) (1.23) (1.36) (-3.30) (3.50)Stage 2 (Sep-Nov) 0.650 -0.445 1.573 *** 5.313 *** -10.592 *** 3.406 ***

(0.53) (-0.15) (3.27) (3.61) (-2.92) (2.65)Stage 3 (Dec-Aug) -2.321 1.346 *** 2.549 *** -6.649 *** 1.299

(-1.64) (3.85) (2.65) (-2.62) (1.10)Lagged PE -0.075 -0.018 -0.080 -0.170 * 0.075 -0.237 ***

(-0.85) (-0.22) (-1.22) (-1.91) (1.00) (-2.69)Forecast (f ) 0.032 -0.240 ** 0.203 ** -0.058 -0.271 *** -0.193 ***

(0.52) (-2.44) (2.13) (-0.54) (-3.35) (-2.94)Negative Forecast -0.176 -0.167 -0.308 ** 0.078 -0.148 0.165

(-1.57) (-1.02) (-2.08) (0.48) (-1.07) (1.43)PPI -0.416 ** 0.132 *

(-2.00) (1.78)IPI 0.792 * 0.061 1.150 * 0.260

(1.98) (0.97) (1.82) (1.47)Oil Price -0.013 -0.026 -0.054 **

(-0.96) (-0.36) (-2.35)FX 0.831 *** 0.147 *** -0.072 -0.238 -0.491 ***

(3.19) (3.98) (-0.59) (-0.53) (-3.89)Post-2006 0.982 -20.980 *** -0.125 6.023 *** -6.041 1.473

(1.06) (-4.00) (-0.18) (3.09) (-0.76) (0.74)N 184.000 414 414 414 414 414



Table 7. Factors Affecting the Direction of Errors in WASDE Soybean Forecasts, 1987/88 -2009/10 Marketing Years

Seed and Residual

Percent Error (PE)

Production Crushings ExportsEnding Stocks

Average Price

36

Table 8. Factors Affecting the Direction of Errors in WASDE Wheat Forecasts, 1987/88 -2009/10 Marketing Years

Dependent Variable

Independent VariablesIntercept 1.163 0.091 -0.150 4.460 -1.471 -7.170 * 2.619

(0.45) (0.12) (-0.11) (0.48) (-0.70) (-1.87) (1.43)Stage 1 (May-Jun) 1.156 -0.134 -1.926 -8.899 4.719 ** -1.108 3.198

(0.38) (-0.20) (-1.53) (-0.92) (2.19) (-0.29) (1.61)Stage 2 (Jul-Aug) 0.594 -0.106 -3.211 *** -19.911 * 3.750 * -1.816 3.562 *

(0.18) (-0.16) (-2.41) (-1.86) (1.71) (-0.43) (1.76)Stage 3 (Sep-May) -0.169 -1.424 -8.705 0.946 1.234 -0.079

(-0.33) (-1.57) (-1.16) (0.77) (0.53) (-0.07)Lagged PE 0.197 ** 0.092 0.285 *** 0.027 0.159 0.263 *** 0.631 ***

(2.24) (0.93) (3.30) (0.31) (1.56) (3.72) (11.20)Forecast (f ) -0.146 -0.396 *** 0.036 -0.265 ** 0.150 -0.032 -0.101 *

(-1.34) (-2.89) (0.35) (-2.42) (1.35) (-0.45) (-1.95)Negative Forecast 0.144 0.278 -0.019 0.190 -0.311 * -0.221 * 0.135

(0.72) (0.95) (-0.11) (1.02) (-1.67) (-1.93) (1.45)PPI -0.421 *** 0.155 **

(-2.93) (2.10)IPI 0.130 * 0.154 -0.930 0.109 -0.268

(1.83) (1.19) (-0.99) (0.26) (-1.26)Oil Price -0.106 *** 0.005 -0.027

(-3.86) (0.09) (-1.01)FX 0.089 ** -0.205 ** 1.556 *** -0.324 ** 0.928 *** -0.312 **

(2.00) (-2.55) (2.84) (-2.22) (3.24) (-2.18)Post-2006 1.281 1.312 0.474 -41.079 *** 0.236 9.612 ** 2.820

(0.67) (1.54) (0.30) (-3.95) (0.09) (2.03) (1.10)N 115.000 368 368 368 368 368 368

(*) indicates significance at the 10% level, two asterisks (**) indicate significance at the 5% level, and three asterisks (***) indicate significance at the 1% level.

Percent Error (PE)

Notes : Negative forecast is forecast (f ) times a negative change indicator. Numbers in parentheses are t-statistics. One asterisk

Food SeedFeed and Residual


Average Price

Production

37

Fig 1. The 2009/2010 WASDE Forecasting Cycle (i =Forecast Month) for Corn, Soybeans, and Wheat Relative to the US Marketing Year

i=1,

May

200

9

i=2,

Jun

e 20

09

i=3,

Jul

y 20

09

i=4,

Aug

. 200

9

i=5,

Sep

t. 20

09

i=6,

Oct

. 200

9

i=7,

Nov

. 200

9

i=8,

Dec

. 200

9

i=9,

Jan

. 201

0

i=10

, Feb

. 201

0

i=11

, Mar

. 201

0

i=12

, Apr

. 201

0

i=13

, May

201

0

i=14

, Jun

e 20

10

i=15

, Jul

y 20

10

i=16

, Aug

. 201

0

i=17

, Sep

t. 20

10

i=18

, Oct

. 201

0

i=19

, Nov

. 201

0

US Marketing Year for Corn and Soybeans

Stage 1 Stage 3Stage 2 Stage 4

i=1,

May

200

9

i=2,

Jun

e 20

09

i=3,

Jul

y 20

09

i=4,

Aug

. 200

9

i=5,

Sep

t. 20

09

i=6,

Oct

. 200

9

i=7,

Nov

. 200

9

i=8,

Dec

. 200

9

i=9,

Jan

. 201

0

i=10

, Feb

. 201

0

i=11

, Mar

. 201

0

i=12

, Apr

. 201

0

i=13

, May

201

0

i=14

, Jun

e 20

10

i=15

, Jul

y 20

10

i=16

, Aug

. 201

0

i=17

, Sep

t. 20

10

US Marketing Year for Wheat

Stage 1 Stage 2 Stage 3 Stage 4

38

Fig 2. Correlations of Ending Stock Forecast Errors with Errors in Other Balance Sheet Categories Across All Months of the Forecasting Cycle, 1987/88 - 2009/10 Marketing Years

‐1

‐0.8

‐0.6

‐0.4

‐0.2

0

0.2

0.4

0.6

0.8

1

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

Production Exports FSI Feed&Residual

‐1

‐0.8

‐0.6

‐0.4

‐0.2

0

0.2

0.4

0.6

0.8

1

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

Production Exports Crushings Seed&Residual

‐1

‐0.8

‐0.6

‐0.4

‐0.2

0

0.2

0.4

0.6

0.8

1

1 2 3 4 5 6 7 8 9 10 11 12 13 14

Production Exports Food

Feed&Residual Seed

Corn

Soybeans

Wheat

39

Fig 3. Average Annual Errors for Selected Categories of WASDE Corn, Soybean, and Wheat Forecasts, 1987/88 - 2009/10 Marketing Years

‐30

‐20

‐10

0

10

20

30Corn

‐30

‐20

‐10

0

10

20

30

Soybeans

‐30

‐20

‐10

0

10

20

30

Production Exports Ending Stocks Price

Wheat

Documents

When Do the USDA Forecasters Make Mistakes?...forecasters fail to efficiently take into account macroeconomic data and tend to show mistakes during certain economic conditions, which