Prediction intervals for growth curve forecasts

Journal of Forecasting. Vol. 14, 413^30 (1995)

Prediction Intervals for Growth CurveForecasts

NIGEL MEADE AND TOWHIDUL ISLAMThe Management School, Imperial College, London, UK

ABSTRACT

Since growth curves are ofteti used lo produce medium- to long-termforecasts for planning purposes, it is obviously of value to be able toassociate an interval with the forecast trend. The problems in producingprediction intervals are well described by Chatfield. The additionalproblems in this context are the intrinsic non-linearity of the estimationprocedure and the requirement for a prediction region rather than a singleinterval. The approaches considered are a Taylor e.xpansion of the varianceof the forecast values, an examination of the joint density of theparameter estimates, and bootstrapping. The performance of the resultantintervals is examined using simulated data sets. Prediction intervals for realdata are produced to demonstrate their practical value.

KEY WORDS prediction intervals; growth curves; empirical cdf

INTRODUCTION

The importance of associating a measure of uncertainty with a point forecast is well known.However, the emphasis in the literature on growth curve forecasting has been on point forecastsvia trend extrapolation. There have been relatively few attempts to quantify the uncertaintyinherent in growth curve forecasts. Meade (1984) found 12 out of 27 papers on growth curveforecasting which discussed uncertainty. In only one of these (Harrison and Pearce, 1972) wereapproximate confidence intervals given.

One explanation for this reluctance to quantify uncertainty lies in the non-linearity of theestimation process. Other explanations, mentioned by Harrison and Pearce, are the lack of faithin the model chosen and problems of defining the correlation structure of the errors. In asignificant contribution to the description of uncertainty in forecasting, Chatfield (1993)reiterates the distinction between

• A confidence interval which describes the uncertainty in an estimate of an unknown butfixed value and

• A prediction interval which describes the uncertainty in an estimate of a future realizationof a random variable.

Chatfield ranges widely over the problems of providing prediction intervals, concentratingmainly on linear models. He cites a number of reasons that prediction intervals may be too narrow;

• Model parameters are assumed to be known when they are, in fact, estimates.

0277-6693/95/050413-18 Received August 1994© 1995 by John Wiley & Sons, Ltd. Revised December 1994

414 Journal of Forecasting Vol. 14. Iss. No. 5

• Innovations may not be normally distributed, or more generally, an incorrect assumption inthe innovation distribution.

• Incorrect model identification.• The data generating process may change, causing parameter estimates to be increasingly

inaccurate or causing a model choice to become inappropriate.

In this study we are concerned with the production of prediction intervals for growth curveforecasts, and the points mentioned above will be discussed in this context.

After a brief literature review, three frameworks for growth curve prediction intervals will bedeveloped. The effectiveness of these frameworks is tested via Monte Carlo simulation andcomparisons made. The problems of dealing with short data series (less than 25 observations)are discussed and prediction intervals for two well-known data series are generated.

REVIEW

Brand et al. (1973) and Hauck (1983) have proposed methods for constructing a confidenceinterval for the two parameter logistic curve:

Both approaches take a linear transformation of the curve, compute confidence intervals for thelinear representation, and then retransform to give the final confidence intervals. In this work thelogistic is regarded as a response curve and thus the path of the curve is fixed but unknown(thus the term 'confidence interval' is appropriate).

There are several obstacles to applying this type of approach to produce prediction intervals.The primary difference is that the underlying sample is assumed to be across the whole range ofthe curve, rather than a sequence up to a particular value of /, as would occur in the forecastingcontext. The absence of a term for the saturation level means that a major source of uncertaintyin forecasting is ignored.

The prediction intervals of the linear transformation must be retransformed, and thecorrection for bias after retransformation has been addressed by several authors (for example,Guerrero, 1993; Pankratz and Dudley, 1987; Dadkah, 1984). However, the nature of thedistribution of the innovations is not obvious, especially for forecasts based on short series.

Migon and Gamerman (1993) used a Bayesian approach to forecasting using a generalizedexponential growth model. This approach generates the information necessary for thecomputation of prediction intervals and the model class includes the logistic and Gompertzcurves.

SUGGESTED FRAMEWORKS FOR PREDICTION INTERVALS

A growth curve model with an additive error term can be represented as follows:

X, = fia,b,c,t)-i-e, (2)

where f{a, b, c, t) represents the growth curve and e, is the error term. The function f(...) isintended to encapsulate the equation of the growth curve. It has the parameters a, b, etc. as wellas time or past observations as arguments as demanded by the curve.

Nigel Meade and Towhidul Islam Prediction Intervals for Growth Curve Forecasts 415

For example, f(a, b, c, t) may be a simple logistic:

fia,b,c,t)= (3)

1 + ce""'

or it may be a Gompertz:

"(c + bt)) (4)where a is the saturation level and i > 0 and c<0. This formulation of the Gompertz (used byLee and Lu, 1992) was used in preference to the conventional representation

f{a,b,c, t) = a exp(-c'(exp(-/>r))

where c'>0. The conventional representation was more prone to producing a near-singularestimate covariance matrix than equation (4).

The third growth curve considered is the local logistic (see Meade, 1985, 1988 for details). Inthis case:

(5)

The forecast at time T, r steps ahead, is the conditional expectation, E{XJ^J\XT, ... XQ) andthe conditional variance is ViXj-^^lX-,-,... Xo). In order to derive prediction intervals it is firstnecessary to compute this conditional variance. Assuming independence between the valueof the growth curve function and the error term, the conditional variance can be writtenthus:

V l,A j - ^ jl A J-, . . . , AQJ = \ \l\a, D, C, I -T T )\Af, ..., A.^) -t \ ytj ^.f\.A.j, — t -^Q) \S')

The errors in parameter estimation are denoted by the vector (£„, Ef,, E^)'. Thus, for example,the estimate of the true value a, used in producing forecasts at time T ,h a+ £,. (The terms£,, etc. are time dependent, as are estimates of their moments, but the subscript T is droppedfor the sake of clarity.) It is assumed that this vector has a zero mean and a covariancematrix that is estimated as a by-product of the parameter estimation procedure. This isdenoted thus:

V Obc2

The variance of the error term, e,, is denoted by o; and this is also estimated during theparameter estimation procedure.

An estimation algorithm that produces the necessary covariance information is obviously aprerequisite for the approaches to be outlined. The NAG routine E04GCF, a quasi-Newtonalgorithm for non-linear least squares, satisfies these requirements. The details of the derivationof conditional variances and covariances of the parameter estimates are given in Appendix 1.

An example of the output from this algorithm is given in Table I. The value of a^ is anestimate, but non-linear least squares estimation gives no direct information about the varianceof this estimate. As will be discussed later, a; tends to be a minor contributor to the totalvariance used to construct the prediction intervals. Thus absence of knowledge of theuncertainty in the value of o^ has no discernible practical implications.

416 Journal of Forecasting Vol. 14, Iss. No. 5

Table I. An example of output from NAG estimation routine E04GCF

Data: Adoption of tractors in Spain from 1951 to 1971 (see Mar-Molinero, 1980)Twenty-one observations were used to estimate a simple logistic, and the resulting output was;

abc

=4.456'0.211

49.809

a; = 0.0018

0.20600 -0.00540 -0.23320-0.00540 0.00017 0.02730-0.23320 0.02730 18.18740

Two ways of proceeding from the assumptions already described are considered anddiscussed below.Prediction intervals based on approximated varianceThe variance of the estimated growth curve function can be approximated by a Taylor seriesexpansion (see Stuart and Ord, 1987, p. 324):

db dc da db2.^ SL (j^

da dc+ 2

db dc(7)

Using equation (6) to combine this estimate of the variance in the underlying trend, with anestimate of the variance of the error term, the 100(1 - a )% prediction interval for Xj^ ^ is:

E(X^,, |X^,. . .Xo)±i^;2(V(Xr^|Xr,. . .Xo))"^ (8)

where k^ij is the appropriate value of the random variable that describes the forecast error. Themost common assumption is that this random variable is normal (see Chatfield, 1993).Altemative assumptions (mentioned by Chatfield) are Students' t by Harvey (1989), andGardner (1988) suggests that a"'^^ is used as k^^^ following from the use of Chebychev'sinequality. An example of the application of this approximation to the 'tractors in Spain' data,using the simple logistic as a forecasting model, is given in Table II.

Prediction interval based on the explicit density of estimation errorThe error in estimating the growth curve function is

ej^^=i{a,b,c,t+ T)-f(a-H E^,b+E^,c+ E^.,t+r) (9)

The density function of ej^^ is a function of the errors in the parameter estimates vector,(£„, £t, £^)'. In order to facilitate progress, it is assumed that this vector is a multivariate normalrandom variable. However, in general, the complexity of the growth curve function precludesan analytic derivation of the density function of e^+p For example, when f(a, b, c, t) representa logistic curve, the random variable

constant -Normal

Constant + Normal x Lognormal


Table II. An example of a prediction interval based on the use of anapproximate variance

For a one-step-ahead forecast t = 22All the estimates used are given in Table I.

f(a,b,c,t)= = 3.0181l + c exp(-fer)

= 0.67733a l + c exp(-6f)

f ^ aci exp{-bt) ^ 21 425d '^

^ _o.oi963c (I + c exp(-bt)f

Substituting in equation (7) gives:

...,X,) = 0.0052

X,) + a, = 0.0070

A 90% prediction interval is 3.018 ± 1.64(0.007)''^ i.e. (2.879, 3.156)

The altemative to analytic derivation explored here is a numerical evaluation of the densityfunction. The univariate probability density functions of £„, E^ and E^ are discretized toprobability mass functions, i.e.

PaiyJ = probc^'^ - d/2 <£,<>-„+a/2)

and Pciyc) are defined similarly.The probability of a particular value of the error vector.

Note that the probability mass function for y^ is conditional on y^ and y^, and the probabilitymass function for j ^ , is conditional on the value of y,.

For each value of (c^, e^, £ J ' , the resulting value of fr*r is found from equation (9) and theassociated probability from equation (10). This process is repeated for all values of y^, y^, >'r,and the resulting values and associated probabilities are aggregated to give the probability massfunction of the discretized ef..^, p^ye)-

This function is then combined with the probability mass function of the discretized errorterm to produce a distribution of the overall error, er*r+ ^r^r- From this distribution, predictionintervals can be calculated.

An example of the derivation of a prediction interval for the simple logistic forecast of the'tractors in Spain' data, using this method is given in Table III. Note that the two predictionintervals differ. The explicit density prediction interval is slightly wider than that given by theapproximated variance method in Table II and it is not symmetric about the forecast. If theapproximated variance confidence interval is examined in terms of the cumulative density

418 Journal of Forecasting Vol. 14. Iss. No. 5

Table HI. An example of a prediction interval based on the use of explicit densities

For a one-step ahead forecast t = 22 and giveti the covariance matrix for the tractorsin Spain data from Table I, the following sequence is followed.

Since E^-NiO; a^), all values of j , , and piy^.) can be found.[a , = 4.265]

\Then E,-

All consequent values of y^ and p(yt,) can be found.

All consequent values of y^ and p{yj can be found.

For each triplet of {y^. y/,. yJ, avalue of the estimation error, 22is found from equation (9), withthe associated probability foundfrom equation (10). These valuesare consolidated to form thedensity function of theestimation error, shown here.

= 0.0805]

The error term, £22 is added togive the density of the total error,shown here in cumulative form.Note the positive skewness of theerror distribution.

From this density function, it canbe found that:Prob{total error>0.214) = 0.95Prob(total erTor< -0.102) = 0.05

Thus since f(22) -3.018, yields a 90% prediction interval of (2.916, 3.232)

function shown in Table III, it is found that

Probability (X22<2.879) = 3% and Probability (X22>3.156)= 14%

Thus the explicit density approach suggests that the 90% approximated variance predictioninterval is an 83% prediction interval.

Nigel Meade and Towhidul Islam Frediction Intervals for Growth Curve Forecasts 419

The success of this approach depends on the sensitivity of the prediction intervals to theassumption of multivariate normality, and the consequent assumption of normality of theunivariate marginal and conditional densities.

Prediction intervals generated by bootstrappingBootstrapping is a non-parametric approach to the estimation of distributions which relieson the resampling of residuals. An example of its use in preparing prediction intervals isgiven by McCullough (1994), who describes the effect of different methods ofbootstrapping on the prediction intervals for autoregressive models. A study ofbootstrapping methods is given by Hinkley (1988) and reviews of their application to thecalculation of confidence intervals are presented by Efron and Tibshirani (1986) andDiciccio and Romano (1988).

Three variants of bootstrapping were initially considered; the percentile method, the biasconection method, and the accelerated bias correction method. All the methods evaluated tendedto produce prediction intervals that were too narrow. The simpler percentile method producedthe least poor results and thus was used for the remainder of the study.

The procedure for the percentile method used here is as follows. The appropriate growthcurve model, a version of equation (2), is fitted to the observations (Xo, X,, . . . , X,-). Coefficientestimates a, b, and c are obtained and the residuals (en.£|- •••^^T) are found. This informationforms the basis of the resampling scheme. A set of pseudo-residuals are sampled withreplacement from (£o. e^, ...Jr)- Let this sample be (i'c e*,.... eV). These are used to form a setof pseudo-observations (Xo,X|, ...,XV) where

X' = fid,b,c,t)-hi',

Given this set of pseudo-data, new coefficients are estimated and the resultant forecasts:E(XV.JXV, ...,X;,Xo) for r = l , 2 , ....A' are generated and stored. This procedure isrepeated B times, until the frequency distribution of the forecast at each lead time is consideredto have converged sufficiently. In this case, B = 1000 was used. Prediction intervals are thenderived from these frequency distributions.

This approach was used with the tractors in Spain data. The 90% prediction interval for X,. is(2.938-3.098) using data from /= 1-21 for estimation. This interval is symmetric about theforecast for X22, but very narrow in comparison with the other prediction intervals. According tothe explicit density based c.d.f. in Table III, the 90% bootstrapping prediction interval has aprobability of 61% including the actual observation.

AN EMPIRICAL INVESTIGATION

Two of the reasons quoted for prediction intervals turning out to be too narrow were incorrectmodel identification and a change in the data-gene rating process. The approach adopted herewill be to produce series with known models and stable data-generating processes. Thus, if theprediction intervals calculated perform correctly in these circumstances, the intervals can thenbe used to indicate poor identification or a change in the underlying process.

Simulation procedures for the simple logistic, Gompertz and local logisticIn order to use representative true values for the parameters a, b, and c, for each curve, theresults of a study of telecommunications time series (Meade and Islam, 1993) were used toprovide the ranges shown in Table IV. For each time series generated, the true values of the


Table IV. Ranges of values used for true valuesof the growth curves

Growth curve

Simplelogistic

Gompertz

Locallogistic

Parameter

abco]

. abco]ab

Lowerbound

4.000.022.000.0014.000.022.000.0014.000.020.001

Upperbound

25.000.35

20.000.02

25.000.35

10.000.02

25.000.350.02

four parameters are sampled from uniform distributions between the bounds shown in the table(the NAG routine G05CAF was used here).

Given these values, the time series is generated by substitution in the appropriate growthcurve equations (3), (4) or (5), followed by repeated sampling of c, from a normal distributionwith zero mean and variance a] (using NAG routine G05DDF). Time series varying in lengthfrom 20 to 60 observations were produced.

For a time series generated with N observations, A^- 10 observations are used for modelestimation and the remaining 10 steps ahead been used for building prediction intervals. Forexample, for a data set with 60 observations, 50 observations are used for model estimation andone- to ten-steps-ahead forecasts are produced with associated prediction intervals. For eachvalue of A , 100 different series are generated.

For each series, a set of forecasts from one to ten periods ahead is produced, and 50%, 70%,80%, 90%, 95% and 99% prediction intervals are calculated for each forecast. In each case, it isnoted whether or not the observation falls outside the prediction interval.

It is necessary to test the validity of the prediction intervals and three hypotheses wereconsidered.

Hypothesis 1: The most obvious requirement of a prediction interval is that, when theunderlying growth curve model is correctly identified, the probability of an observation fallingoutside a 100(1 - a )% interval is a. The compliance of the results with this requirement can beformalized by testing the following hypothesis:

HQ: Prob(an observation falls outside a 1(X)(1 - a )% prediction interval) = a

which is tested against a two-sided altemative

Hi: Prob(an observation falls outside a 100(1 - a )% prediction interval)* a.

Under HQ the number of observations lying outside the interval is a binomial random variable.For each combination of N and a, 1000 observations are available to measure the performanceof the prediction interval.

This hypothesis implicitly assumes that the event of an observation falling within a predictioninterval or not is independent of whether previous observations fell within their interval or not.


Tests were carried out for pairs of observations one period apart and there was no evidence of adeparture from independence. Further examinations of sequences of up to five observationswere made and no evidence to refute the hypothesis of independence was found. On this basis,the use of hypothesis 1 (and 2) was considered valid.Hypothesis 2: In order to be able to detect asymmetric prediction intervals, the proportion ofobservations falling below the lower prediction limit is tabulated. For a symmetric predictioninterval, the hypotheses:

HQ: Prob(an observation falls below the 100(1 - a )% prediction limit) = a/2

which is tested against a two-sided altemative

Hi:Prob(an observation falls below the 100(1 - a )% prediction limit)* a/2.

Hypothesis 2 is tested in a similar manner to hypothesis 1.

Hypothesis 3: A further test can be carried out for a set of forecasts from one to ten (say) stepsahead. The number of these observations lying outside their respective prediction intervals, theprediction region, is also a binomial random variable. Thus each of 100 data sets generated foreach value of A provides an observation of a b(10; a ) random variable. The hypothesis that theobservations follow this distribution can be tested by the Kolmogorov-Smimov one-sampletest.

Performance of the prediction intervalsSince there are ten forecasts for each of the 100 series generated, the proportions ofobservations falling outside the 100(1 - a)% interval are based on lOCX) different intervals. Theresults for the prediction interval for simple logistic forecasts based on approximated variancesare summarized in Table V.

A clear pattem emerges, when 25 or more observations are used for estimation, and theprediction intervals behave appropriately. However, when 20 observations are used forestimation, the 50% prediction interval is significantly too wide (at a 5% level). The majority ofintervals are unsatisfactory when 15 or fewer observations are used for estimation(unsatisfactory in the sense that hypotheses 1 and 2 are rejected at the 1% significance level).Hypothesis 3 was dominated by hypothesis 1 that is, it was never rejected unless hypothesis 1was also rejected. For the sake of clarity and since they offer no additional insight, hypothesis 3results are not shown.

The behaviour of similarly constructed prediction intervals for the Gompertz and the locallogistic are shown in Appendix 2. The results are similar to those for the simple logistic. If 20 ormore observations are used for estimation, the prediction intervals are well behaved. As thenumber of observations available for estimation decreases, the performance of the predictionintervals deteriorates to such an extent that the prediction intervals are useless, when estimatedfrom 15 or fewer observations.

The same set of simulated series were used to produce prediction intervals using the explicitdensity approach and the results are shown in Table VI. There is a gentle downward trend in theproportion of observations falling outside the intervals, as in Table V. However, in this case theresults are more encouraging since these proportions do not fall sufficiently to reject hypotheses1 and 2 at a 5% significance level. Similar output for the Gompertz and local logistic is given inAppendix 2, and again the prediction intervals are well behaved.

The results of the application of the percentile bootstrap to the production of predictionintervals are disappointing, and they are shown in Table VII. All the intervals are too narrow and


Table V. Proportion (%) of observations of simple logistic outside the predictionintervals, using approximated variance. Proportion below lower limit shown in parentheses

Number ofobservations used

for estimation

50

30

25

20

15

10

50%

49.8(24.0)50.1

(24,4)48.5

(24.0)46.3^

(22.0-)43.7"

(20.0")40.0"

(17.0")

70%

29.6(14.0)30.4

(15.0)28.8

(13.8)27.6

(12.0")21.3"(7.4")19.0"(6.5")

Prediction

80%

19.8(10.0)20.0

(10.0)19.7(9.8)18.6(9.0)13.6"(5.0")10.3"(4.6")

interval

90%

10.1(5.0)10.0(4.8)10.4(5.1)9.5

(4.5)6.2"

(2.2")3.9"

(1.3")

95%

5.0(2.4)5.0

(2,6)4.8

(2.4)4.5

(2.1)3.5^

(1.5)1.2"

(0.4")

99%

1.1(0.5)1.2

(0.6)1.2

(0.5)0.8

(0.4)0.8

(0.3)0.2"

(0.0")

The hypothesis described in the text can be rejected at ""5%, ^1%.

Table VI. Proportion of observations of simple logistic outside the predictionintervals, using explicit densities. Proportion below lower limit shown in parentheses


for estimation

50

30

25

20

15

10

50%

51.1(26.0)50.2

(25.0)49.0

(24.6)49.6

(25.0)49.6

(24.9)47.2

(23.7)

70%

30.5(15.2)30.4

(15.1)29.4

(14.8)29.4

(14.6)28.9

(14.4)27.5

(13.8)

Prediction

80%

21.2(11.0)20.8

(10.4)20.8

(10.6)20.0

(10.0)20.1

(10.0)18.5(9.5)

interval

90%

10.6(5.4)10.4(5.3)9.8

(4.8)9.8

(4.9)9.8

(5.0)8.2

(4.2)

95%

5.2(2.6)5.0

(2.4)5.0

(2.4)4.8

(2.4)4.8

(2.5)4.1

(2.1)

99%

1.1(0.5)1.1

(0.5)0.9

(0.5)0.9

(0.4)0.8

(0.4)0.6

(0.3)

would give a severe overestitnate of the reliability of the forecast. The performance ofprediction intervals derived from bootstrapping was so poor that no further work was carried outwith this method. This decision is in accord with ChatHeld's (1994) comments on otherresearchers' experience with resampling methods used in time-series applications.

The explicit density approach generates reliable prediction intervals for data series with asfew as ten observations. Thus, although it is computationally more expensive than theapproximated variance approach, it is worth the extra effort unless the series underconsideration has more than 25 observations for estimation. Given the context of the use ofgrowth curves in medium- to long-term forecasting, computation time is of negUgiblerelevance.


Table VII. Proportion of observations of simple logistic outside the predictionintervals, using percentile bootstrapping. Proportion below lower limit shown inparentheses


for estimation

50

30

25

20

15

10

50%

64.8(30.2)63.2

(30.1)62.0

(30.0)63.0

(29.2)65.0

(31.5)65.8

(31.2)

70%

55,3(25.0)52.8

(24.6)54.4

(26.3)49.6

(23.5)51.0

(24.6)48.6

(22.4)

Prediction interval

80%

44.2(19.2)45.3

(22.3)44.8

(22.0)38.6

(17.8)35.6

(15.0)37.5

(17.4)

90%

34.6(16.3)33.8

(15.8)28.8

(13.6)28.5

(13.1)29.5

(13.2)30.2

(14.2)

95%

27.4(12.3)26.8

(12.2)24.4

(11.3)23.5

(10.3)22.0

(10.5)23.8

(10.8)

99%

17.5(7.3)16.8(7.5)17.0(8.0)14.5(6.5)13.0(6.2)12.6(5.5)

Null hypotheses 1 and 2 are rejected in every case shown.

An explanation of the difference in performance between the explicit density method andbootstrapping lies in the different emphasis placed on the residuals. The nature of growthcurves means that a given curve will tend to overfit a set of data. This overfitting leads to anunderestimate of a; and, consequently, the residuals are misleadingly small in magnitude.The estimation of the parameters a, b, c does, however, capture information about theuncertainty in and correlation between their estimates in the covariance matrix,V[c^, Eh, e j ' . The explicit density method draws on the covariance matrix as its main sourceof information, and the error variance is added at the final stage. Bootstrapping dependsheavily on the values of the residuals and thus is disadvantaged when these values areunderestimated. The worked examples using the tractors in Spain data showed that explicitdensity prediction intervals could be asymmetric. As will be seen in the next section, thisadditional flexibility lends greater credibility to explicit density-based prediction intervals, evenfor longer time series.

Prediction intervals for real data

With real data, the assurance that the choice of model is correct, which was total in the previoussection, is considerably reduced. Intervals are prepared for two data sets previously discussed inthe literature, and in these cases the choice of model is non-controversial. However the stabilityof the data-generating process is obviously not guaranteed. First, a comparatively short series isexamined, where the explicit density method alone is expected to produce reliable intervals.Second, a longer series is analysed where, a priori, both methods should produce reliableintervals.

Prediction intervals for the 'tractors in Spain' data, prepared following on from theexamples given in Tables II and III, are shown in Figure 1. The full data set with one- toten-steps-ahead forecasts, with time origin 1971 using the simple logistic, are shown inFigure l(a) . Twenty-one observations are used for parameter estimation. The predictionintervals from the approximated variance method are shown in Figure l (b) , those from the

424 Journal of Forecasting VoL 14. Iss. No. 5

(TimtOngini n ) » % PrkKlun I

(a)

wM RSH Pnaieiion I

(c)

l«7t 1976 1*7)

(b)

ind S M PiMKOon MwvM

1174 1*71 isn

(d)

Figure 1. Forecasts and prediction intervals for 'tractors in Spain'

explicit density method in Figure l(c), those from bootstrapping in Figure l(d). Thewidths of all three sets of prediction intervals increase with lead time, as one mightexpect. The bootstrapping interval is comparatively narrow. The observations leave the50% prediction interval by 1973 and reach the 95% prediction interval by 1975. Theobservations remain within the 95% regions based on both approximated variance andexplicit density. The asymmetry of the latter region implies a greater probability ofunderforecasting.

For interest, a local logistic forecast with prediction intervals based on explicit densitiesis shown in Figure 2(a). The forecast is better than that from the simple logistic and thedata remain within the 50% prediction interval. Again note the asymmetry of theinterval.

Gamerman and Migon (1991) produced a forecast and a 1.5 standard deviation (equivalentto 87%) prediction interval for the tractor data, using the approach described in Migon andGamerman (1993). They produced a set of one-year-ahead prediction intervals and a set ofone- to six-years-ahead prediction intervals. In Figure 2(b) their intervals are reproducedalongside corresponding results from the simple logistic and the local logistic models using theexplicit density method. The rate at which their interval widens for the one- to six-years-aheadcase is a function of a discount factor (which was set to 0.9). The local logistic predictionintervals are a little wider than those for the simple logistic, and the Bayesian prediction


50% and 95% Prediction IntervalsLocal Logistic (Explicit Density)

6.0

5.5 -

4.5

4.0

)5 3.0 -

2.5 -

2.0

1970 1972 1974 1976 1978

Year(a)

1980 1982

Approximate 87% prediction intervals

6.0

5.5-

^5.0

I 4.5

| 4 . 0 -

^3.5

1=3.0 •

2.5 •

2.0 -i-

1970

Bayesian

Explicit density - local

Explicit density - simpi*

On« step aheadprediction intervals

One U six step aheadprediction jntervats

1972 1974 1976Year

(b)

1978 1980 1982

Figure 2. (a) Local logistic forecast with predictioti intervals for the 'tractors in Spain' data (b) 87%prediction intervals for 'tractors in Spain', comparing Gamerman and Migon's Bayesian model withintervals generated using the explicit density method for the simple and local logistic models


1870

Figure 3. Forecast and prediction intervals of titanium dioxide

intervals are a little wider still for the one-step-ahead forecasts. However, for the one- to six-steps-ahead forecasts the Bayesian prediction intervals become much wider than those of thesimple or local logistic. For both the local and simple logistic prediction intervals,approximately 92% of the variation was explained by uncertainty in parameter estimation.Since no details of the composition of the prediction variance for the Bayesian model weregiven, we can only note rather than explain the discrepancy in the sizes of the predictionintervals.

Harrison and Pearce (1972) proposed that a Gompertz curve be used to forecast the USconsumption of titanium dioxide. A set of one- to ten-steps-ahead forecasts and predictionintervals with a 1958 time origin have been prepared. Thirty-four observations are used forestimation. The data and the forecast is shown in Figure 3 (a). The behaviour of the predictionintervals resulting from the different methods is broadly similar to the previous example, andbootstrapping was not considered here. The symmetric prediction intervals based onapproximated variance are shown in Figure 3(b). The upper 95% limit is breached in 1966 and1968. The asymmetric prediction intervals based on explicit density are shown in Figure 3(c),and the 95% limits are never breached. The explicit density prediction intervals are slightlywider, apart from the ten-steps-ahead interval. However, it is the asymmetry, the implicationthat underforecasting is the more likely error, that explains why the data remain within the 95%region.


Bearing in mind the provisos mentioned at the beginning of this section, the intervals givenby the explicit density method look more plausible, even for the longer series, than theapproximated variance method.

SUMMARY AND CONCLUSIONS

Three approaches to producing prediction intervals for growth curve forecasts have beenconsidered. From trials using simulated data with a known model structure, the followingconclusions may be drawn.

The approximated variance approach is quick to compute and performs adequately for longerdata series. However, it performs poorly for short data series (less than 20 observations). TheTaylor series approximation underlying this approach appears to become too coarse with shortseries, leading to overestimation of the variance, as evidenced by the overwide predictionintervals.

The explicit density method is comparatively computationally intensive but produces well-behaved prediction intervals that perform well for series as short as ten observations. Thus theunderlying assumption of multivariate normality for the parameter estimates appears wellfounded.

Prediction intervals based on the bootstrapping methods considered were far too narrow andthis approach was considered unrewarding in comparison with the other two approaches. Thedemonstrations of the prediction intervals on actual data showed that the explicit densityapproach produces more plausible prediction intervals, mainly due to the asymmetry of theprediction region.

The explicit density method is of general applicability where a least squares algorithm isused for estimation, so the approach discussed here can be extended easily to other growthcurves.

ACKNOWLEDGEMENTS

An early draft of this paper was presented at the 14th International Symposium on Forecasting,Stockholm, 12-15 June 1994. The authors are grateful for helpful comments from S.Bretschneider and R. Bewley.

APPENDIX 1: DETAILS OF THE COMPUTATION OF THE PARAMETER ESTIMATIONCOVARIANCE MATRIX

For an estimation problem, where the objective is to find a set of parameters (a,, 02. •-•• ^minimize the sum of squares S(a), where

The covariance between the estimates of o, and a^ is

N-m


where H is the approximate Hessian matrix (H = 2J^J) and the Jacobian matrix / is composedof the first partial derivatives at the optimal point where (ai,a2, ...,a^) minimizes the sum ofsquares, i.e.

APPENDIX 2: FURTHER RESULTS

Table AI. Proportion of observatiotis of Gompertz outside the prediction intervals, usingapproximated variance


for estimation

50

30

25

20

15

10

50%

49.5(24.5)48.8

(24.4)48.3

(24.0)47.0

(23.0)41.6"

(19.5")39.7"

(18.2")

70%

29.7(14.7)29.6

(14.6)30.6

(15.2)28.4

(14.1)22.9"

(10.0")19.8'(8.8")

Prediction interval

80%

19.7(9.5)19.2(9.4)19.1(9.5)18.8(9.2)14.3"(7.0")14.1"(6.7")

90%

9.7(4.7)9.6

(4.7)9.7

(4.8)8.8

(4.4)7.4"

(3.4")7.1"

(3.0")

95%

4.9(2.4)5.0

(2.5)5.1

(2.6)4.8

(2.3)2.4"

(1.1")2.9"

(1.3")

99%

0.8(0.4)0.7

(0.3)1.0

(0.5)1.0

(0.4)0.3

(0.1)0.4'

(0.2)

The hypothesis described in the text can be rejected at '5%, " 1 % .

Table All. Proportion of observations of local logistic outside the prediction intervals,using approximated variance


for estimation

50

30

25

20

15

10

50%

50.9(25.2)49.8

(25.0)50.8

(24.8)48.8

(24.1)43.8"

(20.1")39.8"

(18.8")

70%

28.9(24.2)30.2

(15.2)29.1

(13.8)28.8

(14.2)25.4"

(12.3-)24.5"

(11-6")

Prediction

80%

19.6(9.6)20.0

(10.0)20.3

(lO.O)19.9(9.0)15.8"(7.3")15.7"(7.1")

interval

90%

9.6(4.8)10.2(4.9)10.1(4.8)9.3

(4.4)7.5"

(3.6^)7.6"

(3.2")

95%

4.8(2.4)5.0

(2.4)4.5

(2.2)4.5

(2.1)2.6"

(1.2")2.7"

(1.1")

99%

1.2(0.6)1.2

(0.6)0.9

(0.4)1.2

(0.5)0.7

(0.3)0.4=

(O.r)

'As Table AI.


Table AIII. Propottion of observations of Gompertz outside the prediction intervals,using explicit densities


for estimation

50

30

25

20

15

10

50%

50.2(25.0)50.2

(25.3)49.6

(24.8)49.8

(25.0)49.6

(24.4)47.2

(23.5)

70%

30.2(15.2)30.2

(15.2)29.8

(15.0)29.9

(15.0)29.4

(14.8)28.5

(14.3)

Prediction interval

80%

20.0(10,0)20.2

(10.2)20.0

(10.0)19.6(9.9)19.6(9.9)18.8(9.5)

90%

10.0(5.0)10.0(5.1)10.1(5.1)9.8

(5.0)9.8

(5.0)9.2

(4.7)

95%

5.0(2.5)4.9

(2.5)4.9

(2.5)4.8

(2.4)4.8

(2.4)4.6

(2.3)

99%

1.1(0.6)0.9

(0.4)0.9

(0.4)0.8

(0.4)0.8

(0.4)0.8

(0.4)

The hypothesis described in the text cannot be rejected at 5%.

Table AIV. Proportion of observations of local logistic ouLside the prediction intervals,using explicit densities


for estitnation

50

30

25

20

15

iO

50%

50.4(25.2)50.2

(25.0)50.2

(25.3)49.8

(25.0)48.8

(24.5)46.8^

(23.8)

70%

30.4(15.3)30.2

(15.2)29.8

(14.8)29.6

(14.8)29.0

(14.9)28.6

(14.6)

Prediction

80%

20.8(10.5)20.4

(10.3)20

(10.1)19.8(9.8)19.4(9.8)18.7(9.5)

interval

90%

10.2(5.1)10.0(5.0)9.8

(5.0)9.8

(4.9)9.6

(4.9)8.8

(4.5)

95%

5.0(2.5)4.9

(2.4)4.9

(2.5)4.8

(2.4)4.6

(2.3)4.3

(2.3)

99%

1.2(0.6)1.0

(0.5)l . l

(0.6)0.8

(0.4)0.8

(0.5)0.6

(0.3)

'The hypothesis described in the text can be rejected at 5%.

REFERENCES

Brand, R. J.. Pinnock, D. E. and Jackson, K. L., 'Large sample confidence bands for the logistic responsecurve and its inverse', The American Statistician, 27 (1973), 157-60.

Chatfield, C , 'Calculating interval forecasts'. Journal of Business & Economic Statistics, 11 (1993),121-35.

Chatfield, C , 'Model uncertainty, data mining and statistical inference'. Statistics Report 94:01 (1994),University of Bath.

Dadkah, K. M.. 'Confidence interval prediction for iogarithtnic models'. Review of Economics andStatistics, 66 (1984), 527-8.

Diciccio, T. J. and Romano, J. P., 'A review of bootstrap confidence intervals'. Journal of the RoyalStatistical Society B, 50 (1988), 338-54.


Efron, B. and Tibshirani, R., 'Bootstrap methods for standard errors, confidence intervals and othermeasures of statistical accuracy'. Statistical Science, 1 (1986), 54-77.

Gamerman, D. and Migon, H. S., 'Tractors in Spain—a dynamic reanalysis'. Journal of the OperationalResearch Society, 42 (1991), 119-24.

Gardner, E. S. Jr, 'A simple method of computing prediction intervals for time series forecasts',Management Science, 34 (1988), 541-6.

Guerrero, V. M., 'Time series analysis supported by power transformation'. Journal of Forecasting, 12(1993), 37-48.

HatTison, P. J. and Pearce, S. F., 'The use of trend curves as an aid to market forecasting'. IndustrialMarketing Management, 2 (1972), 149-70.

Harvey, A. C , Forecasting. Structural Time Series Models and the Kalman Filter, Cambridge:Cambridge University Press, 1989.

Hauck, W. W., "A note on confidence bands for the logistic response curve'. The American Statistician,37 (1983), 158-60.

Hinkley, D. V., 'Bootstrap methods'. Journal of the Royal Statistical Society. B, 50 (1988), 321-37.Lee, J. C , Lu, K. W. and Homg, S. C , 'Technological forecasting with non-linear models'. Journal of

Forecasting, U (1992), 195-206.Mar-Molinero, C , 'Tractors in Spain—a logistic analysis'. Journal of the Operational Research Society,

31 (1980), 141-52.McCullough, B. D.. 'Bootstrapping forecast intervals: applications to AR(;;) models'. Journal of

Forecasting, 13 (1994), 51-66.Meade, N., 'The use of growth curves in forecasting market development—a review and appraisal'.

Journal of Forecasting, 3 (1984), 429-51.Meade, N., 'Forecasting using growth curves—an adaptive approach', Joumal of the Operational

Research Society, 36 (1985), 1103-15.Meade. N., 'A modified logistic model applied to human populations'. Journal of the Royal Statistical

Society. A, ISl (1988), 491-8.Meade, N. and Islam, T., 'Forecasting with growth curves: an empirical comparison'. Working Paper, The

Management School, Imperial College. Exhibition Road, London SW7 2PG (1993).Migon, H. S. and Gamerman. D., 'Generalized exponential growth models—a Bayesian approach'.

Journal of Forecasting, 12 (1993), 573-84.Pankratz, A. and Dudley, U., 'Forecasting of power transformed series'. Journal of Forecasting, 6

(1987), 239-48.Stuart, A. and Ord. K. J., Kendall's Advanced Theory of Statistics: Distrihution Theory, 5th edition. Vol.

1. London: Griffin. 1987.

Authors' biographies:Nigel Meade is a Senior Lecturer in Operations Research and Systems at the Management School,Imperial College, University of London. His research interests are statistical model building in generaland applied time-series analysis and forecasting in particular. He is cutTently Chairman of the UK ORSociety Forecasting Study Group.Towhidul Islam is a telecommunications engineer and worked for a telephone company for eight years.He has an MBA from Dhaka University and is currently a research student at the Management School.Imperial College.

Authors' address:Nigel Meade and Towhidul Islam, The Management School, Imperial College, 52-53 Prince's Gate.Exhibition Road, London SW7 2PG.

Documents

Prediction intervals for growth curve forecasts