02-Demand Forecasting - Prof. Karamouz

8/3/2019 02-Demand Forecasting - Prof. Karamouz

1/89

Water Demand Management WorkshopRegional Center of Urban Water Management (RCUWM- Tehran)

Demand Forecasting

Mohammad KaramouzProfessor, Amir Kabir University (TehranPolytechnic)

Banafsheh ZahraieAssistant Professor, Tehran University

September 6-17, 2003Tehran, Iran


2/89

Points to be AddressedData and Information Needed for Long LeadForecasting of the Regional Water DemandMunicipal Water DemandPopulation Forecasting Methods: Short-term

and Long-termTime Series Modeling: Basic Steps

Agricultural Water DemandsClimate Signals: Prediction of Dry and WetSpells

Environmental Water Demands


3/89

Projected trends of water use serve as measures to

guide planners as they propose:

New water resources facilities Modification of existing systems New or revised operating rules Regulatory changes Revised laws Organizational changes Research Projects

Why is it important to forecast the demands?


4/89

Past histories of water use are valuableinformation in making estimates of future wateruse.

This information indicates the principle factorsin determining the future water use.

What types of information is needed?


5/89

Population projections based on demographicstudies and studies done by the agencies responsiblefor multiple-sector investment decisions

Distribution of urban and rural population amongsubregions

Gross national product (GNP) Expected use of brackish or ocean water

Data and Information Needed for LongLead Forecasting of the Regional Water Demand


6/89

Projected outputs of agricultural, mining, electric power, and major manufacturing sectors for determining the regional distribution of activities

based on the projected GNP

Projected rates of per capita water use based ontechnological advancements and relative shareof instream and offstream uses

The Needed Data and Information for

Long Lead Forecasting of the Regional Water Demand


7/89

Municipal Water Demand

Disaggregate Estimation of Water Us e

Precise estimation of municipal water use can beobtained by breaking down the total delivery of water to urban areas into a number of classes of water use and determining separate average ratesof water use for each class.

Water use within some homogenous sectors is lessvariable compared to the total water use, therefore,greater accuracy in water use estimation can beobtained.


8/89


1. Domestic Washing and cooking Toilets Bath and/or shower

Laundry House cleaning Yard irrigation Swimming pool Car washing Other personal uses (hobbies, etc.)


9/89

Municipal Water Demand2. Public services

Public swimming pools Governmental agencies and private firms Educational services (such as schools, universities)

Firefighting Irrigation of parks, golf courses, etc. Health services (such as hospitals) Public baths, public toilets, etc. Cultural public services ( such as libraries and museums ) Street cleaning and sewer washing Entertainment and sport complexes ( such as cinemas ) Food and beverage services Accommodation services Barber shops and beauty parlors


10/89


3. Small industries (such as laundry stores)4. Construction and public works5. Water losses

Leakage from pipes, valves, meters, etc. Evaporation in open reservoirs Overflow of reservoirs Defective elements of a water distribution network,

such as cracked reservoirs, flow back through one-way valves and pumps, etc.

Loss in production process (cooling, pumping, etc.)

6. TransportationTaxies, buses, and other conveyances stations Ports and airports Railways (stations and workshops)


11/89

Parameters Affecting Municipal Water Demand

1. Population Changes

2. Climate Variations3. Hydraulic Characteristics of the Water

Distribution Network

4. Price and Economic Incentives5. Living Standards

Demand= f (population, price, standards, pastuse, etc.) + g(climate variations, )


12/89

1989 Weber -0.06 / -0.23winter US1982 &1967 Howeet al-0.57 / -0.86summer US

east 1982 &1967 Howeet al-0.43 / -0.52summer USwest 1982& 1967Howe et al

Price Elasticity: The percent of decrease inquantity demanded due to 1 percent increase

in price


13/89

Population Forecasting Methods

Graphical

Mathematical

Ratio and Correlation

Component Methods


14/89

Short-term Estimates for

1 to 10 Years

Graphical Extension Method1. Plot the population of the past census

years against time2. Sketch a curve that fits the data

3. Extend this curve into the future toobtain the projected population


15/89


1 to 10 Years

0

20

40

60

80

100

120

1 8 6 0

1 8 7 0

1 8 8 0

1 8 9 0

1 9 0 0

1 9 1 0

1 9 2 0

1 9 3 0

1 9 4 0

1 9 5 0

1 9 6 0

1 9 7 0

1 9 8 0

1 9 9 0

2 0 0 0

P o p u

l a t i o n

( T h o u s a n

d s )


16/89


1 to 10 Years

y = -0.0551x 3 + 1.1752x 2 + 0.8758x + 13.303

R2 = 0.9991

0

20

40

60

80

100

120

1 8 6 0

1 8 7 0

1 8 8 0

1 8 9 0

1 9 0 0

1 9 1 0

1 9 2 0

1 9 3 0

1 9 4 0

1 9 5 0

1 9 6 0

1 9 7 0

1 9 8 0

1 9 9 0

2 0 0 0

P o p u l a

t i o n

( T h o u s a n

d s

)


17/89


1 to 10 Years

Arithmetic Growth Method: This methodconsiders that the same population increase takes

place in a given period.

dP/dt = K a


18/89


19/89


1 to 10 Years

Declining Growth Rate Method: This methodconsiders that the city has a saturation populationand the rate of growth becomes less as thepopulation approaches the saturation level.

dP/dt = Ka(P

sat-P)

Ka=(-1/ t) . ln[( P sat -P 2) /(P sat -P 1)]


20/89

a

b

c d

Time (years)

P o p u

l a t i o n

Saturation Population, P sat

Geometric GrowthdP/dt P

Arithmetic GrowthdP/dt 1

Declining Rate of GrowthdP/dt (P sat -P)


21/89

Long-term Forecasting

for 10 to 50 years or more

Graphical Comparison Method1. Several larger cities in the vicinity are selected

whose earlier growth exhibited characteristicssimilar to those of the study area.

2. The population-time curves should then beplotted for the selected cities and the study area.

3. Lines parallel to the growth rate of the selectedcities shows a range of future growth.


22/89

Years

Population

C i t y A

C i t y

C

C i t y B

C i t y D


23/89



Mathematical Logistic Curve Method: Thismethod is suitable for the study of large

population centers such as large cities, states, or nations.

On the basis of the study of growth curve, certainmathematical equations of an empirical curveconforming to this shape (S-shape) is proposed.


24/89

Time (years)

P o p u

l a t i o n

Saturation Population, P sat

dP/dtdP/dt == P sat /(1 + a e bt)


25/89



Ratio and Correlation Method: Thismethod is suitable for an area, which is a part of

a region, state, nation, or larger area. It is assumed that the growth of the smaller areahas some relation to the growth of the larger area.


26/89



Component Method: In this method, populationchange is disaggregated to the changes due to:

1. Birth B = K 1 P 0 t2. Death D = K 2 P 0 t

3. Migration(M)P t = P 0 + B D M


27/89

In recent years a great deal of effort has been

devoted to improve the followings:

1. Understanding the stochastic nature of hydrologic variables

2. Modeling procedures

3. Developing new statistical models4. Parameter estimation techniques

5. Model evaluation and fitness tests

Statistical Forecast Models


28/89

A time series is a sequence of values arranged in their order of occurrence in time. A process is a mathematical description of the behavior

of a phenomenon in one or more dimensions in spaceand/or time.

Because all hydrologic phenomena change in space or time, they are hydrologic processes .

If a process contains a random component, it is a stochastic process , which is a family of random

variables, defined on a probability space.

Stochastic processes are subdivided into stationary and non-stationary.


29/89

A stochastic process is stationary if theexpected values of statistical descriptors donot change over time.

If a time series is stationary, the series shouldbe divided into a number of no-overlappingsub-series and the expected values of statistical descriptors of each series should bethe same for each of the subseries.


30/89

Hydrologic variables are mostly nonstationary

due to the following variations that are theresult of natural and human activities:

1. Trend3. Jump

4. Periodicity5. Randomness

Components of the Hydrologic Variables


31/89

Trend is a unidirectional gradual change (increasing or

decreasing) in the average value of the variable. Changes in nature caused by human activities are the

main reason for the over-several-years trends.

Trends are usually smooth, and we should be able torepresent it by a continuous and differentiable function of time.

Trend is usually considered to be deterministic and it canbe modeled by linear or polynomial functions:

Linear function Polynomial function Power functions


32/89

0

1000

2000

3000

4000

5000

6000

7000

1 20 39 58 77 96 115 134 153 172 191 210 229


33/89

Jump is a sudden positive or negativechange in the observed values.

Human activities and natural disruptions arethe main reasons for jumps in time series.

0

1000

2000

3000

4000

5000

6000

7000

8000

9000

1 20 39 58 77 96 115 134 153 172 191 210 229


34/89

Periodicity represents cyclic variations in a

time series. These variations are repetitiveover fixed intervals of time.

0

500

1000

1500

2000

2500

3000

1 20 39

Randomness represents variations due to


35/89

Randomness represents variations due tothe uncertain nature of the stochastic

process.The random component of time series can

be classified as autoregressive or purely rando m.

0

0.2

0.4

0.6

0.8

1

1.2

1 51 101 151 201 251


36/89

TIME SERIES MODELING:

BASIC STEPSSTEP 1. DATA PREPARATION

The main tasks in data preparation phase can besummarized as:1. Trends removal2. Removal of outlying observations (jumps)3. Removal of periodicity

4. Fit a well-known distribution to the data by applyingproper transformations if needed.


37/89

TREND REMOVAL


38/89

TREND REMOVAL


39/89


40/89

REMOVING TREND AND SEASONALITY:

DIFFERENCING

0

1000

2000

3000

4000

5000

6000

7000

1 20 39 58 77 96 115 134 153 172 191 210 229

-2000

-1500

-1000

-500

0

500

1000

1500

2000

1 20 39 58 77 96 115 134 153 172 191 210 229

APPLYING TRANSFORMATIONS


41/89

APPLYING TRANSFORMATIONS

0.0)ln(

0.00.1)(

12)(

111

2),(

2

1

1

21

=+=

>+=

t t

t t

I m I

m I I

In most of the parameter estimation methods, itis assumed that the time series probabilitydistribution is normal, but in many cases thetime series do not follow normal distribution,are asymmetrically distributed, or arebounded by zero.

BOX-COX Transformation:

BOX COX TRANSFORMATION


42/89

BOX-COX TRANSFORMATION

BOX COX TRANSFORMATION


43/89

BOX-COX TRANSFORMATION

Original Series

TransformedSeries

Forecasted Series

Identification of Forecast Model Composition


44/89

Identification of Forecast Model Compositionfor Stochastic Components of Demand

The next step is to decide upon the The next step is to decide upon theuse of ause of a univariateunivariate oror multivariatemultivariate model,model,

or a combination, withor a combination, with desegregationdesegregationmmodels.odels.

This decision can be made based on the This decision can be made based on the

characteristics of the water resources systemcharacteristics of the water resources systemand existing informationand existing information.


45/89

Univariate Models:

Autoregressive(AR) models These models incorporate the correlation between time These models incorporate the correlation between timesequences of variablessequences of variables These are simple models and their development These are simple models and their development

goes back to the application of Markov lag goes back to the application of Markov lag --11models.models.

The basic form of AR(p) is as follows: The basic form of AR(p) is as follows:

t jt

p

j jt Z Z +=

= 1


46/89

Autoregressive Moving Average (ARMA)

models Time series could be better forecasted by adding aTime series could be better forecasted by adding a

moving average component to the AR models.moving average component to the AR models.

The combination of an autoregressive model of The combination of an autoregressive model of order p and a moving average model of order qorder p and a moving average model of order qmakes an ARMA(p,q) model, which is formulatedmakes an ARMA(p,q) model, which is formulated

as follows:as follows:

qt qt t pt pt t Z Z Z +++= ......

1111

Autoregressive Integrated Moving


47/89

Autoregressive Integrated MovingAverage (ARIMA) Models

The ARIMA models are suitable for the data that have The ARIMA models are suitable for the data that havetwo basic characteristics:two basic characteristics:

1. No apparent deviation from1. No apparent deviation from stationaritystationarity2. Rapidly decreasing autocorrelation function2. Rapidly decreasing autocorrelation function

If these conditions are not met by a time series, a properIf these conditions are not met by a time series, a propertransformation should be performed to generate timetransformation should be performed to generate timeseries satisfying the above conditions. This is usuallyseries satisfying the above conditions. This is usuallybeen achieved by differencing, satisfying the essence of been achieved by differencing, satisfying the essence of

ARIMA models. ARIMA models.


48/89

Simple and Multiplicative ARIMA

( )( )( ) ( ) ( )( ) t wt d Dww B B z B B B B = 11

( )( ) ( )t t d B z B B =1

( )p

p B B B B = ....12

21

( ) qq B B B B = ....1 221

Formulation of simple ARIMA is as follows;

Where:

Formulation of multiplicative ARIMA model is as follows;


49/89

Regression based models hasbeen widely used for demand

forecasting:

1-Linear Regression2- Multiple Regression

Regression based ModelsRegression based Models


50/89

Linear Regression Using Least Square

The method of least squares finds that particular linewhere the aggregate deviation of data points above or below it is minimized.

Rather than measuring it's separation in terms of the

physical distance, the procedure is instead uponvertical deviations, which are then squared. This notonly eliminates in measuring perpendicular line

segments, but it provides summary statistics havingdesirable properties

ii bX a X Y +=)(


51/89

Confidence Interval for the Slope of the Model

The following experession is used to construct a

confidence interval of true slope B:

Which t is t-student distribution with n-2 degree of freedom

222 )(1 =

X n

X

S t b B YX

)%1(100


52/89


53/89


54/89

Agricultural Water Demand

Agricultural water demands change from year to year andmonth to month. The parameters affecting agricultural water demands can be summarized as follows:

Crop mi x Irrigated and dry-land farmin g

Period and sequence of croppin g Physical characteristics of the water transfer and irrigationsystem s

Market price s

Climate variatio n Policies related to pricing, importing, and exporting of

agricultural prod-uct s.

Variations of Irrigation Demand Due to


55/89

Variations of Irrigation Demand Due toClimatic Variation

850

900

950

1000

1050

1100

1150

1200

1 3 4 9 1 3 5 1 1 3 5 3 1 3 5 5 1 3 5 7 1 3 5 9 1 3 6 1 1 3 6 3 1 3 6 5 1 3 6 7 1 3 6 9 1 3 7 1 1 3 7 3

)

(

A g r i c u l

t u r a

l W a

t e r

D e m a n d s

( M C M / Y e a r )

A g r i c u l

t u r a

l W a t e r

D e m a n d s

( M C M / Y e a r )


56/89

y = -1.7712x 3 + 30.219x 2 - 167.85x + 493.9R2 = 0.9929

y = 0.3584x 3 - 15.639x 2 + 216.7x - 730.72R2 = 0.8292

y = 5.266x 3 - 354.89x 2 + 7938.5x - 58723R2 = 0.94

y = 0.4417x 3 - 41.429x 2 + 1276x - 12693R2 = 0.9798

0

50

100

150

200

250

300

350

400

3 8 13 18 23 28 33 38

time (year)

rainfall(mm)

3-year moving average of the entire regionand fitted polynomials to dry and wet spells


57/89

E i i f C W D d


58/89

Estimation of Crop Water Demand

Kc

Crop DevelopmentPeriod

InitialInitialStageStage

RapidRapid

GrowthGrowthPeriodPeriod

Middle StageMiddle Stage

of Cropof CropDevelopmentDevelopment

Final StageFinal Stage


59/89

Climate Signals:Prediction of Dry and Wet Spells

A Case Study of SeasonalStreamflow Forecasting

Using ENSO Climate Signals


60/89

Major Climate Signals

ENSO (El-Nino SouthernOscillation)

NAO (North Atlantic Oscillation) PDO (Pacific Decadal Oscillation)

Other Signals


61/89

ENSO Climate Signal

ENSO climate signal has two followingphases: Warm phase (El-Nino) Cold phase (La-Nina)

El-Nino refers to appearance, aroundChristmas, of a warm ocean current off the South American coast, adjacent to

Ecuador and extending into Peruvianwaters.

ENSO Climate Signal


62/89

ENSO Climate Signal

El-Nino

ENSO Climate Signal


63/89

ENSO Climate Signal

La-Nina

ENSO Cli t Si l


64/89

ENSO Climate Signal:

Prediction of Dry and Wet Spells Precipitation over 20-30 percent of the lands are

affected by ENSO (El-Nino Southern Oscillation) events Effect of ENSO on the average global precipitation is

estimated as 15-25 percent. ENSO events have effects with delay on summer

precipitation in East Asia. El-Nino Events usually are followed by:

Summer precipitation less than normal in India andNorth Australia

Winter precipitation higher than normal in southeastAsia



65/89


Prediction of Dry and Wet Spells Results of studies on North Indian Ocean shows

that sea surface water temperature has beensignificantly higher than normal level in the periodof El-Nino events.

In El-Nino years, highest sea surface temperatures

have been occurred in Persian Golf compare toother water bodies around the world


66/89

North Atlantic Oscillation

A sea saw of atmospheric mass which alternated

between the polar and subtropical regions. Changes in the mass and pressure fields leads to

variability in the strength and pathway of stormsystems crossing the Atlantic from the U.S. Eastcoast to Europe.

The NAO is most noticeable during the winterseason (November-April) with maximum amplitudeand persistence in the Atlantic sector.


67/89


68/89


69/89


70/89


71/89

Pacific Decadal Oscillation PDO refers to a to a numerical climate indexbased on sea surface temperatures in a particular

region of the North Pacific, which has aninterannual signature

The warm phase of the PDO (positive numericalindex value) has similar effects in the PacificNorthwest to those experienced in warm ENSO

years, and the effects associated with the cold- phase of PDO resemble those associated with

cold-phase ENSO


72/89

Pacific Decadal Oscillation PDO oscillate with a characteristic period on the

order of 25-50 years. The observed bimodal nature of the PDO on

decadal time scales and the typical persistence of a dominate phase of the PDO for several decades,allows the PDO to be included in real-timeforecasting schemes in a useful manner


73/89

A Case Study: Salt River Basin in Arizona

The Salt River basin is located in the central part of Arizona.

Four dams are located inthis basin.

A total storage capacityof about two million acre-feeton 13,000 square mile Salt-Verdewatershed provide water for Phoenix,the capital city of Arizona.


74/89

A Case Study: Salt River Basin in Arizona

0

100

200

300

400

500

-3 -2 -1 0 1 2 3

SOI index

T o t a l S e a s o n a l

V o

l u m e

( B G a l )

1914-1970 1971-1998

Season 2

Dec.-Feb.


75/89

Proposed algorithm for rule-based forecasting of

hydrologic variables

Step 1 Definition of Hydrologic SeasonsStep 1 Definition of Hydrologic Seasons


76/89

Surface and SubsurfaceSurface and SubsurfaceWater Resources InteractionsWater Resources Interactions

Statistical Analysis of Statistical Analysis of Monthly DataMonthly Data

Definition of HydrologicDefinition of HydrologicSeasonsSeasons

Seasonal Time SeriesSeasonal Time Series

Statistical AnalysisStatistical Analysis

Removing TrendRemoving Trend Removing PeriodicityRemoving Periodicity Removing Outlier DataRemoving Outlier Data

Different Rainfall PatternsDifferent Rainfall Patterns

Monsoon and Tropical StormsMonsoon and Tropical StormsMidMid --Latitude SystemsLatitude SystemsMonthly Variation of RainfallMonthly Variation of Rainfall

Normal ProbabilityNormal ProbabilityTest for Each SeasonTest for Each Season

Redefinition of Redefinition of SeasonsSeasons

Step 1. Definition of Hydrologic SeasonsStep 1. Definition of Hydrologic Seasons


77/89

Step 2. Statistical ForecastingStep 2. Statistical Forecasting

Selection of BestARIMA Models

Generation of ForecastTime Series

Generation of Normal andStandard Forecast Series

Comparison betweenHistorical and Forecast Series

Changing the

Model if needed


78/89

Step 3. RuleStep 3. Rule --Based Forecast ModificationBased Forecast Modification

Forecast Time Series

AverageSnow

Budget

June-NovemberAverage SOI index

Divide Input-OutputSpace into Fuzzy Regions

Forecast Modificationusing the Rule BasedSystem (Verification)

Generate Fuzzy Rulesfrom Given Data

Calculate the Degree of Fulfillment for Each Rule

Assign a weight to each rulebased on the intersection of

membership functions

HistoricalStreamflowTime Series


79/89

Fuzzy Membership Functions for Snow andForecast Error Index

Average snow water equivalent depth membership function

1

0

1

0

0.5 0.8 1.0 1.2 2.0

Error index membership function

0 1 2 3 4 5 6 7 8 9


80/89

Model Verification

0

50

100

150

200

250

300

350

400450

500

1983 1985 1986 1987 1988 1991 1992 1993 1994 1995 1996 1997 1998

s e a s o n a

l s

t r e a m

f l o w

( B G a l )

Actua l ARIMA Modifie d w ith ENSO Modifie d w ithout ENSO

Second SeasonDec.-Feb.


81/89

Results Modified forecasts considering ENSO signals for the third

season are as good as the official forecast for Salt River. In 42percent of the analyzed years modified forecasts have beenbetter than official forecasts.

Significant improvement of the statistical forecasts for the thirdseason (30 percent) was achieved by the proposed method whenconsidering ENSO climate signals.

Only in 7 percent of the seasons the algorithm was not able toimprove or sustain the statistical streamflow forecast.


82/89

Environmental Water Demand

An environmental water demand (EWD) is thewater regime required to sustain theecological values of an aquatic ecosystem at alow level of risk . If this water requirement isadopted, then it is likely that a water bodywill:

Be healthy.

Look after the needs of animals and plants Maintain its biodiversity.

Steps for Identifying Environmental Water Requirements


83/89

1 . Prioritize sub-basins for assessment using informationon current water uses, river and estuarine healthindicators, and water management planningpriorities.

2. Consult basin stakeholders and relevant scientificexperts to determineimportant values for each sub-basin

in the categoriesof ecosystem, recreation, aesthetics,

physical landscape and consumptive/nonconsumptiveuse values

3. Assess environmental water requirements using the mostappropriate scientific methodology on a catchmentbasis.

Streamflow Modeling and Forecasting


84/89

Streamflow Modeling and Forecasting

StreamflowStreamflow Modeling ModuleModeling Module


85/89

StreamflowStreamflow ForecastingForecasting


86/89

ModuleModule

Demand Forecasting Models


87/89

Deterministic Components:1. Population Changes

1. Population Forecasting Models (Long-term and Sort-term)2. Multiple Regression Analysis

2. Price and Economic Incentives1. Analysis of Price Elasticity2. Time Series Analysis

3. Hydraulic Characteristics of the Water DistributionNetwork

1. Hydraulic Modeling of the Water Distribution Network



88/89

In order to incorporate the deterministic parameters inthe long-term demand forecasting usually differentscenarios based on the optimistic and pessimistic

probable conditions are considered.

These scenarios are defined based on the probable range

of:1. Migration2. Birth Rate

3. Death Rate4. Economic Policies and Incentives5. Other Factors



89/89

Stochastic Components:

1. Climate Variations1. Univariate AR, ARMA, and ARIMA Models2. Multivariate AR, ARMA, and ARIMA Models

Demand= f (population, price, standards, pastuse, etc.) + g(climate variations, )

Documents

02-Demand Forecasting - Prof. Karamouz