IBM WindEnergy White

© 2009 IBM Corporation

Forecasting aggregated energy demand in the presence of missing data and outliers with double (or more) seasonality

Avner Abrami – Columbia University / Mentor : Dr. Younghun Kim

August 31st 2015

© 2009 IBM Corporation2

Agenda

Motivation of the problem

Accurate demand forecasts are essential to energy companies

Traditional approaches have limitations

Proposed approach

Formulating exponential smoothing as an optimization problem

Formulation handles missing data

Formulation resists outliers

Future work and extensions

IBM Smarter Energy – Industries and Solutions


MOTIVATION OF THE PROBLEM



Motivation of the problemAccurate demand forecasting is essential for energy companies

• Optimal maintenance and upgrade planning of existing infrastructure

• Competitive advantage in energy trading

• Optimal scheduling generation operations• E.g. Electricity case: 5% reduction in load uncertainty translates into the reduction of

5% spinning reserve requirements, which is 2,046 million kWh (Savings of $204M).• E.g. Gas case: Avoidance of low pressure situation ; efficient compressor station

matintenance and operation.

• Assessing the feasibility of renewable integration (which increases load uncertainty)


Smart meters are the promising big data resource for such quantitative analysis, despite their generating challenging data volume and complexity.


Motivation of the problemReal data


The real energy dataset is primary characterized by a weekly & yearly seasonality

Data from smart meters may have missing data and outliers– Very common to have communication errors from smart meters (Missing data)– Malfunction of smart meters (Outliers)


Motivation of the problemTraditional approaches have limitations

• Seasonal autoregressive integrated moving average models (SARIMA)– Becomes obsolete when dealing with more than one seasonality-> No available R packages to handle multi-seasonality ARIMA– Very difficult to handle missing data in the ARMA framework, especially when

incorporating seasonality.– Not robust to outliers (Autoregressive models)

• Gaussian process regression– Very sensitive to choice of kernel and means functions (which could vary depending

on the kind of seasonal data we are dealing with).– Periodic kernel/ locally periodic kernel allows to model functions which repeat

themselves.– Very efficient on simulated data.– Possibility to handle double seasonality by combining periodic kernels.


)',()',()',()',()',( 21 xxkxxkxxkxxkxxk NoisePeriodicPeriodicRBF

)',()',()',()',( 21 xxkxxkxxkxxk NoisePeriodicPeriodic


PROPOSED APPROACH



Proposed approachAn enhanced additive exponential smoothing formulation

• Exponential smoothing weights past observations with exponentially decreasing weights to forecast future values

• Allow structured modeling of the time series’ evolution:

– Mean

– Trend

– Seasonality

• Formulated as a single source of error (SSOE) statistical model


)1)(1()(1

1

1ty blsl t tptt

bllb tttt 11 )1()(

sbls ptttt

1

1111t1

1 )1()(y



Rsbl ttt

3),,(

tptt

ttt

tttt

tpttt

ssbbbllsbl

1

1

1

11

11

1

111tyRecursive formulation State-space formulation

Step 1: From the recursive to the state-space univariate formulation


sblsbllbblsl

sbl

ptttt

tttt

tptt

tpttt

t

1

1111t1

111

1

1

1t

1

111t

)1()()1()(

)1)(1()(

y

yy

)1,0(~ Nt

R3),,(

Mean, trend, seasonality functions at time t

Noise

yt Observation at time t Smoothing parameters



...

...

2

2

2

2

1

1

1

11

11

2

2

1

111ty

tptt

tptt

ttt

tttt

tptpttt

ssssbbbllssbl

One seasonality Double seasonality(or more..)

Step 2: Introducing double (or more) seasonality

tptt

ttt

tttt

tpttt

ssbbbllsbl

1

1

1

11

11

1

111ty





Recursive form Recursive matrix form

Step 3: Formulate the state-space model in matrix form

(...)

...)(

2

2

2

2

1

1

1

11

11

2

2

1

111ty

tptt

tptt

ttt

tttt

tptpttt

ssssbbbllssbl

ttt

ttt

gA

w

xxxy

1

1

0

'

)',,,( 2

2

1

1 ssblx ptptttt




Recursive matrix form Least-square form

Step 4: Exponential smoothing as least-square

x

xx

y

y

y

wIA

wIA

w

MxY

n

2

1

n

2

1

'

'

'

0

0

0

ttt

ttt

gA

w

xxxy

1

1

0

'

)',,,( 2

2

1

1 ssblx ptptttt




s

wIA

wIA

w

sMxY

x

xx

y

y

y

n

2

1

n

2

1

'

'

'

0

0

0

'

'1

~~

~~

~

gg

ggV

gg

),0(~ VNs

Formulating a maximum likelihood problem is complicated by the singularity of V (Rank(V)=1 hence non invertible).

In this case, the least square problem becomes:

sVMxYts

sgsx

2/1

2,,

:.

||||21min

Where Bayesian optimization was used to fit smoothing parameters g

Which constitutes the smoothing problem


Proposed approachFilling missing data and forecast future time series values


Incorporating missing data is easy in this formulation - we just include a diagonal matrix D, with diagonal entries corresponding to missing values set to 0.

sDVDMxDYts

sgsx

2/1

2,,

:.

||||21min


Which constitutes the smoothing problem with missing data

The forecasting problem is equivalent. We just have to consider future values as missing values!


Proposed approachA large scale prediction can be efficiently solved


sDVDMxDYts

sgsx

2/1

22,,

:.

||||min

sDVDMxDYts

sgsx

2/1

1,,

:.

||||min

sDVDMxDY

tsts

ts

sgsx

2/1

,,

:.

min

Linear Programming RelaxationLinear Programming Relaxation

Norm changeL2 norm to L1 norm Equivalent

The structure of the problem is sparse in most cases, the problem can be solved exploiting the sparsity of the matrices.

Numerically equivalent


Proposed approachSimulated demand


We simulate energy consumption to test different forecasting algorithm as follows:

For example, a weekend high demand and weekend low demand can be effectively captured using the square function

)()2(*2)2sin(*1)(21

ttHAtAoffsetty ff

H: Sawtooth functionH: Sawtooth functionH: Square functionH: Square function


Proposed approachSimulated demand


The algorithm can extract the double seasonality, and all deterministic behaviors of the time series effectively.


Proposed approachSome results on simulated data

Objective NoiseStd

MAPE (%)Sawtooth

MAPE (%)Square

M1 L2 0.75 6.35 5.01

L2 2.25 11.25 12.9

M2 L1 0.75 6.75 5.86

L1 2.25 10.95 11.7

GPs 0.75 4.98 4.74

22.25 9.55 8.49

3 month forecasting 6 month forecasting

12 month forecasting

Objective NoiseStd

MAPE (%)Sawtooth

MAPE (%)Square

M1 L2 0.75 5.9 7.68

L2 2.25 11.18 14.66

M2 L1 0.75 6.2 7.34

L1 2.25 11.05 13.57

GPs 0.75 6.95 10.95

2.25 10.02 11.05

Objective NoiseStd

MAPE (%)Sawtooth

MAPE (%)Square

M1 L2 0.75 6.4 7.39

L2 2.25 11.52 13.24

M2 L1 0.75 6.78 7.10

L1 2.25 10.85 12..98

GPs 0.75 6.5 7.08

2.25 14.12 11.89


Proposed approachSome results on simulated data

Objective NoiseStd

MAPE (%)Sawtooth

MAPE (%)Square

M1 L2 0.75 6.35 5.01

L2 2.25 11.25 12.9

M2 L1 0.75 6.75 5.86

L1 2.25 10.95 11.7

GPs 0.75 4.98 4.74

22.25 9.55 8.49

3 month forecasting 6 month forecasting


Objective NoiseStd

MAPE (%)Sawtooth

MAPE (%)Square

M1 L2 0.75 5.9 7.68

L2 2.25 11.18 14.66

M2 L1 0.75 6.2 7.34

L1 2.25 11.05 13.57

GPs 0.75 6.95 10.95

2.25 10.02 11.05

Objective NoiseStd

MAPE (%)Sawtooth

MAPE (%)Square

M1 L2 0.75 6.4 7.39

L2 2.25 11.52 13.24

M2 L1 0.75 6.78 7.10

L1 2.25 10.85 12..98

GPs 0.75 6.5 7.08

2.25 14.12 11.89

~ Same performance for 10%

~ Same performance for 10%

uniformly missing observations

uniformly missing observations


Proposed approachRobustness


Formulation for SSOE models are inherently robust against outliers providing the training set is large enough (4 years of data at least)

Robust simulation procedure :

- Double seasonal (sinusoid + sawtooth/square) signal with noise = N(0,(15%(max-min)) **2)

- Add 2.5% of outliers : Random(sign)*(mean + Uniform(3*(max-mean),5*(max-min))

- 4 years of training, 6 month of forecasting (but could be more)

)(:.

||||21min

2/1

22,,

sDVDMxDYts

sgsx


Which constitutes the robust problem with missing data


Proposed approachResults: Robustness


• Robust for the 4 year smoothing!

• Robust for the 6 month forecasting!


An enhanced additive exponential smoothing formulationWhy is the procedure robust?


There is a competition between:

- The objective function that aims at minimizing the residual s

- The constraints which enforces the recursive equation to be fulfilled.

When the algorithm detects an outlier, the objective wants to fit it perfectly. Yet if it did, the constraints would impose outliers for the next value of the time series which would result in a much higher ||s||2 than the robust version.

To be robust, we hence need a large enough dataset.

)(:.

||||21min

2/1

2,,

sDVDMxDYts

sgsx

...

...

2

2

2

21

1

1

11

11

2

2

1

111ty

tptt

tptt

ttt

tttt

tptpttt

ssssbbbllssbl

Objective



ss AABCD '

An enhanced additive exponential smoothing formulationWhy is the procedure robust?

So the second scheme is preferred by the algorithm!


24

Simulation OutcomesReal demand


Smoothing problem 1 year forecasting

)',()',()',()',()',( 21 xxkxxkxxkxxkxxk NoisePeriodicPeriodicRBF

GP regression

Enhanced ES


Proposed approachSome results on real data

Objective MAPE (%)

GPs 69.87

M1 L1 15.42

M2 L2 14.28

6 month forecasting


3 month forecasting

Objective MAPE (%)

GPs 71.54

M1 L1 18..58

M2 L2 17.85

Objective MAPE (%)

GPs 75.26

M1 L1 19.25

M2 L2 19.84


FUTURE WORK AND EXTENSION



Forecasting aggregated energy demand in the presence of missing data and outliers with double (or more) seasonality

A tailored optimization to improve speed and process even more data and time series patterns.

Add daily periodicity of energy consumption (computationally intensive except for a tailored solver).

Increase the model’s resistance to jitter in the periodicity.

Exploit correlation for collection of over two time series.



THANK YOU TO

Younghun Kim

&

Sasha Aravkin



Q&A


Documents

IBM WindEnergy White