Upload
others
View
2
Download
0
Embed Size (px)
Citation preview
Electricity price forecasting: from prob-abilistic to deep learning approachesTU Delft & VITO-Energyville
Jesus Lago
October 10, 2019
Outline
1 Introduction
2 Time Series Forecasting
3 Point forecasting
4 Probability forecasting
5 Scenario Generation
6 Conclusion
Jesus Lago Electricity price forecasting: from probabilistic to deep learning approaches 1 / 54 z
Outline
1 IntroductionI Who Am I?I Research TopicI Focus of the talk
2 Time Series Forecasting
3 Point forecasting
4 Probability forecasting
5 Scenario Generation
6 Conclusion
Jesus Lago Electricity price forecasting: from probabilistic to deep learning approaches 1 / 54 z
Who Am I?
Personal InformationI Researcher at Energyville-VITO.
I Last-year PhD student at TU Delft.
I Research topic: algorithms for electricity markets that help increaseintegration of renewable energy sources (RES).
Jesus Lago Electricity price forecasting: from probabilistic to deep learning approaches 2 / 54 z
Outline
1 IntroductionI Who Am I?I Research TopicI Focus of the talk
2 Time Series Forecasting
3 Point forecasting
4 Probability forecasting
5 Scenario Generation
6 Conclusion
Jesus Lago Electricity price forecasting: from probabilistic to deep learning approaches 2 / 54 z
Research Topic
ProblemI Generation of RES is uncertain due to weather dependence.I As RES penetration increases:
1. Electricity prices becomes more volatile.2. Imbalances between generation and consumption increase.
Solution
Control algorithms for energy systems and electricity markets that:
1. Reduce negative effects of RES integration.
2. Increase the profitability of RES.
Jesus Lago Electricity price forecasting: from probabilistic to deep learning approaches 3 / 54 z
Role of Forecasting
Importance of Forecasting
I Forecasting is key to develop these control algorithms.
I Knowledge of future prices allows (among others):
1. Control RES systems to maximize profits.
2. Reduce risks by hedging against uncertainties.
3. Solve stochastic economic dispatch problems.
Jesus Lago Electricity price forecasting: from probabilistic to deep learning approaches 4 / 54 z
Outline
1 IntroductionI Who Am I?I Research TopicI Focus of the talk
2 Time Series Forecasting
3 Point forecasting
4 Probability forecasting
5 Scenario Generation
6 Conclusion
Jesus Lago Electricity price forecasting: from probabilistic to deep learning approaches 4 / 54 z
Focus of the talk
Electricity Markets
I Electricity is traded in several sequential markets.
Topic of the Talk
I Day-ahead price forecasting
Futures Markets Balancing Markets
Day-aheadmarket
Intradaymarket
Months/Weeks aheadPrevious
dayDelivering
dayReal time
Jesus Lago Electricity price forecasting: from probabilistic to deep learning approaches 5 / 54 z
Focus of the talk
MotivationI More volatile than futures and more liquid than intraday
I Large amount of RES traded on it
I Most of the literature focus on the day-ahead market
I Described methods apply to other markets
Futures Markets Balancing Markets
Day-aheadmarket
Intradaymarket
Months/Weeks aheadPrevious
dayDelivering
dayReal time
Jesus Lago Electricity price forecasting: from probabilistic to deep learning approaches 6 / 54 z
Day-ahead forecasting
DefinitionI Before deadline in day d− 1,
predict the 24 (48) day-aheadprices of day d.
Source: Electricity price forecasting: A review of the state-of-the-art with alook into the future
LiteratureI 20-30 years old field with numerous and diverse methods:
Multi-agent modelsFundamental modelsStatistical & machine learning models → Most accurate
I This talk: we focus on statistical & machine learning models
Jesus Lago Electricity price forecasting: from probabilistic to deep learning approaches 7 / 54 z
Outline
1 Introduction
2 Time Series ForecastingI Types of forecastingI Point ForecastingI Probability ForecastingI Scenario Generation
3 Point forecasting
4 Probability forecasting
5 Scenario Generation
6 ConclusionJesus Lago Electricity price forecasting: from probabilistic to deep learning approaches 7 / 54 z
Types of forecasting
Time series forecasting
I The forecast type depends on the type of information needed:
Point forecast: expected pricesProbability forecast: price distributionScenario forecast: possible price realizations
11/12 12/12 13/12 14/12 15/12
40
60
80
Pri
ce
11/12 12/12 13/12 14/12 15/12
40
60
80
Pri
ce
11/12 12/12 13/12 14/12 15/12
40
60
80
Date
Pri
ce
11/12 12/12 13/12 14/12 15/12
40
60
80
Date
Pri
ce
Jesus Lago Electricity price forecasting: from probabilistic to deep learning approaches 8 / 54 z
Outline
1 Introduction
2 Time Series ForecastingI Types of forecastingI Point ForecastingI Probability ForecastingI Scenario Generation
3 Point forecasting
4 Probability forecasting
5 Scenario Generation
6 ConclusionJesus Lago Electricity price forecasting: from probabilistic to deep learning approaches 8 / 54 z
Point forecasting
DefinitionI Point forecast only represent expected price
I It does not model uncertainty, e.g. forecasting error
I It cannot be used for assessing risks
11/12 12/12 13/12 14/12 15/12
40
60
80
Pri
ce
Real
Forecast
Fig: Day-ahead point forecast for the 14/12/2018 in the Nordpool
Jesus Lago Electricity price forecasting: from probabilistic to deep learning approaches 9 / 54 z
Outline
1 Introduction
2 Time Series ForecastingI Types of forecastingI Point ForecastingI Probability ForecastingI Scenario Generation
3 Point forecasting
4 Probability forecasting
5 Scenario Generation
6 ConclusionJesus Lago Electricity price forecasting: from probabilistic to deep learning approaches 9 / 54 z
Probability forecasting
DefinitionI Probability forecast represent price distribution
I It models the uncertainty of the forecasting error
I Two disadvantages:
1. Hard to use in stochastic optimization problems
2. No correlation between prices → unrealistic samples
11/12 12/12 13/12 14/12 15/12
40
60
80
Date
Pri
ce
Real
Forecast
Fig: Day-ahead probability forecast for the 14/12/2018 in the Nordpool
11/12 12/12 13/12 14/12 15/12
40
60
80
Date
Pri
ce
Real
Forecast
Sample
Fig: Day-ahead probability forecast for the 14/12/2018 in the Nordpool
Jesus Lago Electricity price forecasting: from probabilistic to deep learning approaches 10 / 54 z
Outline
1 Introduction
2 Time Series ForecastingI Types of forecastingI Point ForecastingI Probability ForecastingI Scenario Generation
3 Point forecasting
4 Probability forecasting
5 Scenario Generation
6 ConclusionJesus Lago Electricity price forecasting: from probabilistic to deep learning approaches 10 / 54 z
Scenario Generation forecasting
DefinitionI Scenarios represent possible price realizations
I They model not just uncertainty but also correlation
I Easy to use in stochastic optimization problems
11/12 12/12 13/12 14/12 15/12
40
60
80
Date
Pri
ce
Real
Fig: Day-ahead price scenarios for the 14/12/2018 in the Nordpool
Jesus Lago Electricity price forecasting: from probabilistic to deep learning approaches 11 / 54 z
Outline
1 Introduction
2 Time Series Forecasting
3 Point forecastingI DefinitionI Statistical MethodsI Machine LearningI Deep LearningI Summary
4 Probability forecasting
5 Scenario Generation
6 ConclusionJesus Lago Electricity price forecasting: from probabilistic to deep learning approaches 11 / 54 z
Definition
Day-ahead point forecast
I Expected price p at time k + h estimated at time k:
pk+h =M(θ, xk)
I p: expected value of p
I θ: model parameters
I k: midday previous day
I x: model inputs
I M : forecast model
I 24 horizons h1, . . . , h24
11/12 12/12 13/12 14/12 15/12
40
60
80
k k+h1 k+h24
Date
Pri
ce
Real
Forecast
Jesus Lago Electricity price forecasting: from probabilistic to deep learning approaches 12 / 54 z
Model inputs
Definition
Inputs xk defined by two types of data:
1. Historical prices at previous days, i.e. pd−1, . . . , pd−nd
2. Exogenous inputs:
Wind power forecast day dLoad forecast for day d
11/12 12/12 13/12 14/12 15/12
40
60
80
k k+h1 k+h24
Date
Pri
ce
Real
Forecast
Jesus Lago Electricity price forecasting: from probabilistic to deep learning approaches 13 / 54 z
Type of Models
Types of models
Literature very large: numerous and different methods.
Families of methods
Techniques are usually divided into two families:
1. Statistical methods: ARIMA, ARMAX, ARX...
2. Machine learning methods: neural nets, regression trees...
Combining models
Combining different types of models improves accuracy (not covered here)a
aNowotarski, Raviv, et al., “An empirical comparison of alternative schemes for combining electricity spot price forecasts”.
Jesus Lago Electricity price forecasting: from probabilistic to deep learning approaches 14 / 54 z
Outline
1 Introduction
2 Time Series Forecasting
3 Point forecastingI DefinitionI Statistical MethodsI Machine LearningI Deep LearningI Summary
4 Probability forecasting
5 Scenario Generation
6 ConclusionJesus Lago Electricity price forecasting: from probabilistic to deep learning approaches 14 / 54 z
Statistical Methods - Definition
Properties
I No clear definition of a statistical method
I Sometimes the same as some machine learning methods
I In price forecasting, they are defined by their properties:
1. Linear models.
2. Usually including autoregressive terms.
3. Sometimes including moving average terms.
4. Designed to include seasonal patterns.
Jesus Lago Electricity price forecasting: from probabilistic to deep learning approaches 15 / 54 z
Example
ARXI Linear model that considers:
1. Autoregressive inputs2. Seasonal components3. Exogenous inputs
I Example:
pd,h = θ1 · pd−1,24 + . . .+ θm1 · pd−7,h +
θm1+1 · z1 + . . .+ θm1+n · zn
Jesus Lago Electricity price forecasting: from probabilistic to deep learning approaches 16 / 54 z
Statistical Methods - Pros and Cons
I Advantages
3 Easy and fast to implement and estimate
3 For pure time series data, i.e. no exogenous inputs, they typicallyoutperform machine learning methods
3 Small parameter number =⇒ for small datasets they outperformmachine learning methods
I Drawbacks
7 Sometimes too simple for the nonlinear dynamics of prices. Not goodfor markets with rapid variations and high frequency changes
7 If prices depend on several exogenous inputs; e.g. demand, orgeneration; they might not model the complex relations.
Jesus Lago Electricity price forecasting: from probabilistic to deep learning approaches 17 / 54 z
Statistical Methods - State of the art
I State-of-the-art statistical method: fARX-Lassoa
I Evaluated in multiple marketsabc
1. Always better than other statistical methods
2. Sometimes better than machine learning methods
3. Sometimes worse than machine learning methods
I So many ARX in literature, how is this different?
Literature models had limited input features
200+ input features + implicit feature selection via LASSO
aUniejewski, Nowotarski, et al., “Automated variable selection and shrinkage for day-ahead electricity price forecasting”
bUniejewski and Weron, “Efficient Forecasting of Electricity Spot Prices with Expert and LASSO Models”
cLago et al., “Forecasting spot electricity prices: deep learning approaches and empirical comparison of traditional algorithms”
Jesus Lago Electricity price forecasting: from probabilistic to deep learning approaches 18 / 54 z
Statistical Methods - State of the art
fARX-Lasso Improvements
I Variance stabilization transformationa
I Average over different calibration windowsb
aUniejewski and Weron, “Efficient Forecasting of Electricity Spot Prices with Expert and LASSO Models”.
bMarcjasz et al., “Selection of Calibration Windows for Day-Ahead Electricity Price Forecasting”.
Jesus Lago Electricity price forecasting: from probabilistic to deep learning approaches 19 / 54 z
Outline
1 Introduction
2 Time Series Forecasting
3 Point forecastingI DefinitionI Statistical MethodsI Machine LearningI Deep LearningI Summary
4 Probability forecasting
5 Scenario Generation
6 ConclusionJesus Lago Electricity price forecasting: from probabilistic to deep learning approaches 19 / 54 z
Machine Learning (ML) - Outline
Outline
While the field of ML is extensive, forecasting of electricity prices is usuallybased in one of three family of methods:
1. Neural networks
2. Ensemble of trees
3. Support vector regressors
Jesus Lago Electricity price forecasting: from probabilistic to deep learning approaches 20 / 54 z
Machine Learning (ML) vs Statistical Methods (SM)
Summary
I Not clear whether ML methods are better than SM
Some studies have shown ML being betterMany others have shown SM being better
I In general, the best model depends on
1. Dataset/market under study2. Period under study3. Type and number of exogenous inputs
I Several studies have shown neural nets perform poorly
Jesus Lago Electricity price forecasting: from probabilistic to deep learning approaches 21 / 54 z
Machine Learning - Pros and Cons
I Advantages
3 Better suitable for prices with complex nonlinear dynamics, e.g. priceswith rapid variations or prices with frequent and large spikes
3 They can better model the complex relation between some exogenousinputs and prices, e.g. prices in neighboring markets
3 Estimation times larger than most statistical methods, but with currentstandard laptop hardware, below 10 minutes.
I Drawbacks
7 For pure time series data, i.e. no exogenous inputs, they are overkillingand underperform statistical methods
7 Large number of parameters =⇒ they require larger datasets thanstatistical methods.
7 Harder to interpret input-output relations.
Jesus Lago Electricity price forecasting: from probabilistic to deep learning approaches 22 / 54 z
Outline
1 Introduction
2 Time Series Forecasting
3 Point forecastingI DefinitionI Statistical MethodsI Machine LearningI Deep LearningI Summary
4 Probability forecasting
5 Scenario Generation
6 ConclusionJesus Lago Electricity price forecasting: from probabilistic to deep learning approaches 22 / 54 z
Deep Learning (DL) - Outline
MotivationI In recent years, several studies have shown deep learning (DL) models
being better than traditional ML and statistical methods
I Natural question 1: what is DL?
I Natural question 2: are DL methods really better?
Jesus Lago Electricity price forecasting: from probabilistic to deep learning approaches 23 / 54 z
What is deep learning?
Deep Learning vs Machine Learning
Two main and complementary views:
I Large neural networks
I Feature learning, a.k.a. automatic feature extraction
Source: towardsdatascience.com/why-deep-learning-is-needed-over-traditional-machine-learning-1b6a99177063 Source: quantdare.com/what-is-the-difference-between-deep-learning-and-
machine-learning
Jesus Lago Electricity price forecasting: from probabilistic to deep learning approaches 24 / 54 z
What is deep learning?
DefinitionI Wikipedia: artificial neural networks with
multiple hidden layers that can extracthigher level features
I New classes of neural networks
I New complementary algorithms:
New regularization techniques, e.g.dropoutNew optimizers, e.g. AdamNew activation functions, e.g. ReLU
Source: Van Veen, F. & Leijnen, S. (2019). TheNeural Network Zoo
Jesus Lago Electricity price forecasting: from probabilistic to deep learning approaches 25 / 54 z
DL for electricity price forecasting
Observation I
New DL methods for price forecasting arecontinuously being proposed
2016 2017 2018 2019
0
5
10
15
0
3
10
16
#N
um
ber
ofD
Lp
aper
s
Observation II
Most of them claim to have state-of-the-art result. Yet:
I Most only used 2-4 benchmark models (all based on ML)
I None compared with fARX-Lasso (state-of-the-art statistical method)
Motivation
We proposed new DL methods and performed an extensive comparisona:
I 23 literature models (inc. fARX-Lasso) + commercial software
aLago et al., “Forecasting spot electricity prices: deep learning approaches and empirical comparison of traditional algorithms”.
Jesus Lago Electricity price forecasting: from probabilistic to deep learning approaches 26 / 54 z
DL Case Study - Definition
Study Description
I 4 DL models proposed:
Deep feedforward network (DNN)2 Recurrent network (LSTM and GRU)Convolutional network (CNN)
I Evaluated BELPEX (Belgian) market
High forecasting errors and volatile pricesDifficult market for statistical methods
I Comparison against 23 literature models + commercial software
Study importance
I Remains to date as the only comparison of DL against severalstatistical methods
I Remains to date as the only comparison of DL against fARX-Lasso
Jesus Lago Electricity price forecasting: from probabilistic to deep learning approaches 27 / 54 z
DL Case Study - Results
3/4 DL models better than literature
Performance separation between ML and SM
fARX-Lasso as good as traditional ML
Model SMAPE [%] Class
DNN 12.34
ML
GRU 13.04
LSTM 13.06
MLP 13.27
SVR 13.29 ML
SVR-SOM 13.36
SVR-ARIMA 13.39
GBT 13.74
fARX-EN 13.76 SM
CNN 13.91 ML
fARX-Lasso 13.92 SM
Commercial 14.11
RBF 14.77 ML
fARX 14.79 SM
RF 15.39 ML
IHMARX 16.72
SM
DR 16.99
TARX 17.08
ARX 17.34
SNARX 17.58
TBATS 17.9
ARIMA-GARCH 19.3 SM
AR 19.31
DSHW 19.4
WARIMA-RBF 22.82
WARIMA 22.84
Jesus Lago Electricity price forecasting: from probabilistic to deep learning approaches 28 / 54 z
DL Case Study - Discussion
DL PerformanceI Why the DL models performed so good?
1. Market under study2. Depth and number of neurons (previous studies used shallow networks)3. Dataset size4. Optimization method
I Papera provides analysis of these factors (not discussed here)
aLago et al., “Forecasting spot electricity prices: deep learning approaches and empirical comparison of traditional algorithms”.
Statistical Methods vs Machine Learning
Statistical methods performed worse than most ML. However:
I BELPEX is a difficult market for statistical methods
I fARX-Lasso still performs as good as traditional ML
Jesus Lago Electricity price forecasting: from probabilistic to deep learning approaches 29 / 54 z
DL - Summary
1. DL models might obtain state-of-the-art results
2. However, this might be conditioned to different factors, e.g. marketunder study or exogenous inputs
3. Experimental results limited: more studies needed
Current work: evaluation of deep neural networks against fARX-Lassofor Nordpool, PJM, and EPEX-FR.
Jesus Lago Electricity price forecasting: from probabilistic to deep learning approaches 30 / 54 z
Outline
1 Introduction
2 Time Series Forecasting
3 Point forecastingI DefinitionI Statistical MethodsI Machine LearningI Deep LearningI Summary
4 Probability forecasting
5 Scenario Generation
6 ConclusionJesus Lago Electricity price forecasting: from probabilistic to deep learning approaches 30 / 54 z
Point Forecasting - Summary
1. No method is the best under all conditions
2. Best model will depend on different factors, e.g.
Without exogenous inputs, DL or ML are overkilling
For complex price dynamics, statistical methods might not suffice
3. DL models might obtain state-of-the-art results conditioned to thesefactors
Jesus Lago Electricity price forecasting: from probabilistic to deep learning approaches 31 / 54 z
Outline
1 Introduction
2 Time Series Forecasting
3 Point forecasting
4 Probability forecastingI DefinitionI Main MethodsI Quantile MethodsI New Interesting TrendsI Summary
5 Scenario Generation
6 ConclusionJesus Lago Electricity price forecasting: from probabilistic to deep learning approaches 31 / 54 z
Definition
Day-ahead probability forecast
I Cumulative distribution (CDF) of price p at time h estimated at k:
F (ph) =M(θ, xk)
I F (p): estimated CDF of p
I θ: model parameters
I k: midday previous day
I x: model inputs
I M : probabilistic model
I 24 CDFs: F (p1), . . . , F (p24)
12/12 13/12 14/12 15/12
40
60
80
k
F (p14)
Date
Pri
ce
40 60 80
0
0.25
0.5
0.75
1P (p14 < 62)
= 0.75
PriceF(p
14)
Jesus Lago Electricity price forecasting: from probabilistic to deep learning approaches 32 / 54 z
Outline
1 Introduction
2 Time Series Forecasting
3 Point forecasting
4 Probability forecastingI DefinitionI Main MethodsI Quantile MethodsI New Interesting TrendsI Summary
5 Scenario Generation
6 ConclusionJesus Lago Electricity price forecasting: from probabilistic to deep learning approaches 32 / 54 z
Probability forecasting methods
Types of models
Two families of methods:
I Parametric models
I Quantile models
Parametric modelsI The forecast is given by a full parameterization of the probability
distribution, e.g:ph ∼ N (µph , σph)
I Two main parametric distributions
Johnson’s SU distributionSkew-t distribution
Jesus Lago Electricity price forecasting: from probabilistic to deep learning approaches 33 / 54 z
Probability forecasting methods
QuantilesI Define random variable p and its CDF F (p).
I Quantile qα of p is the value at which theprobability of p is less than or equal to α,i.e. α = F (qα). 40 60 80
0
0.25
0.5
0.75
1 P (p14 < 62)
= 0.75
q75 = 62
q10(·)
q50(·)
q90(·)
Price
F(p
14)
Quantile functionsI F (p) is approximated building quantiles models qα(θ,x)I 4 main methods exist:
1. Empirical quantiles2. Quantile regression3. Quantile regression averaging4. Bootstrapping
Jesus Lago Electricity price forecasting: from probabilistic to deep learning approaches 34 / 54 z
Parametric vs Quantile Methods
1. Parametric models expected to perform worse than quantile modelsa
Parametric models limited by distribution assumption
2. Recent study: parametric model performs similar to quantile methoda
Skew-t distribution slightly better than linear quantile regression
However:
Other parametric models were worse than quantile modelQuantile model similar performance as the best parametricBest quantile model not considered
3. Here we focus on quantile methods
More general: no assumptions neededOften better accuracy
aGianfreda et al., “A stochastic latent moment model for electricity price formation”
Jesus Lago Electricity price forecasting: from probabilistic to deep learning approaches 35 / 54 z
Outline
1 Introduction
2 Time Series Forecasting
3 Point forecasting
4 Probability forecastingI DefinitionI Main MethodsI Quantile MethodsI New Interesting TrendsI Summary
5 Scenario Generation
6 ConclusionJesus Lago Electricity price forecasting: from probabilistic to deep learning approaches 35 / 54 z
Prediction intervals vs Quantiles
DifferenceI Some probabilistic forecasting papers provide prediction intervals (PI)
I Some others provide quantiles qα
I Quantiles are just a generalization of prediction intervals
Example: 90% PI equals the interval [q5, q95]
I In this talk, we use the word quantile as a general term
11/12 12/12 13/12 14/12 15/12
40
60
80
Date
Pri
ce
11/12 12/12 13/12 14/12 15/12
40
60
80
Date
Pri
ce
90% PI
q5, q95Jesus Lago Electricity price forecasting: from probabilistic to deep learning approaches 36 / 54 z
Quantile Method I - Empirical Quantiles
Algorithm
1. Consider past point forecasts at hour h:
[pd,h, . . . , pd−n,h]>
2. Compute historical forecasting errors εd, . . . , εd−n.
3. Compute empirically quantile distribution qα(ε) of errors.
4. Quantile function of price at hour h given by:
qα(pd,h) = pd,h + qα(ε)
i.e. point prediction plus quantile function of errors.
Jesus Lago Electricity price forecasting: from probabilistic to deep learning approaches 37 / 54 z
Quantile Method II - Quantile Regression
Quantile Regression
I Parameterizes quantile functions qα(·) by model Mα(θ, x).
I Estimates Mα(θ, x) by solving:
minθ
N∑i=1
(α− 1)max(0,Mα(θ, xi)− pi
)+ αmax
(0, pi −Mα(θ, xi)
)where:
{(xi, pi)}Ni=1 dataset of prices and inputsInputs xi the same as for point forecasts
Examples
I Most common model: linear quantile regression Mα(θ, x) = θ>x
I Nonlinear version: Mα(θ, x) as a neural network
Jesus Lago Electricity price forecasting: from probabilistic to deep learning approaches 38 / 54 z
Method III - Quantile Regression Averaging
Quantile Regression Averaging (QRA)
I Estimate quantiles using point forecasts and linear quantile regression:
1. Build N different point forecasts
2. Use the N predictions as vector of input features x
3. Apply standard quantile regression
Motivation
Estimate quantiles for nonlinear dynamics with linear method
Jesus Lago Electricity price forecasting: from probabilistic to deep learning approaches 39 / 54 z
Method IV- Bootstrapping
Conceptual Idea
1. Generate datasets obtained via resampling with replacement
2. Estimate a point forecast ph model for each dataset
Source: https://hub.packtpub.com/ensemble-methods-optimize-machine-learning-models/
Jesus Lago Electricity price forecasting: from probabilistic to deep learning approaches 40 / 54 z
Method IV- Bootstrapping
Conceptual Idea
1. Generate datasets obtained via resampling with replacement
2. Estimate a point forecast ph model for each dataset
3. Use models to estimate quantiles qα(εm) of model errors
4. Use qα(εm) to estimate quantiles qα(εp) of process errors
5. Quantile function of price at hour h given by:
qα(ph) = E{ph}+ qα(εp) + qα(εm)
CharacteristicsI It distinguishes between model and process errors
I More computationally demanding than the others
Jesus Lago Electricity price forecasting: from probabilistic to deep learning approaches 41 / 54 z
Quantile Methods - State of the art
I Several studies have evaluated and compare quantile methods, e.g.
1. Global Energy Forecasting Competition (GEFCom2014)2. Different papersabc
I State-of-the-art method: Quantile regression averaging (QRA)
I Inexistent bootstrapping vs QRA comparison.
In our experience, bootstrapping performs worse.
aNowotarski and Weron, “Recent advances in electricity price forecasting: A review of probabilistic forecasting”
bUniejewski, Marcjasz, et al., “On the importance of the long-term seasonal component in day-ahead electricity price forecasting: Part II —Probabilistic forecasting”
cMaciejowska et al., “Probabilistic forecasting of electricity spot prices using Factor Quantile Regression Averaging”
Jesus Lago Electricity price forecasting: from probabilistic to deep learning approaches 42 / 54 z
Outline
1 Introduction
2 Time Series Forecasting
3 Point forecasting
4 Probability forecastingI DefinitionI Main MethodsI Quantile MethodsI New Interesting TrendsI Summary
5 Scenario Generation
6 ConclusionJesus Lago Electricity price forecasting: from probabilistic to deep learning approaches 42 / 54 z
New Interesting Trend I - Methods better than QRA
Recent Developments
Last months: 2 approaches that performs similar to QRA were proposed:
1. A Bayesian stochastic volatility modela
2. A conformal prediction modelb
Importance
Methods that perform in some cases better than the state-of-the-art
Word of cautionI QRA still perform similar to these two
I New methods tested in one study, QRA in many of them
aKostrzewski et al., “Probabilistic electricity price forecasting with Bayesian stochastic volatility models”
bKath et al., “Conformal Prediction Interval Estimations with an Application to Day-Ahead and Intraday Power Markets”
Jesus Lago Electricity price forecasting: from probabilistic to deep learning approaches 43 / 54 z
New Interesting Trend II - Deep Learning (DL)
Summary
I Research on DL for probability forecasting is very limited
I Many claim to do DL for probability forecastinga, however
DL is only used for point forecastingProb. forecasting is made based on standard methods
aAfrasiabi et al., “Probabilistic deep neural network price forecasting based on residential load and wind speed predictions”.
Probability Forecasting based on DL
To the best of my knowledge, only two worksab in DL
I Drawback: not compared with QRA or other standard methods
aBrusaferri et al., “Bayesian deep learning based method for probabilistic forecast of day-ahead electricity prices”.
bHu et al., “Distribution-Free Probability Density Forecast Through Deep Neural Networks”.
Jesus Lago Electricity price forecasting: from probabilistic to deep learning approaches 44 / 54 z
Outline
1 Introduction
2 Time Series Forecasting
3 Point forecasting
4 Probability forecastingI DefinitionI Main MethodsI Quantile MethodsI New Interesting TrendsI Summary
5 Scenario Generation
6 ConclusionJesus Lago Electricity price forecasting: from probabilistic to deep learning approaches 44 / 54 z
Probability Forecasting - Summary
1. Two main family of methods: parametric and quantile models
Parametric worse accuracy due to distribution assumption
2. There are several quantile models:
State-of-the-art: Quantile regression averaging (QRA)
3. Research on deep learning for probability forecasting is limited
Several works use deep learning as a buzzword
Jesus Lago Electricity price forecasting: from probabilistic to deep learning approaches 45 / 54 z
Outline
1 Introduction
2 Time Series Forecasting
3 Point forecasting
4 Probability forecasting
5 Scenario GenerationI IntroductionI Main MethodsI Comparison
6 Conclusion
Jesus Lago Electricity price forecasting: from probabilistic to deep learning approaches 45 / 54 z
Definition
Scenario generation forecast
I Recall: we need possible price realization:
Probability functions hard to use in stochastic optimization
Probability functions do not consider price correlations
I Goal: generate N possible price scenarios S1, . . . , SN
Si = [pi1, . . . , pi24]: possible realization of day-ahead prices
Scenarios with marginal distributions equal to probability forecasts
11/12 12/12 13/12 14/12 15/12
40
60
80
Date
Pri
ce
Real
Jesus Lago Electricity price forecasting: from probabilistic to deep learning approaches 46 / 54 z
Scenario Generation - Literature
Observations
1. Specific scenario generation literature is scarce
2. In general, existing papers do not propose new methods:
Their research goal is to solve a stochastic optimization problem, e.g.optimal market bidding
They consider a generic scenario generation method
3. No paper comparing different methods
Consequences
I Many methods could be presented
I It would be hard to draw comparisons
I We briefly present the main families and explain their differences
Jesus Lago Electricity price forecasting: from probabilistic to deep learning approaches 47 / 54 z
Outline
1 Introduction
2 Time Series Forecasting
3 Point forecasting
4 Probability forecasting
5 Scenario GenerationI IntroductionI Main MethodsI Comparison
6 Conclusion
Jesus Lago Electricity price forecasting: from probabilistic to deep learning approaches 47 / 54 z
Scenario generation methods
Types of methods
Scenario generation methods used for electricity prices and stochasticoptimization can be classified into three families:
1. Sampling-based methods
2. Optimization-based methods
3. Copulas-based methods
Jesus Lago Electricity price forecasting: from probabilistic to deep learning approaches 48 / 54 z
Scenario generation - Sampling based methods
Steps
1. Fit stochastic model to prices
ARMA model with Gaussian errorsa
ARMA for prices with GARCH model for conditional error varianceb
Neural network model with Gaussian errorsc
2. Use stochastic model to recursively simulate scenarios
aFleten et al., “Stochastic programming for optimizing bidding strategies of a Nordic hydropower producer”.
bFaria et al., “Day-ahead market bidding for a Nordic hydropower producer: taking the Elbas market into account”.
cVagropoulos et al., “ANN-based scenario generation methodology for stochastic variables of electric power systems”.
11/12 12/12 13/12 14/12 15/12
40
60
80
Date
Pri
ce
Real
Jesus Lago Electricity price forecasting: from probabilistic to deep learning approaches 49 / 54 z
Scenario generation - Optimization based methods
Steps
1. Define the statistical metrics of scenarios
Metrics usually based on historical dataExample: the first four moments of the pricesab
2. Solve optimization problem to generate scenarios:
Scenarios as optimization variablesObjective: difference between desired and scenario metrics
3. Result: scenarios that satisfy statistical metrics
aHøyland et al., “A heuristic for moment-matching scenario generation”.
bJensen et al., “A comparison of scenario generation methods for the participation of electric vehicles in electricity markets”.
Jesus Lago Electricity price forecasting: from probabilistic to deep learning approaches 50 / 54 z
Scenario generation - Copula based methods
Sklar’s Theorem applied to pricesa
I Define the multivariate CDF of the 24 prices by H(p1, . . . , p24)
I Define the marginal CDFs of each price by F1(p1), . . . , F24(p24)
I There exists a copula function C such thata:
C(F1(p1), . . . , F24(p24)
)= H(p1, . . . , p24)
i.e. the marginal and multivariate distributions are related by C
aSklar, “Fonctions de Repartition a n Dimensions et Leurs Marges”.
Consequence
I We can obtain a multivariate distribution based on probabilityforecasts and sample from it to generate scenarios
Jesus Lago Electricity price forecasting: from probabilistic to deep learning approaches 51 / 54 z
Scenario generation - Copula based methods
Steps
1. Use probability forecasting methods to obtain the marginaldistributions F1(p1), . . . , F24(p24)
2. Define the copula type, e.g.
Empirical copulaa
3. Generate or estimate copula using the marginal distributions
4. Generate scenarios by sampling from the copula
aToubeau et al., “Deep Learning-Based Multivariate Probabilistic Forecasting for Short-Term Scheduling in Power Markets”.
12/12 13/12 14/12 15/12
40
60
80
F (p14)
Date
Pri
ce
40 60 80
0
0.25
0.5
0.75
1P (p14 < 62)
= 0.75
Price
F(p
14)
11/12 12/12 13/12 14/12 15/12
40
60
80
Date
Pri
ce
Real
Jesus Lago Electricity price forecasting: from probabilistic to deep learning approaches 52 / 54 z
Outline
1 Introduction
2 Time Series Forecasting
3 Point forecasting
4 Probability forecasting
5 Scenario GenerationI IntroductionI Main MethodsI Comparison
6 Conclusion
Jesus Lago Electricity price forecasting: from probabilistic to deep learning approaches 52 / 54 z
Comparison
I Hard to compare the different methods in terms of accuracy
No empirical comparison exists (to the best of my knowledge)We can list the advantages and drawbacks
I Sampling-based methods
3 Simpler and easier to estimate7 Bad approximations with few scenarios
I Optimization based method
3 Very flexible: generated scenarios can display any desired metrics7 Large computational complexity
I Copula-based methods
3 Generated scenarios follow marginal distributions3 For large number of scenarios, less complex than optimization7 Distribution depends on selected copula
Jesus Lago Electricity price forecasting: from probabilistic to deep learning approaches 53 / 54 z
Outline
1 Introduction
2 Time Series Forecasting
3 Point forecasting
4 Probability forecasting
5 Scenario Generation
6 Conclusion
Jesus Lago Electricity price forecasting: from probabilistic to deep learning approaches 53 / 54 z
Conclusion
1. We have presented forecasting methods for electricity prices
Point forecastsProbability forecastScenario generation methods
2. For the three fields, the best forecasting model depend upon:
Market under studyType and size of input dataOthers
3. Deep learning models are continuously being proposed:
Nearly all of them are limited to point forecasting
Further comparison against state-of-the-art methods is still needed
Jesus Lago Electricity price forecasting: from probabilistic to deep learning approaches 54 / 54 z
Thank you! Any Questions?
Email: [email protected]
References I
M. Afrasiabi et al. “Probabilistic deep neural network priceforecasting based on residential load and wind speed predictions”.In: IET Renewable Power Generation 13.11 (2019), pp. 1840–1848.doi: 10.1049/iet-rpg.2018.6257.
Alessandro Brusaferri et al. “Bayesian deep learning based methodfor probabilistic forecast of day-ahead electricity prices”. In: AppliedEnergy 250 (2019), pp. 1158–1175. doi:https://doi.org/10.1016/j.apenergy.2019.05.068.
Eduardo Faria and Stein-Erik Fleten. “Day-ahead market bidding fora Nordic hydropower producer: taking the Elbas market intoaccount”. In: Computational Management Science 8.1 (Apr. 2011),pp. 75–101. doi: 10.1007/s10287-009-0108-5.
Jesus Lago Electricity price forecasting: from probabilistic to deep learning approaches 1 / 7 z
References II
Stein-Erik Fleten and Trine Krogh Kristoffersen. “Stochasticprogramming for optimizing bidding strategies of a Nordichydropower producer”. In: European Journal of OperationalResearch 181.2 (2007), pp. 916–928. doi:https://doi.org/10.1016/j.ejor.2006.08.023.
Angelica Gianfreda and Derek Bunn. “A stochastic latent momentmodel for electricity price formation”. In: Operations Research 66.5(2018), pp. 1189–1203.
Kjetil Høyland, Michal Kaut, and Stein W Wallace. “A heuristic formoment-matching scenario generation”. In: Computationaloptimization and applications 24.2-3 (2003), pp. 169–185.
T. Hu et al. “Distribution-Free Probability Density ForecastThrough Deep Neural Networks”. In: IEEE Transactions on NeuralNetworks and Learning Systems (2019), pp. 1–14. issn: 2162-237X.doi: 10.1109/TNNLS.2019.2907305.
Jesus Lago Electricity price forecasting: from probabilistic to deep learning approaches 2 / 7 z
References III
Ida Græsted Jensen et al. “A comparison of scenario generationmethods for the participation of electric vehicles in electricitymarkets”. In: International Transactions on Electrical EnergySystems 29.4 (2019), e2782. doi: 10.1002/etep.2782.
Christopher Kath and Florian Ziel. “Conformal Prediction IntervalEstimations with an Application to Day-Ahead and Intraday PowerMarkets”. In: arXiv preprint arXiv:1905.07886 (2019).
Maciej Kostrzewski and Jadwiga Kostrzewska. “Probabilisticelectricity price forecasting with Bayesian stochastic volatilitymodels”. In: Energy Economics 80 (2019), pp. 610–620. doi:https://doi.org/10.1016/j.eneco.2019.02.004.
Jesus Lago, Fjo De Ridder, and Bart De Schutter. “Forecasting spotelectricity prices: deep learning approaches and empirical comparisonof traditional algorithms”. In: Applied Energy 221 (July 1, 2018),pp. 386–405. doi: 10.1016/j.apenergy.2018.02.069.
Jesus Lago Electricity price forecasting: from probabilistic to deep learning approaches 3 / 7 z
References IV
Katarzyna Maciejowska, Jakub Nowotarski, and Rafa l Weron.“Probabilistic forecasting of electricity spot prices using FactorQuantile Regression Averaging”. In: International Journal ofForecasting 32.3 (2016), pp. 957–965. doi:10.1016/j.ijforecast.2014.12.004.
Grzegorz Marcjasz, Tomasz Serafin, and Rafa l Weron. “Selection ofCalibration Windows for Day-Ahead Electricity Price Forecasting”.In: Energies 11.9 (2018), p. 2364. doi: 10.3390/en11092364.
Jakub Nowotarski, Eran Raviv, et al. “An empirical comparison ofalternative schemes for combining electricity spot price forecasts”.In: Energy Economics 46 (Nov. 2014), pp. 395–412. issn:0140-9883. doi: 10.1016/j.eneco.2014.07.014. (Visited on04/04/2017).
Jesus Lago Electricity price forecasting: from probabilistic to deep learning approaches 4 / 7 z
References V
Jakub Nowotarski and Rafa l Weron. “Recent advances in electricityprice forecasting: A review of probabilistic forecasting”. In:Renewable and Sustainable Energy Reviews 81 (2018),pp. 1548–1568. doi:https://doi.org/10.1016/j.rser.2017.05.234.
Abel Sklar. “Fonctions de Repartition a n Dimensions et LeursMarges”. In: Institut Statistique de l’Universite de Paris 8 (1959),pp. 229–231.
J. Toubeau et al. “Deep Learning-Based Multivariate ProbabilisticForecasting for Short-Term Scheduling in Power Markets”. In: IEEETransactions on Power Systems 34.2 (Mar. 2019), pp. 1203–1215.doi: 10.1109/TPWRS.2018.2870041.
Jesus Lago Electricity price forecasting: from probabilistic to deep learning approaches 5 / 7 z
References VI
Bartosz Uniejewski, Grzegorz Marcjasz, and Rafa l Weron. “On theimportance of the long-term seasonal component inday-ahead electricity price forecasting: Part II — Probabilisticforecasting”. In: Energy Economics 79 (2019), pp. 171–182. doi:https://doi.org/10.1016/j.eneco.2018.02.007.
Bartosz Uniejewski, Jakub Nowotarski, and Rafa l Weron.“Automated variable selection and shrinkage for day-aheadelectricity price forecasting”. In: Energies 9.8 (2016), p. 621. doi:10.3390/en9080621.
Bartosz Uniejewski and Rafa l Weron. “Efficient Forecasting ofElectricity Spot Prices with Expert and LASSO Models”. In:Energies 11.8 (Aug. 2018), p. 2039. doi: 10.3390/en11082039.
Jesus Lago Electricity price forecasting: from probabilistic to deep learning approaches 6 / 7 z
References VII
Stylianos I. Vagropoulos et al. “ANN-based scenario generationmethodology for stochastic variables of electric power systems”. In:Electric Power Systems Research 134 (2016), pp. 9–18. doi:https://doi.org/10.1016/j.epsr.2015.12.020.
Jesus Lago Electricity price forecasting: from probabilistic to deep learning approaches 7 / 7 z