Embed Size (px)
Citation preview
Exogenous versus endogenous dynamics in price formation
Vladimir Filimonov Chair of Entrepreneurial Risks, D-MTEC, ETH Zurich [email protected]
Chair of Quantitative Finance, École Centrale, Paris, France. May 16, 2014
Algorithmic and High-Frequency Trading
0%
10%
20%
30%
40%
50%
60%
2004 2005 2006 2007 2008 2009 2010
EquitiesFuturesOptionsFXFixed Income
Adoption of algorithmic execution by asset classes
~65%
U.S.
~54%
Europe
~19%
Asia
Source: Aite group
Source: Aite group
HFT trade volume as %of total market
2
1998 2000 2002 2004 2006 2008 2010 20121 msec
10 msec
100 msec
1 sec
Year
Brent CrudeWTIE−mini S&P 500
Typical market makers’ reaction time
• Filimonov V., Sornette D. (2014) La vie èconomique
Data source: TRTH
3
Financialization of commodities
Increasing market share of commodity speculators
Finance Watch/MiFID2
Investing not betting
39
revenues. A third category of participants, ‘traditional speculators’, play the role
PG�DPVOUFSQBSUJFT�XIFO�DPOTVNFST�PS�QSPEVDFST�EP�OPU�mOE�BOPUIFS�DPNNFSDJBM�
counterparty to hedge their risks. These speculators are looking for a remuneration of
UIFJS�SJTL�CZ�HBJOJOH�GSPN�UIF�VOEFSMZJOH�DPNNPEJUZ�T�QSJDF�nVDUVBUJPO��
Central to protecting the price formation mechanism of these markets is that
speculators be restricted to a minority of participants: indeed as long as this is the case,
their projections will be based, although indirectly, on fundamental supply and demand
factors as these will determine the behaviour of participants looking to hedge. When
speculators gain a dominant position in a commodity derivative’s market, they base their
projections on the potential behaviour of other speculators, thereby disconnecting futures
prices from fundamentals. Producers and consumers make commodities futures markets
FGmDJFOU �OPU�TQFDVMBUPST�
Figure 9: Increasing market share of commodity speculators
Traditional Speculator 16%
7%
Index Speculator
Index Speculator41%
Physical Hedger31%
Traditional Speculator
28%
Physical Hedger 77%
Long Open Interest – 1998 Long Open Interest – 2008
6RXUFH��&)7&�ÀJXUHV��FKDUWV�E\�0LNH�0DVWHUV��%HWWHU�0DUNHWV�64
Orderly functioning of commodity derivatives markets, as described in the previous
paragraph, is not just important to protect the price of instruments traded, it also has a
direct impact on the price of the underlying (physical) commodity. Because commodity
spot markets are so dispersed (due, among other factors, to the cost of transportation),
they have for a long time relied on local supply and demand to determine prices. As
consumption and production went global, the price on spot markets started to be based on futures prices. For most commodities today, the reference price is the futures
QSJDF �BEKVTUFE�UP�MPDBM�TVQQMZ�BOE�EFNBOE�TQFDJmDJUJFT��
This is a very important phenomenon to understand as it is different from what takes
QMBDF�PO�GVUVSFT�NBSLFUT�SFMBUFE�UP�mOBODJBM�VOEFSMZJOH�BTTFUT��5IF�QSJDF�PG�B�GVUVSF�
DPOUSBDU�SFMBUFE�UP�B�mOBODJBM�BTTFU�FRVJUZ �HPWFSONFOU�CPOEy�JT�EFSJWFE�GSPN�UIF�QSJDF�
of the underlying asset and follows a relationship linked to the relative cost of carrying the
GVUVSF�DPOUSBDU�BOE�UIF�VOEFSMZJOH�mOBODJBM�BTTFU��
In the case of commodity futures, the relationship is, in most cases, inverted because
CVZJOH�UIF�VOEFSMZJOH�QIZTJDBM�DPNNPEJUZ�JT�NVDI�NPSF�EJGmDVMU �DVNCFSTPNF�BOE�
costly (transportation costs, storage costs, etc.) than buying a government bond or
B�CBTLFU�PG�TIBSFT�PO�UIF�TUPDL�FYDIBOHF��$POUSBSZ�UP�mOBODJBM�GVUVSFT�PO�TFDVSJUJFT �
DPNNPEJUZ�GVUVSF�QSJDFT�mOE�UIFNTFMWFT�JO�UIF�QPTJUJPO�PG�ESJWJOH�UIF�QSJDFT�PG�UIF�
underlying assets.
64 via Michael Masters testimony before the Commodities Futures Trading Commission, 25 March 2010.
Speculation brings social value only when it remains a minority activity
But speculators today make up the biggest part of the market
8QOLNH�ÀQDQFLDO�DVVHWV��commodity futures drive the price of the underlying commodity
Finance Watch/MiFID2
Investing not betting
39
revenues. A third category of participants, ‘traditional speculators’, play the role
PG�DPVOUFSQBSUJFT�XIFO�DPOTVNFST�PS�QSPEVDFST�EP�OPU�mOE�BOPUIFS�DPNNFSDJBM�
counterparty to hedge their risks. These speculators are looking for a remuneration of
UIFJS�SJTL�CZ�HBJOJOH�GSPN�UIF�VOEFSMZJOH�DPNNPEJUZ�T�QSJDF�nVDUVBUJPO��
Central to protecting the price formation mechanism of these markets is that
speculators be restricted to a minority of participants: indeed as long as this is the case,
their projections will be based, although indirectly, on fundamental supply and demand
factors as these will determine the behaviour of participants looking to hedge. When
speculators gain a dominant position in a commodity derivative’s market, they base their
projections on the potential behaviour of other speculators, thereby disconnecting futures
prices from fundamentals. Producers and consumers make commodities futures markets
FGmDJFOU �OPU�TQFDVMBUPST�
Figure 9: Increasing market share of commodity speculators
Traditional Speculator 16%
7%
Index Speculator
Index Speculator41%
Physical Hedger31%
Traditional Speculator
28%
Physical Hedger 77%
Long Open Interest – 1998 Long Open Interest – 2008
6RXUFH��&)7&�ÀJXUHV��FKDUWV�E\�0LNH�0DVWHUV��%HWWHU�0DUNHWV�64
Orderly functioning of commodity derivatives markets, as described in the previous
paragraph, is not just important to protect the price of instruments traded, it also has a
direct impact on the price of the underlying (physical) commodity. Because commodity
spot markets are so dispersed (due, among other factors, to the cost of transportation),
they have for a long time relied on local supply and demand to determine prices. As
consumption and production went global, the price on spot markets started to be based on futures prices. For most commodities today, the reference price is the futures
QSJDF �BEKVTUFE�UP�MPDBM�TVQQMZ�BOE�EFNBOE�TQFDJmDJUJFT��
This is a very important phenomenon to understand as it is different from what takes
QMBDF�PO�GVUVSFT�NBSLFUT�SFMBUFE�UP�mOBODJBM�VOEFSMZJOH�BTTFUT��5IF�QSJDF�PG�B�GVUVSF�
DPOUSBDU�SFMBUFE�UP�B�mOBODJBM�BTTFU�FRVJUZ �HPWFSONFOU�CPOEy�JT�EFSJWFE�GSPN�UIF�QSJDF�
of the underlying asset and follows a relationship linked to the relative cost of carrying the
GVUVSF�DPOUSBDU�BOE�UIF�VOEFSMZJOH�mOBODJBM�BTTFU��
In the case of commodity futures, the relationship is, in most cases, inverted because
CVZJOH�UIF�VOEFSMZJOH�QIZTJDBM�DPNNPEJUZ�JT�NVDI�NPSF�EJGmDVMU �DVNCFSTPNF�BOE�
costly (transportation costs, storage costs, etc.) than buying a government bond or
B�CBTLFU�PG�TIBSFT�PO�UIF�TUPDL�FYDIBOHF��$POUSBSZ�UP�mOBODJBM�GVUVSFT�PO�TFDVSJUJFT �
DPNNPEJUZ�GVUVSF�QSJDFT�mOE�UIFNTFMWFT�JO�UIF�QPTJUJPO�PG�ESJWJOH�UIF�QSJDFT�PG�UIF�
underlying assets.
64 via Michael Masters testimony before the Commodities Futures Trading Commission, 25 March 2010.
Speculation brings social value only when it remains a minority activity
But speculators today make up the biggest part of the market
8QOLNH�ÀQDQFLDO�DVVHWV��commodity futures drive the price of the underlying commodity
Source: CFTC figures charts by Mike Masters, Better Markets.
Increasing market share of commodity speculators
Finance Watch/MiFID2
Investing not betting
42
ever goes to commodity producers and that calling such funds ‘investment’ funds is
therefore a falsehood: the only proper name to describe commodity index funds is
‘speculation’ or ‘betting’ funds.
Figure 12 shows that assets allocated to commodity index trading strategies have risen
from $13 billion at the end of 2003 to $260 billion as of March 2008, and the prices of the
25 commodities (the orange line in the chart) that compose these indices have risen by an
BWFSBHF�PG������JO�UIPTF�mWF�ZFBST�
)LJXUH�����+RZ�VSHFXODWLYH�ÁRZV�LPSDFW�WKH�SULFH�RI�SK\VLFDO�commodities
100
200
300
400
500
600
700
1970
1972
1974
1976
1978
1980
1982
1984
1986
1988
1990
1992
1994
1996
1998
2000
2002
2004
2006
Mar
ch 2
008
$-
$50
$100
$150
$200
$250
$300
(Bill
ions
of D
olla
rs)
Com
mod
ity In
dex
“Inv
estm
ent”
S&P
GSC
I Spo
t Pric
e C
omm
odity
Inde
x OthersDJ-AIGSP-GSCIS&P GSCI
Source: Goldman Sachs, Bloomberg, CFTC Commitments of Traders CIT Supplement
2. They distort the price discovery function of commodity futures markets, WKHUHE\�PDNLQJ�WKRVH�PDUNHWV�VLJQLÀFDQWO\�OHVV�XVHIXO�IRU�KHGJHUV�
5IJT�QPJOU�JT�FTTFOUJBM��UIF�AmOBODJBMJTBUJPO��PG�DPNNPEJUZ�NBSLFUT�IBT�UIF�FGGFDU�PG�
making commodity futures markets less effective for their real economic purpose,
which is the hedging of risk for natural (real) buyers and sellers of commodities. This
phenomenon happens for the following reason: commodity index speculators all behave
according to one unique trading pattern and this has a strong distorting impact on the
price discovery function of commodity futures markets as huge amounts of liquidity pour
into passive long-only strategies. This, in turn, contributes to making commodity futures
markets less and less economically useful for true hedgers.72
While the traditional commodity speculator can bring liquidity to the market, taking
long and short positions based on price variations (thereby contributing to both increases
and decreases in prices and being able to provide ‘the other side of the transaction’
to hedgers), index funds always ‘consume’ liquidity as they follow long-only strategies,
buying systematically large quantities of commodity derivatives for long periods of
time.73 Moreover, their replication strategy has the mechanical effect of pushing prices
IJHIFS �UIFSFCZ�DSFBUJOH�CVCCMFT�BOE�GFFEJOH�UIF�TFMG�GVMmMMJOH�CVMMJTI�QSPQIFDJFT�GPVOE�JO�
commodity index fund marketing brochures.
Another major impact of index funds, as demonstrated by the team of Professor Bar-
Yam of the New England Complex System Institute (see Box 6), is the increase of volatility
in physical markets. His research demonstrates that two factors play a special role in
agricultural commodity price increases: corn-to-ethanol conversion and speculation
72 For a complete description of this phenomenon, the reader can report to: Michael W. Masters June 24, 2008 ´7HVWLPRQ\ EHIRUH�WKH�&RPPLWWHH�RQ�+RPHODQG�6HFXULW\�$QG�*RYHUQPHQWDO�$IIDLUV�8QLWHG�6WDWHV�6HQDWH�-XQH���������µ
73 Most buyers of these Index-Funds are mutual or pension funds with long-term strategies.
+XJH�LQÁRZV�RI�LQGH[�money distort futures prices for genuine hedgers
Commodity index funds have channeled $500 billion of investment funds into what can only be described as ‘betting’
Source: Goldman Sachs, Bloomberg, CFTC Commitments of Traders CIT Supplement
1998
2008
4
■ How does introduction and adoption of algorithmic (including HFT) trading affect price discovery mechanisms?
■ Is it possible to quantify the interplay between exogeneity (external impact) and endogeneity (internal self-excitation) in price formation?
■ How efficient are financial and financialized commodity markets?
5
Two views on the price discovery mechanism
Efficient Markets (exogenous dynamics)
Prices are just reflecting news: the market fully and instantaneously absorbs the flow of information and faithfully reflects it in asset prices.
In particular, financial crashes are the signature of exogenous negative news of large impact.
News Prices News Prices
“Reflexivity” of markets(endogenous dynamics)
Markets are subjected to internal feedback loops (e.g. created by collective behavior such as herding or informational cascades).
Prices do influence the fundamentals and this newly-influenced set of fundamentals then proceed to change expectations, thus influencing prices.
6
Sources of reflexivity (endogeneity) in financial and financialized markets■ Behavioral mechanisms such imitation and informational
cascades leading to herding; ■ Speculation, based on technical analysis, including
algorithmic trading; ■ Hedging strategies (also increase cross-excitation between
markets); ■ Pricing of “structured products” such as ETFs (also
contribute to cross-excitation) ■ Methods of optimal portfolio execution and order splitting; ■ Margin/leverage trading and margin-calls; ■ High frequency trading (HFT) as a subset of algorithmic
trading; ■ Stop-loss orders and etc.
7
The test subject: HF price dynamics
Time
Price Last transaction priceBest bid priceBest ask priceMid-quote priceTransactionMid-quote price change
Limit orders to sell
Limit orders to buy
Bid
Ask
Sell market order
Buy market order
Price
8
The model: Self-excited Hawkes process
Applications of the Hawkes model: ■ High-frequency price dynamics ■ Order book construction ■ Critical events and estimation of VaR ■ Default times in a portfolio of companies
Self-excited Hawkes process is the point process whose intensity λt(t) is conditional on its history:
0 100 200 300 400 500 600 700 800 900 10000
20
40
60
80
100
120
Time
Intensity
Exogenous activity
Endogenous feedback
Background intensity Self-excitation part
■ Triggered seismicity (earthquakes) ■ Sequence of genes in DNA ■ Epileptic seizures of brain ■ Crime and violence propagation 9
�(t|Ft�) = µ+ nX
ti<t
�(t� ti)
Branching structure of Hawkes process
Crucial parameter of the branching process is the “branching ratio” (n), which is defined as an average number of “daughters” per one “mother”
For n < 1 system is subcritical (stationary evolution)For n = 1 system is critical (tipping point)For n > 1 system is supercritical (with prob.>0 will explode to infinity)
In subcritical regime, the branching ratio (n) is equal to the fraction of endogenously generated events among the whole population.
Time0 0 01 1 1 1 1 1 1 12 2 2 22 22 2 23 3 3 34
n = 0.88
10
Calibration of the model
§ Maximum Likelihood method Estimation of the parameters can be performed by maximizing log-likelihood function, which is given by the expression: !!!
§ Residual analysis Under the null hypothesis that the data ({ti}) was generated by the Hawkes process with given parameters, the following transformed point process ({τi}) should be Poisson with unit intensity:
logL(t1, . . . , tN ) = �Z T
0�(t|Ft�)dt+
NX
i=1
log �(ti|Fti�)
t̃i =
Z ti
0�(t|Ft�)dt
11
Calibration issues. Kernel
§ Exponential kernel !!
§ Power law kernels (a) Omori-type kernel
(b) Power law kernel with cut-off
(c) Approximate power law kernel
�(t) =1
⌧e�t/⌧�(t)
�(t) =✓c✓
t1+✓�(t� c)
�(t) =✓c✓
(t+ c)1+✓�(t)
�(t) =1
Z
"M�1X
i=0
1
⇠1+✓i
exp
✓� t
⇠i
◆� S exp
✓� t
⇠�1
◆,
#, ⇠i = cmi
12
Fraction of modified durations, %
n est
0 0.5 1 1.5 2 2.5 30.6
0.7
0.8
0.9
1
1.1
Calibration issues. Kernel: sensitivity to outliers
Table 1: Empirical quantiles and maximum values of inter-quote durations (between consecutive mid-quote price changes during Regular trading Hours) of E-mini S&P 500 Futures Contracts in differenttime periods. Values are given in seconds.
Date from Date to Q90 Q95 Q99 Max
01-02-2002 01-04-2002 13.7 20.6 41.7 458.901-02-2006 01-04-2006 23.3 39.6 90.4 933.101-02-2009 01-04-2009 5.1 8.7 19.4 329.901-02-2011 01-04-2011 4.2 10.8 38.7 888.0
Table 2: Theoretical quantiles and maximum values of inter-event durations for time series generatedwith the Hawkes process with the approximate power law kernel (7) for µ = 0.02, ϵ = 0.15, n = 1.0and τ0 given in the first column. The data is obtained by numerical simulation of the Hawkes processon the interval (0, 108 + 105] with burning of the interval (0, 108].
τ0 Q90 Q95 Q99 Max
1.0 4.2 5.8 10.1 29.50.1 2.3 3.4 6.2 22.20.01 0.9 1.5 2.8 10.7
For our tests, we introduce a few outliers (extreme inter-event intervals) in synthetictime series generated by the Hawkes process, so as to mimic the phenomenon observedin Table 1 compared with Table 2. We create different synthetic time series of theHawkes process, with duration (0, 105 + 104] seconds and fixed exogenous intensityµ = 0.3 and branching ratio n = 0.7 using
(i) the exponential kernel (4) with τ = 0.1 or
(ii) the power law kernel (5) with c = 0.1 and θ = 0.5 or
(iii) the approximate power law kernel (7) with τ0 = 0.1 and ϵ = 0.5.
In order to get rid of the edge effects, we burn the initial period (0, 105] seconds (wediscuss the impact of the edge effect in details in section 3.3). We then replace asmall fraction of the durations in these sets with values that are M-times (M = 2 andM = 5) larger than the maximum observed value of the initial synthetic time series.On these time series with a small fraction of outliers, we calibrate the Hawkes modelwith the same kernel as the one used to initially generate the synthetic time series.This is repeated 100 times to obtain a statistical average and standard deviation of thebranching ratio n.
The results are shown in Figure 1, which gives the estimated branching ratio as afunction of the fraction of introduced outliers for the three types of memory kernels.One can observe that the estimations of the time series generated with an exponential
8
same kernel as the one used to initially generate the synthetic time series. This is repeated
100 times to obtain a statistical average and standard deviation of the branching ratio n̂.
In Table 3.3, we show the estimated criticality index (branching ratio) as a function of the
number of introduced outliers for the three types of memory kernels. One can observe that
the estimations of the time series generated with an exponential kernel are robust to the in-
troduction of outliers, as the estimated n remains within one standard deviation (which is
approximately equal to 0.035) of the true value 0.7 used to generate the synthetic time series.
In contrast, estimation for power law kernels shows a lack of robustness as even a small frac-
tion of outliers can significantly bias upward the estimated parameter. Just a single outlier
Table 3.1: Empirical quantiles and maximum values of inter-quote durations (between con-
secutive mid-quote price changes during Regular Trading Hours) of Futures on
the Hang Seng Chinese Enterprise Index (HCEI/SEHK), the Hang Seng Index
(HSI/SEHK) and the Futures spread on KOSPI200 (KS/KRX) in different time pe-
riods. Values are given in seconds.
Contract Date from Date to Q90 Q95 Q99 Max
HCEI/SEHK
2011-11-01 2011-12-01 0.8 1.4 3.2 1230.6
2012-03-01 2012-04-01 1.2 2.0 4.5 1378.8
2012-08-01 2012-09-01 2.1 3.7 9.0 1735.0
2013-03-01 2013-04-01 1.7 2.8 6.7 1517.4
HSI/SEHK
2011-11-01 2011-12-01 0.7 1.1 2.6 733.3
2012-03-01 2012-04-01 0.9 1.5 3.4 1241.5
2012-08-01 2012-09-01 1.4 2.3 5.4 1369.1
2013-03-01 2013-04-01 1.4 2.3 5.2 1331.9
KS/KRX
2011-11-01 2011-12-01 4.1 8.4 21.3 139.3
2012-03-01 2012-04-01 9.1 19.3 54.1 348.7
2012-08-01 2012-09-01 6.0 13.6 42.1 473.1
2013-03-01 2013-04-01 6.9 15.3 46.5 357.3
Table 3.2: Theoretical quantiles and maximum values of inter-event durations for time series
generated with the Hawkes process with the exponential kernel (2.4) for µ= 1, n =0.7 and τ given in the first column. The data is obtained by numerical simulation
of the Hawkes process on the interval of 10 minutes (600 seconds).
τ0 Q90 Q95 Q99 Max
1.0 0.7 1.1 2.2 5.4
0.1 1.0 1.6 3.2 7.2
0.01 1.1 1.9 3.5 8.3
18
Empirical quantiles of inter-quote durations in E-mini S&P 500
Futures Contracts within RTH
Theoretical quantiles of inter-event durations for Hawkes process with
exponential kernel and µ=1 and n=0.7
Sensitivity of the estimation of branching ratio (n) to “outliers” in
inter-event durations
— power law kernel (small outliers) — power law kernel (large outliers) — exponential kernel (large outliers)
• Filimonov V., Sornette D. (2013) Working paper. arXiv:1308.6756
Data source: TRTH
13
Branching ratio, n
Estim
ated
bra
nchi
ng ra
tio, n
(i)
(ii)
(iii)
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
1.1
Calibration issues. Kernel: regularization
• Filimonov V., Sornette D. (2013) Working paper. arXiv:1308.6756
Sensitivity of the estimation of branching ratio (n) to the mis-specification of the power law kernel
— Hawkes model with approximate power law kernel being calibrated on the data generated with Omori-type kernel — Hawkes model with Omori-type kernel being calibrated on the data generated with approximate power law kernel
14
10−610−4
10−2100
102
10−4
10−2
100
1024.8
4.9
5
5.1
5.2
5.3
5.4
5.5
5.6
5.7
5.8
o0¡
Cos
t fun
ctio
n
cθ
Calibration issues. Multiple extremaSurface of the reduced cost-function used for calibration of the Hawkes model on the mid-
price changes of E-mini S&P 500 Contracts in March 1 - April 30, 2001, using the data randomized within millisecond intervals (see paper for details)
µ = 0.3031 n = 0.0751 c = 0.00028 θ = 2.4604
µ = 0.0150 n =1.1054 c = 2.8089 θ = 0.1442
• Filimonov V., Sornette D. (2013) Working paper. arXiv:1308.6756
Data source: TRTH
15
1998 2000 2002 2004 2006 2008 2010 20120
5
10
15
20
Year
Ove
rnig
ht fr
actio
n of
dai
ly vo
lum
e, %
1998 2000 2002 2004 2006 2008 2010 20120
10
20
30
40
50
60
70
Year
Ove
rnig
ht fr
actio
n of
dai
ly m
id−p
rice
chan
ges,
%
Calibration issues. RTH and overnight trading
• Filimonov V., Sornette D. (2013) Working paper. arXiv:1308.6756
Fraction of total daily volume (left) and total daily mid-quote price changes (right) that is observed outside of Regular Trading Hours (9:30 to 16:15 CDT)
on E-mini S&P 500 Futures Contracts.
Data source: TRTH
16
0
250
500
2002
0
200
400
0
200
400
2004
0
150
300
0
200
400
2006
0
200
400
0
1000
2000
2008
0
750
1500
0 500 1000 1500 20000
750
1500
2009
0 200 400 600 800 1000 1200 14000
500
1000
0
10000
20000
2010
0
1000
2000
0
7500
15000
2011
0
700
1400
0 20 40 60 80 1000
7500
15000
Time between FAST/FIX packages, msec
2012
0 5 10 15 20 250
300
600
Processing time, msec
Calibration issues. Resolution of timestamps (I)
• Filimonov V., Sornette D. (2013) Working paper. arXiv:1308.6756
Data source: TRTH
Histograms of the time between consecutive FAST/FIX packages (left panels) and overhead for the data processing (right panels) for E-mini S&P 500 Futures Contracts over RTH
17
FAST/FIX Package 3FAST/FIX Package 21 second
Δ
Events atthe “Exchange”
Packages atthe “Collection”
Eventsrandomized
100 101 102 1030
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
6, msec
Bran
chin
g ra
tio, n
Calibration issues. Resolution of timestamps (II)
• Filimonov V., Sornette D. (2013) Working paper. arXiv:1308.6756
Illustration of the randomization procedure, when the resolution of timestamps is mis-specified.
Bias in estimation of the branching ratio (n) that results from improper assumptions on the duration ∆ of randomization intervals, when real inter-packet time is 1 second. !— exponential kernel (n=0.5) — power law kernel (n=0.5) — Poisson process (n=0)
18
0
0.5
1
Sep
04
Raw data
0
0.4
0.8After "detrending"
0
0.4
0.8
Sep
17
0
0.4
0.8
0
3
6
Sep
18
0
2.5
5
0
0.4
0.8
Sep
19
0
0.4
0.8
0
0.75
1.5
Oct
11
0
0.7
1.4
10:00 12:00 14:00 16:000
0.4
0.8
Time (EST)
Oct
29
10:00 12:00 14:00 16:000
0.25
0.5
Time (EST)
Calibration issues. Intraday trends
• Filimonov V., Sornette D. (2013) Working paper. arXiv:1308.6756
Data source: TRTH
Unconditional intensity of flow of mid-quote price changes of E-mini S&P 500 Futures Contracts on some dates of September–October, 2007. !Left panels present the raw data (black bars) and the average intensity over the period of September 1–October 30, 2007 (red line). !Right panels present the unconditional intensity after “detrending” using the average intensity.
19
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 20.5
0.6
0.7
0.8
0.9
1
1.1
µ2
Bran
chin
g ra
tio, n
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10.5
0.6
0.7
0.8
0.9
1
1.1
n2
Bran
chin
g ra
tio, n
Calibration issues. Nonstationarity (I)
20• Filimonov V., Sornette D. (2013) Working paper. arXiv:1308.6756
Bias of the estimation of the branching ratio (n) in case of regime switch in background intensity (concatenation of 2 independent
samples with µ1=1 and µ2, n=1)
Bias of the estimation of the branching ratio (n) in case of regime switch in branching
ratio intensity (concatenation of 2 independent samples with n1=0.5 and n2)
0
1000
2000
3000
4000
5000
6000
7000
8000
2002 0
5000
10000
15000
20000
2007 0
10000
20000
30000
40000
50000
2011
Calibration issues. Nonstationarity (II)
21• Filimonov V., Sornette D. (2013) Working paper. arXiv:1308.6756
Data source: TRTH
Dynamics of daily numbers of mid-quote price changes counted over RTH for the Front Month Contract of the E-mini S&P 500 Futures
(time period of February 1 to April 1 in three different years)
9
Nonfarm Payrolls -- June 1, 2012
134−09 134−10 134−11 134−12 134−13 134−14 134−15 134−16 134−17 134−18 134−19 134−20 134−21 134−22 134−23 134−24 134−25 134−26 134−27 134−28 134−29
07:26:00 07:27:00 07:28:00 07:29:00 07:30:00 07:31:00 07:32:00 07:33:00 07:34:00
CDT on Fri 01 Jun 2012
2000 lots
PCE
● ●●●●●●●●●● ● ● ●
●●●● ●
● ●
●●●●●
●
●●●●●●●
●
●
●
● ● ●●●●●●
●●●●●●●●●●●●●● ●● ● ●
●●
●●
●
●●
●●●●●●● ● ●●●●●●●●
●
●
●
●
●●
●●●●●●●●● ● ●●●●●●●●
●
●●●●●●●●●●●●●●●●●●●● ●
●● ●●
●
●
●●●●●●●●●●
●
●
●●●
●●●●
●● ● ●●●●●●
●
● ●●●●●●●●
●●●●●●●
●
●●●●●●
●
●●●●●●●●●●●●●● ●
●●●●●
●●●
●●●●● ● ●●●●●●●●●●
●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●
●●●●●●● ●
●●●
●●●●
● ●●●●
●
● ● ● ●
●●●●●●
●●●●●
●● ●
●
●●●●●●●
●●
●●●●●●
●
●●●
●
●●●●
●●●●● ●●●
●●
● ● ●●●●●●●●●
●
● ●●●●●●●●●●●
●●●
●
●
● ●●
●●
●●
●●●●●●●●●●
● ● ●●●●●
●●●
●●●●
●●
●
●
●●
●●●●
●●●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●●●●
●
●
●
●
●
●
●
●●
●
●●●●●●●●●●●●
●
●●
●●
●
●
●
●
●
●
●●
●
●●●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●
●●●●●●
●●●
●
●
●●
●●
●
●●
●
●
●
●●●●
●
●●●
●
●●
●
●
●●●
●●●●
●
●●●●●●●●●●●
●
●
●
●
●
●
●●
●●●●●●
●
●
●●●●●●
●
●●
●●●●●
●●●●●●●●
●●●●●●
●●●●●●
●
●●●
●
●
●●●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●●●●
●
●●●
●●
●
●
●
●●●●
●●●
●
●●●●●●●●●
●
●●●●●
●●●●●●
●
●●●●●●●
●
●●●●●●●●●●
●
●
●
●●●
●
●●
●
●●
●
●●●●●
●
●
●●
●
●
●
●●●●●●●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●●●
●●
●●
●
●
●
●
●
●
●
●
●
●●●●●●●●●●●
●
●
●
●
●
●
●●●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●●●●●●●
●
●
●
●
●
●
●
●
●●
●●●
●
●
●●●●●
●
●●●●
●
●●●●●●
●
●
●●●●●●●●●●●●●●
●
●
●
●●●●
●●
●
●●●●●●●●●●
●
●
●●
●
●
●
●
●●
●
●●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●●
●
●
●
●
●●
●
●●●
●
●●
●
●
●●
●●●●●●●●
●●●
●●●●
●●●
●●
●
●●
●●
●
●●●
●●●
●●
●
●●
●
●
●
●
●
●
●●●●●●
●●
●
●●
●●●●●●●●
●
●●
●
●
●
●●●●●●●
●
●
●
●
●
●
●●
●●
●●
●
●●●●●
●
●●
●●
●
●
●
●
●
●
●
●
●●●●●
●●●●●●
●
●●
●
●●●●●●●●●
●
●●
●
●●
●
●
●●
●●
●●
●●
●
●
●●●●●●
●
●
●
●●●
●
●●
●●●●●
●●●
●
●
●
●●
●
●
●●●●●
●
●●●●●●●●
●
●
●●
●●●●
●●●●●●●●●
●
●●●
●●●
●
●
●●
●
●●
●●
●●●
●
●●
●●
●●●●●
●●
●
●●●●●●●●●●●●
●●●●
●
●●
●
●●●●
●
●
●
●
●
●
●●
●
●
●●●●
●
●●●
●
●●●●
●
●
●●
●●
●●●●●
●
●●
●●●●
●●
●
●
●
●
●
●●●●
●●●
●
●●●●
●
●●●●●●●●●●●●
●
●
●●●
●●●
●
●●●●●●●
●
●●●●●●●●●●
●
●●●●●●●●
●
●●●●●●●●●
●●●
●
●
●●●●●●●●●●●●●●●
●
●
●
●●●
●
●
●●●●●●●●●●●●●●●●●●●●●●●●●
●
●●●●●
●
●●●
●
●●●●●●●
●
●●●●
●●●
●●●●
●
●●●●●●●●●●●●●●●●
●●
●●●●●
●
●●●●
●●
●●●●●
●●●●●●●●●●●●●●●●
●
●●●●●●●●●
●
●●●
●●●
●●●●●●●●●●●
●
●●●●
●
●
●●●●●
●●
●●●●●●●●●●●●
●●●
●
●●
●●●●●●●●●●●
●
●●
●
●●●
●●
●
●
●●●●●
●
●●
●●●
●●●
●
●●●●
●
●●●●●●
●
●
●●●●
●
●●●●●●●●●●●●●
●
●●●●●●●●●
●
●●●●
●●●●
●
●●●●●●●●●
●●●●●
●●●
●●
●
●●●
●
●
●
●●●●●●●●●●●●●●●●●
●●
●●●●●●●
●●
●
●●●●●●
●●●●●●●●
●●
●●●●●●●●●●●●●●
●●●●
●●●●●●
●●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●●●
●●●●●●●●●●●●
●
●●●●●●
●
●●●●●●●●●●
●●●●
●●
●●●●●●
●
●
●
●
●
●
●
●●●●●●●●●●●●●●●●●●●●●●●
●
●
●
●
●
●●●●●●●●●●
●●●●
●
●●●●●
●
●●●●
●●●
●
●●●●
●●●●●●
●
●●●●
●
●●●●●
●●
●●●
●
●●●●●
●
●
●
●
●
●●
●
●●●●●●●●●●●●
●
●
●
●
●
●
●
●●●●●●●
●
●●
●●
●
●●
●●
●●●●●●●●●
●
●●●●●
●
●●●●●●●●●●●●
●●●
●●
●
●●
●
●●
●
●●●
●
●
●
●●●●●●●
●
●
●
●●
●
●
●
●
●
●●●●●●●●
●●
●●●●
●●
●
●
●
●
●●
●
●
●●●●
●●
●●
●
●●●●●●●●●●
●
●●●
●
●●●●
●
●●●●●●●●●●●●●●●●●●●●●●●●
●
●●●
●
●●●●●●●
●
●●●
●●
●●●●
●
●●●
●●
●●●●●●●●●●●●
●●●
●●●●●●●●●●●●●●●●
●
●●●●
●
●
●●●●●●●●●●●●●
●
●●●●
●
●
●
●●
●
●●
●
●●
●
●●●
●●
●●●●●●
●
●●
●
●●
●
●
●
●●●●●●●●●●●●●●●●●●
●●●
●
●
●●●
●●●●●●
●●●●●●●
●●●
●●
●
●●
●●●●●●●●●●
●
●●●●●●●●●●●
●
●
●●●●●●●●●●●
●
●●●●●●
●●●●●●●●●●●
●
●
●●●●●●●
●
●●●●●●●●●●●●●●
●●
●●●●●●●●●
●●●
●●
●●●●●●●●●●
●
●●●●●●●●●
●
●●●●●●●●●
●●●
●●●●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●●●●●●●●●●●
●●●●●●●●●
●
●
●
●●●●●●●●●●●
●
●
●
●●●●●●●●●
●
●●●●●●●●●●●●●●●●●●●
●
●
●●
●●●●●●●●●●●●
●
●
●●●
●
●
●
●●●
●●●●●
●
●●●●●●●●●●●●●●●●●●●●●●●●
●
●
●
●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●
●
●●●●●●●●●
●
●●●●
●
●
●●●●●●●●●●●●●●●●●●●
●
●
●●●
●●●●●
●
●●●●●●●●●
●●
●
●
●
●
●●●●●●●●
●●
●●
●
●
●
●●●●●●●●●
●●
●●●●●●●●●●●●●●●●●●
●
●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●
●●●
●●
●●
●
●●●●●●●●●●●●●●●●
●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●
●●●●●●●●
●
●
●
●●●●●●●●●●
●
●●●●●●
●
●●
●●
●●●
●
●●●●
●
●●●●●●●●●●●●●●●●●●●●●
●●
●
●●
●
●●●●●●●●●
●●●●●●
●
●
●
●●●●●●●
●
●●
●●●●●●●●
●
●●●●●●●●●●●●
●
●●
●●
●●●●●●●●●●●●●
●
●●●●●●●●●●
●●●
●●●●●
●
●●●●●●●●●
●●
●
●●
●●●●●●
●
●●●●●
●
●●●●●●●●●●●●●●
●●
●
●
●
●●●●●●●●
●
●●●●●●●●●●●
●
●●●●●
●
●
●●
●●
●●
●
●
●●●●●●●●●●●
●●●●●●
●●
●
●
●
●●
●●●●
●
●●●
●
●●●●●●
●
●●●●●●●
●
●●●●●●●●●
●
●
●●●
●
●
●
●●
●●●●●●●●●●●●●●●●●●●●●●
●
●●●●
●●●●●
●●●●
●●●●
●●●●●●●●
●
●●●●●●
●
●●●
●●●●
●●●●●●●
●
●
●
●●●
●
●
●
●●
●●●●
●●
●●●
●●●●●●●●●●
●
●
●
●●●●●●●●●●●●●●●●●●●●●●●●
●
●●●●●●●●●●●
●●
●
●
●●●●
●●●●●●
●
●
●●●●●●●●●●●●●●
●●●●●●●●●●●
●
●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●
●●●●●●●●●●●●●●●●●●●
●●
●●●●●●●●●●●
●●●●●
●
●
●
●●●●●●●●●
●●
●●
●
●●
●●●●●●●●
●●●
●
●●
●●●●●●●●●●●●●●●●●
●
●
●●●●●●●●●●
●
●●●●●●●●●●●●●●●●●●●●●●●●●
●●
●●●●●●●●●●●●
●●●●●●
●
●●●●●●
●●●●●●●●●●
●●
●
●●●●●●
●
●●●●●●●
●
●●
●●●●●●●●●●●●●●●●●●
●
●●●●●●●
●
●●●●●●●●●●●●
●
●●●●●●
●
●●●●●●●●●●●
●
●●●●
●●●●●●●●●●●
●
●
●
●●●
●●●●●●●●●●●●●●●●●●
●
●●●
●●
●●
●●●●●●●●●●●●
●
●●●●●●●●●●●●●●●●●●●
●
●●●●
●
●●
●● ●●●●●●●●
●
●●●●
●
●●●●●●●
●
●●●●●●●●●●●●●●
●
●
●●●●●●
●
●●●●
●●●●●●●●●●●●●●●●●
●
●●●●●●●
●
●●
●
●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●
●
●●
●●●●●●●●
●
●●
●
●
●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●●●
●
●●●●●●●●●●●●●●●●●●●●●●●
●
●
●●
●●●●
●●
●●●●●●●●●●●●●●●●●●
●
●●●●●●●●●●●●●●●●●●●●●●●●●●
●
●●●●●●●●●●
●
●●●●●●●●●●●●●●●
●
Sept 10-yr
Calibration issues. Nonstationarity (III). Scheduled macroeconomic announcements
22
Source: R. Almgren (2012)Quantitative Brokers
Calibration issues. The choice of proxy
23
Dynamics of last transaction price (red) and mid-quote price (blue)
Dynamics of bid (red), ask (blue), mid-quote price (green) and micro-price (black)
Methodology
61.8
62
62.2
62.4
62.6
Pric
e
March 23, 2007
09:00 10:00 11:00 12:00 13:00 14:000
50
100
150
200
250
Time
Inte
nsity
10:30 10:32 10:34 10:36 10:38 10:4062.2
62.25
62.3
Price
n = 0.43
■ We split the entire interval of the analysis (2005-2012) into 10 minutes intervals, rolling them with a step of 1 minute within the RTH
■ In each of these windows we have calibrated the Hawkes model with the short-term exponential kernel on the timestamps of mid-quote price changes
■ Each calibration resulted in a single estimation of the branching ration (n)
■ We have performed residual analysis and rejected “bad” fits (using KS-test)
■ Collecting all estimates for each month (~6000-7000 estimates) we have averaged them to construct the “endogeneity index” for the given month 24
Mechanisms of self-reflexivitymilliseconds seconds minutes hours days weeks months years
High-frequency trading
Stop-loss orders
Algorithmic trading
Optimal execution
Margin calls
Long-term herding
Imitation
25
Benchmark: Financial markets (E-mini S&P 500)
0
50M
100M
Volu
me
0
1M
2M
3M
4M
Num
ber o
f eve
nts
Monthly volumeNumber of events per month
0
0.05
0.1
Vola
tility
500
1000
1500
Pric
e
Daily volatilityDaily closing price
Back
grou
nd a
ctiv
ity
0
0.5
1
Year
Bran
chin
g ra
tio
1998 2000 2002 2004 2006 2008 2010 20120.3
0.4
0.5
0.6
0.7
0.8
Trading activity proxied by volume and
number of mid-price changes
Dynamics of price and volatility
Rate of exogenous events (triggered by idiosyncratic
“news”)
Branching ratio that quantifies endogeneity of the system
(fraction of endogenous events in the system)
• Filimonov V., Sornette D. (2012) Physical Review E 85(5), 056108 • Filimonov V., Bicchetti D., Maystre N., Sornette D. (2014) J. of Int. Money and Finance, 42, 174-192
Data source: TRTH
26
Crude Oil: Brent and WTI
0
0.05
0.1
Vola
tility
0
50
100
150
Pric
e
Daily volatilityDaily closing price
0
1M
2M
3M
4M
5M
Volu
me
Year
Bran
chin
g ra
tio
2005 2006 2007 2008 2009 2010 2011 2012
0.4
0.5
0.6
0.7
0.8
0
0.05
0.1
Vola
tility
0
50
100
150
Pric
e
Daily volatilityDaily closing price
0
2M
4M
6M
8M
10M
Volu
me
Year
Bran
chin
g ra
tio
2005 2006 2007 2008 2009 2010 2011 2012
0.5
0.6
0.7
0.8
• Filimonov V., Bicchetti D., Maystre N., Sornette D. (2014) J. of Int. Money and Finance, 42, 174-192
Brent Crude (ICE Europe) WTI (NYMEX)
Data source: TRTH
27
Exogenous vs endogenous shocks in HFApril 27, 2010: Significant fall of most of US markets following the cut of the credit rating of Greece and Portugal May 6, 2010 (“flash-crash”): The activity of high-frequency traders of the S&P 500 E-mini futures contracts leaded to a dramatic fall in other markets
Pric
e
April 27, 2010
A11170
1180
1190
1200
1210
Volu
me
B1
0K
20K
40K
60K
80K
100K
Tota
l rat
e
C1
0
0.5
1
1.5
2
2.5
3
Time, EST
Bran
chin
g ra
tio
D1
09:30 10:30 11:30 12:30 13:30 14:30 15:300.6
0.7
0.8
0.9
May 06, 2010
A21060
1080
1100
1120
1140
1160
B2
0K
20K
40K
60K
80K
100K
C2
01234567
Time, EST
D2
09:30 10:30 11:30 12:30 13:30 14:30 15:300.6
0.7
0.8
0.9
May 6, 2010April 27, 2010
Volume and Trading activity behave similar in both cases
Branching ratio (“endogeneity index”) reveals fundamental difference between two shocks
Source: V. Filimonov, D. Sornette (2012) PRE 85 (5): 056108. 28
References
§ Filimonov V., Sornette D. (2012) Quantifying reflexivity in financial markets: Toward a prediction of flash crashes. Physical Review E, 85(5), 056108. doi:10.1103/PhysRevE.85.056108, http://ssrn.com/abstract=1998832
§ Filimonov V., Bicchetti D., Maystre N., Sornette D. (2014) Quantification of the High Level of Endogeneity and of Structural Regime Shifts in Commodity Markets. Journal of International Money and Finance, 42, 174-192. doi:10.1016/j.jimonfin.2013.08.010, http://ssrn.com/abstract=2237392
§ Filimonov V., Sornette D. (2013) Apparent criticality and calibration issues in the Hawkes self-excited point process model: application to high-frequency financial data. arXiv:1308.6756
§ Filimonov V., Wheatley S., Sornette D. (2013) Effective measure of reflexivity of the self-excited Hawkes and Autoregressive Conditional Duration point processes. arXiv:1306.2245
§ Wheatley S., Filimonov V., Sornette D. (2014) Estimation of the Hawkes Process with Renewal Process Immigration using an EM Algorithm. Working paper
29
