Introduction to FX Data Mining
Andrew Kreimer
Algonell – Scientific FX Trading
FX, FOREX or Foreign Exchange Market
• The biggest market in the world
• ~5 ∗ 1012 (~5 trillion) daily trading volume (2015)
• 5 days a week, 24 hours a day, Monday – Friday
• Players: three main levels
• Banks, Investment Companies
• Brokers
• Traders
• Speculative nature – most of the trading is just for the difference
• The simple goal - gain difference in pips (1/10000 change of price)
• Buy low, sell high (Long)
• Buy EURUSD @ 1.01000 and then Sell @ 1.02000 +100pips gain
• Sell high, buy low (Short)
• Sell EURUSD @ 1.01000 and then Buy @ 1.00000 +100pips gain
Data Mining and Machine Learning• Data Mining
• Massive amounts of data (We have it in FX)
• Non trivial extraction of knowledge from data (We need it in FX)
• Data Mining methods
• Classification – spam & fraud detection
• Association – YouTube & Amazon
• Clustering – unstructured data
• Process Mining – log mining
• Text Mining – news mining
• Machine Learning
• Algorithms for knowledge discovery
• Neural Networks (Widely used in FX)
• Random Forest (Not appreciated as should be)
• Linear Regression (Too simple, but well known)
Algorithmic Trading or Quantitative Investment
• Trading algorithm
• Mathematics: Fibonacci, Chebysheb, Markov and etc.
• Data Mining
• Trading automatically
• Programming language: MQL, Java, C#, LUA, C++ and etc.
• No psychology
• Speed and robustness
• Deterministic
• Note: in this case it’s long term rather than HFT
Algorithmic FX Trading and Data Mining
• Historical Data
• GIGO – Garbage In Garbage Out
• Broker or Yahoo Finance?
• Programming skills and continuous debugging
• Creating ,implementing and testing (Currently done manually).
• Trial and Error – Explore!
WEKA – Waikato Environment for Knowledge Analysis
• Open Source Data Mining framework
• University of Waikato, Hamilton, New Zealand
• Provides
• Implementation of Machine Learning algorithms
• Data preprocessing filters
• Data visualization
• Software
• http://www.cs.waikato.ac.nz/ml/weka/
• Book
• Data Mining: Practical Machine Learning Tools and Techniques, Third Edition (The Morgan Kaufmann Series in Data Management Systems) by Ian H. Witten et al.
• http://amzn.com/0123748569
Loading numerical data to WEKA
Numerical visualization in WEKA
Predicting Close Price with Random Forest
Loading nominal data to WEKA
Nominal visualization in WEKA
Clustering entry points with Simple K Means
Summary
• FX trading has infinite number of trading systems
• Data Mining can help us in creating unique trading models
• Tools and data are available
• Good luck with the mining!
Profit
FX
Data Mining
Historical Data
Algorithmic Trading