Equity Trading: Performance, Latency & ThroughputPrepared for the 2012 Exactpro ExTent Conference, Kostroma, Russia.11° April 2012

Author: Phil Penhaligan

2

Agenda

• Low Latency– What are we measuring?– How Fast Is Fast Enough?– Who really needs it?– What’s the next best thing?– The Technology

• How did we get here?• What next?

• Performance & Capacity Management– The LSE/TQ KPIs– System behaviour under extreme conditions

3

Low Latency: What are we measuring?

• 3 distinct attributes– Measured from just outside LSE/TQ firewalls– Excludes transmission time to/from external client/venue– Excludes latency of client/venue’s own system(s)– Excludes network packet retransmissions due to slow

consumers

1. Order latency• Time from Order receipt to order acknowledgment• Commonly referred to as Order:Ack

2. Market Data latency• Time from Order acknowledgement to public

broadcast Commonly referred to as Ack:Tick

3. Reference price latency (Turquoise Dark Book only)

• Time from Tick receipt to matching engine usage

4

Low Latency: What are we measuring?

• 2 key metrics– Average

• Mathematical mean– Consistency

• Max 99.9 percentile– Discard the worst 0.1% of orders and then take the

maximum

• Production Systems– Continuous real-time capture– Historical data stored in database for statistical analysis and

reporting

• QA Systems– New software releases evaluated during technical tests– To confirm/verify expected latency improvements– To detect unexpected latency regressions

5

Low Latency: Example – Turquoise Order:AckLatency Distribution Average and Consistency

Metric uS

Average 100% 108

Average 99.9% 103

Max 99.9% 381

Max 99 191

6

Low Latency: How Fast Is Fast Enough?• Customer feedback

– LSE/TQ averages are already fast enough – better Consistency is most important!

• Our Focus– Since the initial deployment of Millennium (Oct 2009 Turquoise and Feb 2010

LSE) • We have improved order:ack average by 30% & consistency by 90%• Multicast ack:tick has improved by 50%

– We continue to focus on improving consistency (see later slides for details)• My Observations

– Since migration to Millennium IT, the exchange latency is an order of magnitude better than latency within client applications + transmission to/from our venues

• Therefore for the majority of clients who are not co-located, further improvements to latency will have little impact

• E.g. a 10% improvement to exchange latency = client 1% improvement

– Co-location customers whose algorithms interact exclusively with LSE/TQ markets have the most to gain from further reductions in latency

– Our latency is a tiny fraction (<0.0005%) of human reaction times (typically <200mS)

7

Low Latency: Who really needs it?

• Clients/Participants

• The Venue/Exchange– Venues with the lowest latency Market Data are at the

front of the queue when BBO updates occur– Quality of dark book trades depends on ref price latency– Builds strong technology reputation– Generally good for Sales & marketing

Category Least Sensitive Most Sensitive

Manual Trading Brokers,“High Touch” Traders.

Arbitrage Traders,Portfolio Traders.

Automated Trading Retail Service Providers (RSP),DMA Clients.

Market Makers.

Algorithmic engines & Smart Order Routers.

Remote Algorithms e.g. VWAP, POV, IS typically driven more by historical data curves than real-time ticks.

HFTs, Best Execution SORs,Co-lo Algorithms (e.g. arbitrage, momentum).

8

Low Latency: What’s the next best thing?

• Consistency– Fewer outliers, and outliers closer to the average rather

than simply seeking ever lower average latency is the most important feature for any trading strategy whether manual or automated.

– If your system cannot compete on latency average, you can always make improvements by making it more consistent.

• Resilience– Closely tied to consistency is resilience, since an

unresilient system will never be able to provide consistent results!!

9

Low Latency: Technology

MIT/LSE/Turquoise Customers

How did we get here? • C++• Linux• Infiniband• Native + ITCH• 1Gb Extranex

• Fibre Optics• Co-Location• FPGA

What next? • Rebalancing• More channels• More partitions• More threads• FPGA + processors• Tickerplant• Lower level API• Generation 8 …• R&D• Watch

competitors!

• More automation• Better Algorithms• Faster switches• Faster firewalls• FPGA + processors

10

Performance & Capacity Management – KPIs

# KPI Requirements / Target

1 Total Daily Transactions Max (4 x average, 2 x peak)

2 Total daily Trades Max (4 x average, 2 x peak)

3 Order Latency Agreed levels per market

4 Market Data Latency Agreed levels per market

5 Reference Price Latency Agreed levels per market (TQ only)

6 Transactions per second (1 sec peak) Max (4 x average, 2 x peak)

7 Transactions per second (10 sec average)

Max (4 x average, 2 x peak)

8 Transactions per second (60 second average)

Max (4 x average, 2 x peak)

9 System Availability 100%

11

Performance & Capacity Management

• Each system is technically tested with every software release to– Prove KPI levels– Reconfirm/prove behaviour and known breaking points

• Preliminary tests take place on pre-production• At least one test cycle takes place on the actual

production hardware on a weekend• Golden Rules

1. A client message will always get a valid response (ack or nack)

2. Any component that has been taken so far above the KPI level that it fails must do so gracefully, and the system must continue to obey golden rule 1.

• Because we know the system/component limitations & bottlenecks we can manage growth as the markets evolve

12

Turquoise Lit Book Matching Engine Peak TPS last 12 months

Performance & Capacity Management

13

To Summarise• Low Latency

– What are we measuring?– How Fast Is Fast Enough?– Who really needs it?– What’s the next best thing?– The Technology

• How did we get here?• What next?

• Performance & Capacity Management– The LSE/TQ KPIs– System behaviour under

extreme conditions

(1) Order:Ack, (2) Ack:Tick, (3) Ref Prices.

Millennium Average OK – Focus on Consistency.

Predominantly Market Makers, HFTs, Co-Lo Algos.

Consistency and Resilience.

C++/Linux/Infiniband/Native/ITCH

Balancing/threading/FPGA/Tickerplant,low level API/Generation 8 …

Trans,Trades,latencies,TPS (1/10/60), Availability.

1. Always respond, 2. Fail gracefully.

14

Questions?