High Frequency TradingSystems Designs

An "Idiots Guide" to the Design and Implementation of High Frequency Trading

Systems using standard CPUs, "Custom chips" and networking components

Created by

Power BroadcastingComputational Research Department

Adelaide / Wellington / Vancouver

This is a thought experiment (Gedankenexperiment) research white paper.This is a thought experiment (Gedankenexperiment) research white paper.This research should not be construed as formal financial advice.This research should not be construed as formal financial advice.

Version : 03 September 2012

How can one understand a complex system? Design one.

Most complex computing technology is especially hard to understand unless you have designed something similar to it.

Computer technology is essentially logic gates fed by Computer technology is essentially logic gates fed by a clock signal, generally with an operating systema clock signal, generally with an operating system.

Complicated systems are often less complex than one might imagine.

The "original system designers" hate complexity too. Complexity++ == Reliability--

What do most honest HF trading systems do?

Carry out programmed trades very quickly.Carry out programmed trades very quickly. The programmed trades must be reliable and traceable,

and must be backed up. HF trading systems must be interoperable with existing HF trading systems must be interoperable with existing

trading systems. trading systems. A lot of sitting and waiting. Yes, this is in the specifications. Trading systems must be made of materials and systems Trading systems must be made of materials and systems

“common enough” to be modular, reparable and “common enough” to be modular, reparable and replaceable. replaceable.

The systems mostly should not be interesting.

What do dishonest HF trading systems do?

Run trades (by using existing intercept technology to intercept "trades in progress" and take countermeasures to obtain a small transaction profit).

Use programmed strategies to manipulate prices to obtain a slightly larger transaction profit.

Work with other similar systems for general market manipulation.

Malfeasance did not come 1st

The dishonest “price and market manipulation” HF trading systems features were not in the original specifications for the technology.

These dishonest features evolved out of a general regulatory vacuum and an unconcerned public in Europe and the Americas and significant parts of Asia.

The honest, passive transactional stuff is in the original design specification for HF trading systems. From this one must work one's way to the original design base systems specificaitons.

What is the fastest way to complete a transaction?

Simplify a "transaction" into its most elementary operations. Do this by using a device independent language like Systems Query Language (SQL).

A 3 to 5 binary record structure can imitate the “inputs and outputs” to a SQL trading database.

~150000 logic gates can do a “trade transaction” with not too much complexity. This is less logic than a 68000 CPU.

SQL to VHDL (Hardware Description Language) translation is not easy!

Translate like this SQL → C++ → (VHDL or Verilog)

The minimalist system will not be capable of doing anything else but completing a transaction.

PS: VHDL ~= Verilog

Poof! Completed.A chip that does simple trades...

It will have to be a FPGA or a similar device. You still have to make it communicate with the

outside world using existing standards – so no proprietary interfaces.

The FPGA will have to live on a removable system board with another CPU, GPU etc...

No cryogenic cooling is assumed here, good designs don't need this botheration (but it does wonders for speed).

What does an FPGA board look like?

On the system board...On the system board...

… … the FPGA now lives. There must be oceans of RAM the FPGA now lives. There must be oceans of RAM memory – and spare CPUs & GPUs (General Purpose memory – and spare CPUs & GPUs (General Purpose CPUs) on the board to be super fast.CPUs) on the board to be super fast.

So order 1000 system boards! There will be burnout. RISC > CISC CPUs : The system board must have a

controlling CPU that is dead simple (yet fast) so you can catch your design flaws.

Now the interfacing software is needed. You need a Real You need a Real Time Operating System (RTOS)!Time Operating System (RTOS)!

Application-specific integrated circuits (ASICs) can also do what FPGAs do, so there will always be more than one way of making your system board...

The system board CPU & GPU...

… has to be RISC (Reduced Instruction Set Computing) aka SPARC, ARM architecture, etc to be fast enough to do this job ...

… has to be multicore, hyperthreading if possible.

Post 2010 GPUs (Graphics Processing Units) will work here too, but are not ideal for this kind of application. Experiment here if you want.

The system board OS ...

RTOSes worth using : VxWorks, Nano-RK, QNX, FreeRTOS, etc ... as Cyclic Executives are just too rigid to be useful.

All the system board OS has to do is run a real time database + applicable support applications and be programmable in real time.

There is a cesspit of complexity here, but somehow the systems actually used are actually very simple.

The server, it holds the system board ... and is not much different.

The servers containing real time trading systems can have more software on them to make them more usable.

Massive parallelism design rules : for reasons of maximizing reliability and bandwidth flexibility.

Interfacing the real time trading system with the "market standard" systems that are slower poses hundreds of buffering and coherency problems.

Deep pockets (aka money) can solve these difficult and obstinate problems.

Interfaces

Your real time trading system has to be accessible on your trading floor (and at home), so it must have a user friendly interface!

A real time trading system that is accessible across the ocean can help.

So London and Amsterdam must be able to trade in New York and Chicago etc ... Sydney and Tokyo.

So, lots of computing power is needed for lots of communications link encryption.

To keep within real time, you need a fiber optic path along all points of the transaction chain.

Transaction redundancy

As a rule, each market transaction must be backed up in at least 10 places to be domestically legal and 20 places to be internationally legal.

Redundancy : It's mostly just temporary buffering ... deletable from 3 days to 3 years.

So, buy lots of Hard Disks (and Redundant Arrays of them aka RAIDs).

On a slow day, a firm might use 20% of a RAID but a fast day may yield 335% RAID consumption.

Yes, there will be LOG FILES!

Malfeasance 101

OK, the HF trading system is up. Now what? Do everything in New York City or London. Not much

regulation here. Anything goes ... and honestly gained normal trading efficiencies made the system pay for itself in 2 weeks.

This system has probably cost 20 millions SDRs, enough to run Radio Canada International for a year – but can it "earn" 120 millions? Maybe.

So, lets get programming …

Oops! Design constraints...

Cross market malfeasance takes work. HF trading systems designed for stocks will not

work in the FOREX market. Several specialized trading platforms need to be built -- and these platforms must talk to each other.

This will take time, and some protocol reworking is needed. Ding, 60 millions SDRs.

Coordinated HF trading systems may have payback horizons of a week vs a month.

Malfeasance 101 : Tax avoidance

A trading system that engages in trades that are designed in such a way so as to avoid taxes can be a real money saver for the firm.

Being a party of both sides of the transaction can really help in tax avoidance.

The real time transactions have to be complex enough so that the Inland Revenue can't comprehend it and thus can't tax it.

So, use the system in the 'black pools' of the unregulated derivative markets.

Malfeasance 101"Price manipulation"

Price Manipulation using HF trading systems is an art and a science. Possibly it is the main reason for ~20% of all HF trades.

Little is actually known about the full extent of price manipulation, but the results can be clearly seen.

HF Trading systems have to be on 'both sides of the transaction' (where possible) else mainly do 'man in the middle' trading attacks.

This amoral practice can be harmless but … megalomania is megalomania is rampant in multi-generational CEOs who have never rampant in multi-generational CEOs who have never actually worked for a livingactually worked for a living. So grand scale price manipulation schemes will come down autocratically and often.

Malfeasance 101 : Tech support

The IT Department that makes all your HF trading possible has to be paid off. This is a Cost of Doing Business write off (in the USA anyway).

Oddly, most (in the know) IT staff that create and oversee these HF trading systems are probably massively underpaid.

IT staffs may be underpaid by an order of 125x to 2048x, so there is plenty of theft going around.

Any finance firm CEO paying their IT departments less than 20% (of all profits extracted from HF trading systems) are begging for 25 years in prison or execution in China.

HF trading strategies, externalities

The cost of having any market sector (like FOREX) saturated with HF trading systems engaging in programmed malfeasance is not obvious to anyone but 'day traders' – who can actually see the outcomes.

You can't beat a fast [and clever] machine that has more knowledge than you do. This is a law of nature.

If HF trading systems are possibly causing you to lose money in the markets -- then HF trading systems are directly causing you to lose money in the markets ...

HF Trading, externalities – fixes

Yes, it is possible to fix the issue of malfeasance in HF Trading systems – but this may take decades for any government to do anything. Governments are sadly in on the corruption in the finance system, as nepotism and favoritism (with the private sector) as all but obliterated functioning government.

However, for an individual there are quick fixes.

Liquidate your holdings cautiously and leave the market.

Most people opt for cash -- and later on Gold or Silver.

Australia, Canada, New Zealand – are not exempt. This is not a problem exclusive to the US or UK.

HF trading, more externalities

HF trading systems alone will not cause the collapse of a nation or region.

Societies (and nations) do truly collapse long before their economies finally collapse.

Failure and collapse has a thousand parentsFailure and collapse has a thousand parents, and HF trading is just one of them.

Complex systems that have become corrupted and unstable implode with clockwork regularity, as a general rule of nature. Finance systems are no exception.

HF Trading, Historical Perspective

HF trading is one of the dozens of significant (but not actualbut not actual) causes of the North American 'Great Depression'. Stock tickers in the 1920s ran at about 5 characters per second, and slowed down the 1929 crash by at least a fortnight.

There have been some flash recessions in the developing world since the 1980s. High speed transaction finance has contributed to at least two (and possibly five) flash recessions since 1975.

Small investors “in the modern era” almost never directly cause flash recessions. These recessions are more the product of large institutions. The small investor has lost ground with respect to the large investor each decade after 1970 -- with respect to causing “money flight events”.

Why small investors will not cause the Why small investors will not cause the next global finance meltdown...next global finance meltdown...

Mainly, it boils down to actual market knowledge and access to information.

Small investors are swamped with seas of useless information – that does little to help them know what is actually going on.

Small investors actually work, and available time is not as great. Institutional investors have time.

Large investors (banks, trading houses etc...) have clear government backing, and deep pockets. This allows for the possibilty of infinite malfeasance.

The “power disparity” is as true in North America as it is in Europe, Asia and Australasia.

End

If you liked this research document, please feel free to make a donation. This research may be suitable for a book or documentary later on.

Please contact the author at : dist23 at juno dot com