What is wrong with the statistics in quantitative trading?

It has been used very commonly to screen good guys from the market with a trading strategy

developed by statistical method (regression mainly). For example, some financial parameters or

combination of parameters selected by conducting regressions are used in stock pick-ups in the

real world.

However, even though tremendous empirical tests based on historical data are conducted before

PMs apply them, those trading strategies built by regression usually fail to bring the expected

return as it was in the empirical tests. There is nothing wrong with the quantitative techniques

themselves. What is wrong is something that happened during the process of applying those

concepts in practice, a very common mistake that exists in asset management field where people

focus only on how they use the tools or formulas rather than going further to have an in-depth

understanding of the original concepts.

In fact, the PMs and analysts who use statistics technology do not really understand these

theories, especially the assumptions based on which they are made theoretically sound. One of

the assumptions for regression is that the population from which historical data are extracted to

conduct empirical test has to be the same one where projections will be applied. As we flip a

coin, the odds are always 50% and 50% because we are using the same coin and the same hand.

However, the financial data of the past several decades is definitely not in the same population as

the next day data is, because the market keeps changing every second. If we look back at today’s

data 300 years later, the current data or the data before today is only a small portion of the whole

population. Then why empirical tests always show a perfect match with financial measurements

generated from regression? It is because all data used in test come from the same period. If we

want to project stock’s performance of tomorrow, we have to conduct regression using the data

including tomorrow. It is certainly impossible to have accurate data of tomorrow included in

today’s test. Therefore, an inaccurate projection is inevitable.

It seems a strategy based statistics method works sometimes. It is all because the stock market

for the period of projection happens to present some similar properties with the market for the

period of generating model and conducting tests, they are not exactly same though. Therefore, if

we are able to identify the factors related the whole market that can be used successfully in

measuring similarity of the different periods of market, it is totally possible to increase the

accuracy of using trading strategy based on regressions in some circumstances.