Win Rate: The Most Important Performance MeasureFebruary 9, 2015 5:00 am36 commentsViews: 1867

– Michael Harris, Price Action Lab Blog

Shortly after my post on Kelly maximization I received a number of emails from traders who are developing systems but are, understandably so in my opinion, a bit confused about which performance criterion or criteria to use when evaluating them. I understand why those traders are confused, or to be more exact, what or who has confused them and why.

The most important criterion to use when measuring the performance of a trading strategy is its success rate, a.k.a. win rate. A year ago, in the post “What Every Trader Should Know About the Win Rate, Profit Factor and Payoff Ratio“, I mentioned a formula I derived 20 years ago that first appeared in a book of mine published in 2000 and in a few papers in popular magazines. The formula describes the relationship between the win rate, payoff ratio and profit factor:

w = pf/(pf+r)   (1)

where w is the win ratio, expressed as the ratio of the number of winning trades to the total number of trades, pf is the profit factor calculated as the sum of winning trades divided by the sum of losing trades and r is the ratio of average winning trade to average losing trade, also known as the payoff ratio.

Now, a frequent argument is that the win rate can be low, such as, for example, 30%, but the ratio r can be high enough so that the resulting profit factor is greater than 1, i.e., a profitable strategy. The formula for the profit factor can be derived from equation (1):

pf = w x r /(1-w)   (2)

We can see from equation (2) that if w =0.4 and r = 1.5, then pf = 1.0. Thus, if r is kept above 1.5, then the strategy will be profitable. Then the argument is that r is maybe more important than w, and strategies should be developed for maximum r. For example, trend-following strategies have usually low w but high r.

I will try to shed some light on these issues; Trend-following strategies need to have high r but there is no guarantee for it. It is not up to strategy to decide what value of r it will have as that depends on market conditions. If the market moves sideways, then trend-following generates a profit factor less than 1. This was the case during 2011 with most trend-following funds. The ratio r is not something that can be controlled by the trader. If you rely on hopes then you can measure performance based on the ratio r. But if you rely on skill, then you measure performance based on win rate and for maximum achievable ratio r.

The Source(s) of the Confusion

Why do most traders and system developers prefer to use metrics such as net profit, Sharpe ratio, payoff ratio, profit factor, max drawdown, etc., when developing systems instead of the straightforward win rate?

The answer in my opinion is that finding strategies with high win rate for maximum achievable payoff ratio r is extremely difficult where use of the other ratios often facilitates curve-fitting but the strategies usually have low win rate, in the range of 40% to 60%, but high payoff ratio r, as a result of the optimization. As a matter of fact, this is what most strategy development tools based on neural networks, genetic optimization and programming usually accomplish because of their nature. These algorithmic approaches have been successfully applied to many fields but are misapplied in the case of trading system development. One reason is that they generate TYPE-I optimized systems and the data-mining bias is too high.

Even more important is the fact that the risk of ruin of a trading strategy depends primarily on its win rate. The lower the win rate, the highest the risk of ruin. In the special case of ruin due to consecutive losers, this can be seen from this simple equation:

RoR = (1-w)^R

where ROR is the risk of ruin, w is the win rate and R is the inverse of risk percent. It may be seen that for fixed risk percent, for example 2% of capital, the risk of ruin decreases as w increases. However, consecutive losers are a special case of ruin and in general the probability is higher. This special case was used to show the importance of win rate.

Summary

If you cannot develop a strategy with a sufficiently high win rate, higher than 70% in my opinion, regardless of the value of a sufficiently high payoff ratio, the risk of ruin is high. Strategies with low win rate that appear good during backtesting or even perform well during the first couple of years of actual trading may rely on luck and specifically on the payoff ratio remaining high. Software vendors who implement various types of metrics to assist traders in developing trading systems often do so because that offers many more choices of curve-fitting strategies that appear to have a high profit factor and payoff ratio at the expense of win rate. Such strategies carry high risk of ruin because they make unrealistic assumptions about the future behavior of the markets, such as, for example, that the market will keep on rewarding a trader with a low win rate for an extended period of time.

– Michael Harris, Price Action Lab Blog

Update:  Jeff here…I’ve coded the Win Rate into a function which you can use in your own strategies. Download it here.

Long/Short Symmetry Effect on Trading System PerformanceJanuary 26, 2015 5:00 am1 commentViews: 481

– Michael Harris, Price Action Lab Blog

Although some quants prefer long/short symmetric trading systems, there is no conclusive evidence that they offer any advantages, especially when dealing with markets with a structural bias. In this blog post I present two examples of the performance of trading systems that were machine designed last year.

There are at least two kinds of long/short symmetry: (A) natural, or by design and (B) forced. A 50-200 long/short moving average system is of the former kind because, by design, long trades exit short and short trades exit long. This blog deals with the latter kind, i.e. systems where logic is added to force symmetry. This may be achieved in several ways but the simplest is by removing any exit targets or other exit signals and then letting long signals exit short and short exit long. Protective stops may be kept to limit risk.

SPY Trading System

This system was machined designed by Price Action Lab after the close of the last trading day of 2013 with unadjusted SPY data from inception. Below is the workspace used (results are fully reproducible as no randomness was involved):

Amibroker AFL code for all patterns discovered by Price Action Lab with the above workspace settings was used to backtest the resulting system (no selection bias was involved) on SPY data from 01/02/2014 to 11/07/2014. Below are the results for 2.5% profit target and stop-loss. Commission of one cent per share traded was used in all backtests and the starting capital was $100K. Equity was fully invested.

The YTD return is 10.88% on a total of 26 trades, 19 long and 7 short (more details in the table below).

Next, the machined designed SPY trading system is forced to obey long/short symmetry by removing the stops. The resulting equity performance is shown below:

The YTD return is this case is 14.35%, much higher than in the previous case, on a total of 15 trades, 8 long and 7 short (the last trade is an open long – more details in the table below).

It may be seen that both systems have profited from the recent V-bottom but the long/short symmetric one has achieved that with just one long trade.

click to enlarge

The system with forced long/short symmetry is superior to the original system with the profit target and stop-loss, resulting in a much higher Sharpe ratio and a net return exceeding the 10% buy and hold (no dividend reinvestment) by 4.35%. The original system also exceeded the buy and hold return by just 0.88%.

DIA Trading System

But we should not hurry to draw any conclusions from just one test. Next, we consider a higher reward/risk system for DIA that was machined designed by Price Action Lab on April 21, 2013, as reported in the blog, using an in-sample from inception (01/28/1998) to 12/31/2009. The out-of-sample cross-validation test was done in the period from 01/04/2010 to 03/28/2013. Below is the workspace used (results are fully reproducible):

Amibroker AFL code for all patterns discovered by Price Action Lab with the above workspace settings was used to backtest the resulting system (no selection bias was involved) on DIA data from 01/02/2014 to 11/07/2014. Below are the results for 4% profit target and 3% stop-loss. Commission of one cent per share traded was used in all backtests and the starting capital was $100K. Equity was fully invested.

The YTD return is 17.81% on a total of 9 trades, 6 long and 3 short (more details in the table below).

Next, the DIA trading system above is forced to obey long/short symmetry by removing the stops. The resulting equity performance is shown below:

The YTD return is this case is 9.06%, a lot lower than in the previous case, on a total of 7 trades, 4 long and 3 short (the last trade is an open long – more details in the table below).

It may be seen that the performance of the long/short symmetric system was adversely affected by the recent V-bottom in prices because a long signal arrived late after all the gains from a previous short were erased and in addition the short trade was exited with a loss.

click to enlarge

The system with forced long/short symmetry has a degraded Sharpe ratio although the payoff ratio is higher as expected. Note that both systems exceeded the unadjusted for dividends buy and hold return of 6.10% for DIA, YTD.

Therefore, and as expected, long/short symmetry is a tricky concept. In some cases performance increases substantially but in other cases there is degradation. It is hard to identify the main factors that contribute to better or worse performance that will allow choosing a biased over a long/short symmetric trading system. Possibly, using both types to get average performance may be a better approach and the one I recommend.

– Michael Harris, Price Action Lab Blog

Finding Out What Works, And What Doesn’t WorkSeptember 29, 2014 5:00 am11 commentsViews: 2487

Many traders who try system trading have previously had difficulty at discretionary or manual trading. Most of these folks eventually recognize the benefit of trading a system with well defined rules – a system that has performed well in the past. It is

nice to know a trading approach has historically worked, but as with all things related to trading, past performance is no guarantee of future results.

Unfortunately, many people who try systematic/algorithmic/mechanical/rule-based trading for the first time bring along a lot of the baggage that they have acquired from their previous method. Depending on the pre-conceived notions they bring into mechanical trading, these new systematic traders may run into a lot of frustration and trouble.

Many times, for example, traders will always test with a few core concepts, such as always closing by the end of the day. This is what they were used to as a discretionary or manual trader, and therefore they never even think to test ideas out of their old comfort zone. Perhaps removing these “comfort” rules would dramatically improve performance.

In this article, I will examine three common items that new systematic traders test, and see how these items actually work when they are subjected to rigorous testing.

Ground Rules

In all the testing that follows, I will test in 7 different markets, across a range of commodities:

Wheat (W) 10 Year Treasury Notes (TY) Lean Hogs (LH) Australian Dollar (AD) Heating Oil (HO) Cotton (CT) e-mini Nasdaq (NQ)

I picked these at random, one from each major market group. The test period will be from 1/1/2007 to 12/31/2011, a 5 year period that includes some quiet, and some very chaotic, markets.

Finally, I will assume $5 round turn commissions, and $30 round turn slippage. The $30 slippage might be excessive, but I’ve always found it is better to be conservative than to underestimate slippage. I’d rather not be disappointed in real time with larger than expected slippage.

The System

I will use an extremely simple strategy for the tests I am running: a simple breakout based on closing prices. The system is always long or short, and will enter at the next bar open, long if the close is the highest close of the past X bars, and short if the close if the lowest close of the last X bars. One version of the system always exits at the end of the day, and the other version is a swing system. X will be varied from 5 bars to 100 bars.

Here is the code for the swing version, Strategy A:

Figure 1

Here is the code for the day trading version, Strategy B:

Figure 2

Day, or Night?

Since most markets these days run nearly 24 hours, we can run into issues when testing historically, as many markets traded “pit” hours years ago. Because of that, and because most of the volume is during these traditional pit hours, I will use the old pit session times, but still use electronic trading data.

Exit 1 Minute Before Close

If you use Tradestation to test, you may be familiar with their keyword “setexitonclose.” This is a neat function for backtesting, but in live trading the order is sent after the market is closed, rendering it ineffective. So, I set up the custom sessions above to exit 1 minute before the old “pit” closing time. Then, I know my strategy will exit properly, and will therefore work in backtest and in real time.

Final Pre-Test Info

Now that we have everything properly setup, it is time to run some tests. For each instrument given above, I will run tests with 5, 15, 30, 60, 120 and Daily bars. That gives us 7 instruments x 6 bar sizes = 42 tests. Then, for each test, I will run the breakout length X from 5-100 in steps of 1, which yields 96 iterations per test. Since I am running 2 strategies overall, I am running 42 x 96 x 2 = 8,064 unique performance sets.

For each of the 42 tests for each strategy, I will record the best Net Profit (out of the 96 iterations) and its corresponding maximum drawdown (if best Net Profit is greater than zero). For each of the 42 tests, I will also record the percentage of iterations that are profitable.

All the tests being run are shown graphically in Figure 3.

Figure 3

Questions To Answer

Once all the tests are run, I will see if I can answer the three questions below. These questions are directly related to the desires of many discretionary day traders.

1. Many traders love strategies that are the same across multiple instruments. An example of this is price action trading – many feel the principles of price action hold across all instruments. So, when these traders test algorithms, one demand is that a strategy is viable ONLY if it is profitable across many

instruments. Question: Is multiple market profitability a reasonable requirement?

2. Most day traders need to be out by the end of the day. Question: Does forcing an exit at the end of each day improve or decrease strategy performance?

3. Many day traders try to go for the smallest (shortest) bar size possible. The theory is that trades will be more frequent, losses will be smaller, and exits will be more responsive to market conditions. Question: What kind of impact does bar size have on performance?

Before I get to the results, I should point out that this is one study, with two strategies, across only seven markets. So, the conclusions I reach may not hold up over all markets, and may be different for different strategies. I’m guessing, though, that the conclusions probably do hold in general.

Results

Results of all 8,046 performance tests are shown in Table 1 for the swing Strategy A, and Table 2 for the intraday Strategy B. I will refer to these results as I discuss each of the three questions.

Table 1

 

Table 2

1. Is profitably across multiple instruments a reasonable requirement?

Figures 4 and 5 show the best case Net Profit for both the swing and intraday version of the strategy. With one notable exception (Heating Oil intraday trading), what is good in one market is generally good in another market. Although the maximum profit varies wildly from market to market, all the swing markets are profitable, and all but one of the intraday markets are unprofitable. This should give you some confidence that the strategy is sound (or not). Of course, it doesn’t mean the strategy is tradable in each market, since I am looking only at maximum net profit based on optimization. But, clearly the intraday Strategy B is not viable in most markets.

Figure 4

Figure 5

So, the results show that profitably across multiple markets is possible, even with markedly different instrument, suggesting that is indeed a valid requirement.

2. Is exiting end of day (intraday trading) a good idea?

Figure 5 shows the answer loud and clear – the answer is no. You’ll get more profit by swing trading. Unfortunately, it might also mean enduring more drawdowns, but think of it like this: the market “pays” people for holding overnight and weekends, and taking on that risk.

3. Are shorter timeframes better?

Since Strategy A is clearly the better strategy, I will use those results to look at the impact of bar length (timeframe). This is shown in Figure 6, where I look at the percent of iterations that were profitable for each combination of instrument and bar size. So, for example, Wheat with 5 minute bars shows that 62% of iterations were profitable (denoted with a red circle in Figure 6). This means that as I varied the breakout length from 5 to 100 in increments of 1 (96 total iterations), 60 of those runs resulted in a profitable end result. Obviously, the best case is 100% – where no matter what value you use for breakout length, the end result is positive.

Figure 6

Results of Figure 6 show that in general profitability increases as the bar length increases, although this effect is not very pronounced, and it does not hold for all

instruments. This is confirmed by the results of Figure 4, where the maximum Net Profit generally increase with increasing bar period, but not dramatically.

Why is this so? Profitability could be better as bar period increases, since random market noise plays a smaller role in longer period bars (i.e., the true trend is more easily seen in larger period bars). Of course, drawdown should also be considered when looking at bar size, but in this study it does not seem to change greatly with bar size.

Conclusion

Results with this simple strategy lead to three conclusions:

1. Striving for profit in multiple markets, as a confirmation of a strategy, is indeed possible.

2. Swing trading is likely more profitable than intraday trading.3. Longer timeframes are generally superior to short time periods, but this is not

a major effect.

Follow On

Of course, I made these conclusions based on one study. What if the strategy was different? What if the timeframes or markets were different? What if different years were used for the test period? Will the conclusions reached here still hold? I’ll examine those questions in Part 2.

If you would like to learn more about building trading systems be sure to get a copy of my latest book, Building Winning Algorithmic Trading Systems.

Download

Breakout Strategies (TradeStation ELD)Breakout Strategies WorkSpace (TradeStation TWS)

— Kevin J. Davey of KJ Trading Systems

Finding Out What Works, And What Doesn’t WorkSeptember 29, 2014 5:00 am11 commentsViews: 2487

Many traders who try system trading have previously had difficulty at discretionary or manual trading. Most of these folks eventually recognize the benefit of trading a system with well defined rules – a system that has performed well in the past. It is nice to know a trading approach has historically worked, but as with all things related to trading, past performance is no guarantee of future results.

Unfortunately, many people who try systematic/algorithmic/mechanical/rule-based trading for the first time bring along a lot of the baggage that they have acquired from their previous method. Depending on the pre-conceived notions they bring into mechanical trading, these new systematic traders may run into a lot of frustration and trouble.

Many times, for example, traders will always test with a few core concepts, such as always closing by the end of the day. This is what they were used to as a discretionary or manual trader, and therefore they never even think to test ideas out of their old comfort zone. Perhaps removing these “comfort” rules would dramatically improve performance.

In this article, I will examine three common items that new systematic traders test, and see how these items actually work when they are subjected to rigorous testing.

Ground Rules

In all the testing that follows, I will test in 7 different markets, across a range of commodities:

Wheat (W) 10 Year Treasury Notes (TY) Lean Hogs (LH) Australian Dollar (AD) Heating Oil (HO) Cotton (CT) e-mini Nasdaq (NQ)

I picked these at random, one from each major market group. The test period will be from 1/1/2007 to 12/31/2011, a 5 year period that includes some quiet, and some very chaotic, markets.

Finally, I will assume $5 round turn commissions, and $30 round turn slippage. The $30 slippage might be excessive, but I’ve always found it is better to be conservative than to underestimate slippage. I’d rather not be disappointed in real time with larger than expected slippage.

The System

I will use an extremely simple strategy for the tests I am running: a simple breakout based on closing prices. The system is always long or short, and will enter at the next bar open, long if the close is the highest close of the past X bars, and short if the close if the lowest close of the last X bars. One version of the system always exits at the end of the day, and the other version is a swing system. X will be varied from 5 bars to 100 bars.

Here is the code for the swing version, Strategy A:

Figure 1

Here is the code for the day trading version, Strategy B:

Figure 2

Day, or Night?

Since most markets these days run nearly 24 hours, we can run into issues when testing historically, as many markets traded “pit” hours years ago. Because of that, and because most of the volume is during these traditional pit hours, I will use the old pit session times, but still use electronic trading data.

Exit 1 Minute Before Close

If you use Tradestation to test, you may be familiar with their keyword “setexitonclose.” This is a neat function for backtesting, but in live trading the order is sent after the market is closed, rendering it ineffective. So, I set up the custom sessions above to exit 1 minute before the old “pit” closing time. Then, I know my strategy will exit properly, and will therefore work in backtest and in real time.

Final Pre-Test Info

Now that we have everything properly setup, it is time to run some tests. For each instrument given above, I will run tests with 5, 15, 30, 60, 120 and Daily bars. That

gives us 7 instruments x 6 bar sizes = 42 tests. Then, for each test, I will run the breakout length X from 5-100 in steps of 1, which yields 96 iterations per test. Since I am running 2 strategies overall, I am running 42 x 96 x 2 = 8,064 unique performance sets.

For each of the 42 tests for each strategy, I will record the best Net Profit (out of the 96 iterations) and its corresponding maximum drawdown (if best Net Profit is greater than zero). For each of the 42 tests, I will also record the percentage of iterations that are profitable.

All the tests being run are shown graphically in Figure 3.

Figure 3

Questions To Answer

Once all the tests are run, I will see if I can answer the three questions below. These questions are directly related to the desires of many discretionary day traders.

1. Many traders love strategies that are the same across multiple instruments. An example of this is price action trading – many feel the principles of price action hold across all instruments. So, when these traders test algorithms, one demand is that a strategy is viable ONLY if it is profitable across many instruments. Question: Is multiple market profitability a reasonable requirement?

2. Most day traders need to be out by the end of the day. Question: Does forcing an exit at the end of each day improve or decrease strategy performance?

3. Many day traders try to go for the smallest (shortest) bar size possible. The theory is that trades will be more frequent, losses will be smaller, and exits will be more responsive to market conditions. Question: What kind of impact does bar size have on performance?

Before I get to the results, I should point out that this is one study, with two strategies, across only seven markets. So, the conclusions I reach may not hold up over all markets, and may be different for different strategies. I’m guessing, though, that the conclusions probably do hold in general.

Results

Results of all 8,046 performance tests are shown in Table 1 for the swing Strategy A, and Table 2 for the intraday Strategy B. I will refer to these results as I discuss each of the three questions.

Table 1

 

Table 2

1. Is profitably across multiple instruments a reasonable requirement?

Figures 4 and 5 show the best case Net Profit for both the swing and intraday version of the strategy. With one notable exception (Heating Oil intraday trading), what is good in one market is generally good in another market. Although the maximum profit varies wildly from market to market, all the swing markets are profitable, and all but one of the intraday markets are unprofitable. This should give you some confidence that the strategy is sound (or not). Of course, it doesn’t mean the strategy is tradable in each market, since I am looking only at maximum net profit based on optimization. But, clearly the intraday Strategy B is not viable in most markets.

Figure 4

Figure 5

So, the results show that profitably across multiple markets is possible, even with markedly different instrument, suggesting that is indeed a valid requirement.

2. Is exiting end of day (intraday trading) a good idea?

Figure 5 shows the answer loud and clear – the answer is no. You’ll get more profit by swing trading. Unfortunately, it might also mean enduring more drawdowns, but think of it like this: the market “pays” people for holding overnight and weekends, and taking on that risk.

3. Are shorter timeframes better?

Since Strategy A is clearly the better strategy, I will use those results to look at the impact of bar length (timeframe). This is shown in Figure 6, where I look at the percent of iterations that were profitable for each combination of instrument and bar size. So, for example, Wheat with 5 minute bars shows that 62% of iterations were profitable (denoted with a red circle in Figure 6). This means that as I varied the breakout length from 5 to 100 in increments of 1 (96 total iterations), 60 of those runs resulted in a profitable end result. Obviously, the best case is 100% – where no matter what value you use for breakout length, the end result is positive.

Figure 6

Results of Figure 6 show that in general profitability increases as the bar length increases, although this effect is not very pronounced, and it does not hold for all

instruments. This is confirmed by the results of Figure 4, where the maximum Net Profit generally increase with increasing bar period, but not dramatically.

Why is this so? Profitability could be better as bar period increases, since random market noise plays a smaller role in longer period bars (i.e., the true trend is more easily seen in larger period bars). Of course, drawdown should also be considered when looking at bar size, but in this study it does not seem to change greatly with bar size.

Conclusion

Results with this simple strategy lead to three conclusions:

1. Striving for profit in multiple markets, as a confirmation of a strategy, is indeed possible.

2. Swing trading is likely more profitable than intraday trading.3. Longer timeframes are generally superior to short time periods, but this is not

a major effect.

Follow On

Of course, I made these conclusions based on one study. What if the strategy was different? What if the timeframes or markets were different? What if different years were used for the test period? Will the conclusions reached here still hold? I’ll examine those questions in Part 2.

If you would like to learn more about building trading systems be sure to get a copy of my latest book, Building Winning Algorithmic Trading Systems.

Download

Breakout Strategies (TradeStation ELD)Breakout Strategies WorkSpace (TradeStation TWS)

— Kevin J. Davey of KJ Trading Systems

Finding Out What Works, And What Doesn’t Work – Part IIOctober 27, 2014 5:00 am6 commentsViews: 1618

In Part 1 of this series, found here, I ran a simple study of a trading system, in order to look at some of the common components (such as end of day closes, short versus long bar periods) that many system developers add to their trading system. Could some of these strategy building blocks be hurting their development efforts? To answer the question, I analyzed one trading system over 7 different markets for a 5 year period. Even though I was able to extract some conclusions from the study, I also said the following:

“Of course, I made these conclusions based on one study. What if the strategy was different? What if the timeframes or markets were different? What if different years were used for the test period? Will the conclusions reached here still hold? I’ll examine those questions in Part 2.”

To summarize the earlier work, the conclusions from Part 1 were:

1. Striving for profit in multiple markets, as a confirmation of a strategy, is indeed possible.

2. Swing trading is likely more profitable than intraday trading.3. Longer timeframes are generally superior to short time periods, but this is not

a major effect.

In this second part of the study, I’ll now look at a different group of markets, a completely different strategy and some different timeframes. I’ll also try to answer the following questions: Will the conclusions still hold up? What type of insights, if any, do conducting studies like these actually provide? Finally, can studies like these improve your own strategy development?

Ground Rules

Previously in Part 1, I tested 7 different markets across a range of commodities.

(Group 1):

Wheat (W) 10 Year Treasury Notes (TY) Lean Hogs (LH) Australian Dollar (AD) Heating Oil (HO) Cotton (CT) the e-mini Nasdaq (NQ)

I picked these at random, one from each major market group. The test period will be from 1/1/2007 to 12/31/2011, a 5 year period that includes some quiet, and some very chaotic, markets.

For Part 2, I will test another basket of commodities.

(Group 2):

Live Cattle (LC) Canadian Dollar (CD) Copper (HG) Natural Gas (NG) Sugar (SB)

I will also test the period 1/1/2012 to present, a smaller test period outside of the range previously tested.

Finally, I will assume $5 round turn commissions and $30 round turn slippage.

The Systems

I will use two extremely simple base strategies for the tests I am running: a simple breakout based on closing prices (Strategies A and B), or a reverse breakout (countertrend – Strategies C and D). In all cases, the breakout length X will be varied from 5 bars to 100 bars.

Here is the code for all four versions of the strategy:

Results – Repeat of Part 1 Test, Different Instruments

First, I will conduct a test of Strategies A and B for the same period, with the second group of commodities. To keep things manageable, only the group summary results are shown below:

The results of Group 2 are generally in line with Group 1 results, which show that swing trading (strategy A) is better than intraday trading (Strategy B), and that longer timeframes are slightly better than short timeframe bars.

Results – Different Timeframes, Different Time Period, Different Strategy

Of course, concluding that long timeframe bars and swing trading is superior based on only a few tests is dangerous. So, let’s look at Strategies C & D (the opposite Strategies of A & B), and see if the results have the same tendencies for different timeframes and test period, on a random subset of Group 1 and Group 2 instruments.

For timeframes, the results show a tendency for the smaller timeframes to be better than the longer timeframes, exactly the opposite of what was found earlier. But this effect is small, which could mean it is not actually true, and could be just an artifact of the limited testing. This is a good example why it is dangerous to draw conclusions from only a few tests.

For the swing vs. intraday testing, the results are pretty clear – intraday testing is not nearly as good as swing trading. This agrees with the earlier conclusion.

Conclusion

Through both Part 1 and Part 2 of this study, a lot of tests have been run over various bar timeframes, instruments, strategies and time periods. Yet, it would be dangerous to make any sweeping conclusions based on these tests. Although these results show that intraday trading is less profitable than swing trading, there are surely cases where intraday trading is preferable over swing trading.

So, what can this study actually teach us? I think the following lessons come out of this study:

1. Intraday trading is less profitable than swing trading. This does not mean this always holds, but if you do most of your testing intraday and you are struggling to develop a strategy, perhaps opening up the strategy to allow swing trading is a good idea. It could be that you have a good swing strategy, but the end of day exit alone makes the strategy look bad.

2. Longer timeframes are better (Part 1 conclusion), except when they aren’t (Part 2 conclusion). So, be careful of making conclusions when the effect is small, or results are contradictory. Since many strategy developers tend to find something that does or does not work and then focus only on that (or avoid it), the lesson here is that the developer should not “box” themselves in. Maybe short timeframes rarely work with your strategies, but that does not mean you should never test them. There will be other strategies where shorter time bars are an asset.

3. Since a developer can never test everything, it may be worthwhile to vary your test approach. By looking at different timeframes, instruments, etc., you may discover new paths to take in your strategy development.

The key to successful strategy testing and development is to keep an open mind, and allow the data to point you in the right direction. At the same time, though, don’t let the data trick you into making false conclusions or sweeping generalizations. Remaining flexible in your development approach can lead to new and profitable strategies – ones that you might now be dismissing based solely on past testing.

Download

Breakout Strategies (TradeStation ELD)Breakout Strategies WorkSpace (TradeStation TWS)

— Kevin J. Davey of KJ Trading Systems

Probability Tools For Better Forex TradingSeptember 1, 2014 5:00 am6 commentsViews: 863

In order to be successful, forex traders need to know the basic mathematics of probability. After all, it’s difficult to achieve and maintain trading gains without first having the ability to understand the numbers and measure them.

Many traders use a combination of black box indicators to develop and implement trading rules. Yet, the difference between a “good” trader and a great one is his or her understanding of the metrics and methods for calculating performance and gains.

Probability and statistics are the key to developing, testing and profiting from forex trading. By knowing a few probability tools, it’s easier for traders to set trading goals

in mathematical terms, create and operate effective trading strategies, and assess results.

It’s helpful to review the most basic concepts of probability and statistics for forex trading. By understanding the math of probability, you’ll know the logic used by mechanical trading systems and expert advisors (EA).

Normal Distribution

The most basic tool of probability in forex trading is the concept of normal distribution. Most natural processes are said to be “normally distributed.”

“Uniform distribution” implies that the probability of a number being anywhere on a continuum is about equal. This is the sort of distribution that would result from artificially spreading objects as evenly as possible across an area, with a uniform amount of spacing between them.

However, instead of a uniform distribution, a currency-pair’s price will likely be found within a certain area at any given time. This is its “normal distribution,” and probability tools can show an approximation of where that price is likely to be found.

Normal distribution offers forex traders predictive power regarding the likelihood that a currency-pair price will reach a certain level during a certain time frame.

Computers use a random-number generator to calculate the means (averages) of forex prices in order to determine their normal distribution.

If a large number of sample prices are checked, the normal distribution will form the shape of a bell curve when plotted graphically. The greater the number of samples, the smoother the curve will be.

The rules of simple averages are helpful to traders, yet the rules of normal distribution offer more useful predictive power. For example, a trader may calculate that the “average” daily price move of a forex pair is, say, 50 pips.

Yet, the normal distribution can also tell the trader the likelihood that a certain daily price move will fall between 30 and 50 pips, or between 50 and 70 pips.

According to the rules of normal distribution and standard deviation, approximately 68% of the samples will be found within one standard deviation of the mean (average), and about 95% will be found within two standard deviations of the mean. Finally, there is a 99.7% likelihood that the sample will fall within three standard deviations of the mean.

Normal distribution and standard deviation functions in expert advisors (EA) and trading systems help forex traders assess the probability that prices may move a certain amount during a given period of time.

Yet, traders should be cautious when using the concept of normal distribution alone for purposes of risk management. Even though the probability of a rare event (such as a price decrease of 50%) may seem low, unforeseen marketplace factors can make the possibility much higher than it appears during normal distribution calculations.

Reliability of Analysis Depends On Quantity and Quality of Data

When modelling normal distribution curves, the amount and quality of input price data is very important. The greater the number of samples, the smoother the curve will be. Also, to avoid calculation errors resulting from insufficient data, it’s important that each calculation be based on at least thirty samples.

So, for testing a forex-trading strategy by estimating the results from sample trades, the system developer must analyze at least 30 trades in order to reach statistically-reliable conclusions regarding the parameters being tested. Likewise, the results from a study of 500 trades are more reliable than those from an analysis of only 50 trades.

Dispersion and Mathematical Expectation to Estimate Risk

For forex traders, the most important characteristics of a distribution are its mathematical expectation and dispersion. Mathematical expectation for a series of trades is easy to calculate: Just add up all the trade results and divide that amount by the number of trades.

If the trading system is profitable, then the mathematical expectation is positive. If the mathematical expectation is negative, the system is losing on average.

The relative steepness or flatness of the distribution curve is shown by measuring the spread or dispersion of price values within the area of mathematical expectation. Typically, the mathematical expectation for any randomly-distributed value is described as M(X).

So, dispersion can be defined as D(X) = M[(X-M(X)]2.

And, a dispersion’s square root is called its standard deviation, shown in mathematical shorthand as sigma (σ).

Dispersion and standard deviation are critically important for risk management in forex trading systems. The higher the value of the standard deviation, the higher will be the potential drawdown, and the higher the risk. Likewise, the lower the value for standard deviation, the lower will be the drawdown while trading the system.

For example, below is a sample risk assessment for a test of a forex trading system:

Trade Number X (Trade Gain or Loss)

1. -17.082. -41.003. 47.804. -159.975. 216.976. 98.307. -87.758. -27.839. 12.3410. 48.1411. -60.92

12. 10.6213. -125.4314. -27.8215. 88.0216. 32.9417. 54.8218. -160.1019. -83.3720. 118.4021. 145.6522. 48.4323. 77.3924. 57.4925. 67.7526. -127.1027. -70.1928. -127.6029. 31.3130. -12.55

In the above example based on the minimum number of thirty trades for an adequate sample, it’s important to note that the mathematical expectation is positive, so the forex trading strategy is indeed profitable.

However, the standard deviation is high, so in order to earn each dollar the trader is risking a much larger amount; this system carries significant risk.

Here’s the rest of the math: To determine the mathematical expectation for this group of trades, add together all the trades’ gains and losses, then divide by 30. This is the mean value M(X) for all the trades. In this case, it equals an average gain of $4.26 per trade. Thus far, the system looks promising.

Next, to calculate the standard deviation of the dispersion, the above average $4.26 is subtracted from the results of each trade, then it’s squared, and the sum of all these squares is added together. The sum is divided by 29, which is the total number of trades minus 1.

By using the formula for Dispersion of (X) = M[(X-M(X)]2 given above, here’s a check of the calculation from the first trade in our example:

Trade 1: -17.08 – 4.26 = -21.34, and (-21.34)2 = 455.39

The same calculation is performed for each trade in the test series. In this example, the dispersion over the series equals 9,353.62 and by definition its square root equals the standard deviation (σ), which in this case is $96.71.

Thus, the forex trader sees that the risk for this particular system is fairly high: The mathematical expectation is indeed positive, with a mean profit of $4.26 per trade, yet the standard deviation is high when compared with that profit.

It can be seen that the trader is risking about $96.71 for each opportunity to earn $4.26 in profit. This risk may be acceptable, or the trader may choose to modify the system in search of lower risk.

Z-score

Beyond the riskiness of a particular trading system, forex traders can also use normal distribution and standard deviation to calculate the Z-score, which indicates how often profitable trades will occur in relation to losing trades.

During the process of developing a winning forex trading system, the trader may wonder how many of the profitable trades seen during testing were “random,” and how many consecutive losing trades must be tolerated in order to achieve winning trades.

For example, let’s assume the average expected profit from a given forex trading system is four times less than the expected loss amount from each stop-loss order triggered while trading this system.

Some traders may assume that the system will win over time, as long as there is an average of at least one profitable trade for each four losing trades. Yet, depending upon the distribution of wins and losses, during real-world trading this system may draw down too deeply to recover in time for the next winner.

Normal distribution can be used to generate a Z-score, sometimes called a standard score, which lets traders estimate not only the ratio of wins to losses, but also how many wins/losses are likely to occur consecutively.

A positive Z-score represents a value above the mean, and a negative Z-score represents a value below the mean. To obtain this value, the trader subtracts the population mean from an individual raw value then divides the difference by the population standard deviation.

The basic standard score calculation for a raw score designated as x is:

Z = (x – μ) / σ

Where μ is the population mean and σ is the population standard deviation. It’s important to understand that calculating the Z score requires that the trader know the parameters of the population, not merely the characteristics of a sample taken from that population.

Z represents the distance between the population mean and the raw score, expressed in units of the standard deviation. So, for a forex trading system:

Z = [N x (R – 0.5) – P] / [(P x (P – N)] / (N – 1)]½

When:

N is the total number of trades during a series;

R is the total number of series of winning and losing trades; P equals 2 x W x L W is the total number of winning trades during a series L is the total number of losing trades during a series

Individual series can be represented by a consecutive sequence of pluses or minuses (for example ++++ or —). R counts the number of such series.

Z can offer an assessment of whether a forex trading system is operating on-target, or how far off-target it may be.

Just as importantly, a trader can use Z-score to determine whether a trading system contains fewer or greater series of winners and losers than expected from a random sequence of trades– In other words, whether the outcomes of consecutive trades are dependent upon each other.

If the Z-score is near 0, then the distribution of trade results is near the normal distribution. The score of a sequence of trades may indicate a dependency between the results of those trades.

This is because a normal random value will deviate from the average value by not more than three sigma (3 x σ) with a certainty of 99.7%. Whether the Z value is positive or negative will inform the trader about the type of dependence: A positive Z value indicates that the profitable trade will be followed by a loser.

And, positive Z indicates that the profitable trade will be followed by another profitable one, and a loser will be followed by another loss. This observed dependency lets the forex trader vary the position sizes for individual trades in order to help manage risk.

Sharpe Ratio

The Sharpe Ratio, or reward-to-variability ratio, is one of the most valuable probability tools for forex traders. As with the methods described above, it relies on applying the concepts of normal distribution and standard deviation. It gives traders a method to check the performance of a trading system by adjusting for risk.

The first step is to calculate the Holding Period Returns (HPR). For example, a trade which resulted in a profit of 10% has a HPR calculated as 1 + 0.10 = 1.10 while a trade which loses 10% is calculated as 1 – 0.10 = 0.90.

Likewise, HPR can be calculated by dividing the after-trade balance amount by the before-trade amount. The Average Holding Period Returns (AHPR) is then calculated by adding up all individual holding-period returns, then dividing by the number of trades.

AHPR by itself produces an arithmetic average which may not properly estimate the performance of a forex trading system over time. Instead, a trading system’s investment efficiency can be more closely estimated by using the Sharpe Ratio, which shows how AHPR minus the risk-free rate of long-term investment returns relates to the standard deviation of the trading system.

Sharpe Ratio = [AHPR – (1 + RFR)] / SD

When AHPR is the average holding period return, RFR is the risk-free rate of return from “safe” investments such as bank interest rates or long-term T-bond rates, and SD is the standard deviation.

Since more than 99% of all random values will fall within a distance of ±3σ around the mean value of M(X) for a given trading system, the higher the Sharpe Ratio, the more efficient the trading system.

For example, if the Sharpe Ratio for normally-distributed trade results is 3, it indicates that the probability of losing is less than 1% per trade, according to the 3-sigma rule.

The concepts of normal distribution, dispersion, Z-score and Sharpe Ratio are already incorporated into the logarithms of EAs and mechanical trading systems, and their usefulness is invisible to most traders.

Yet, by knowing how these basic probability tools work, forex traders can have a deeper understanding of how automated systems perform their functions, and thereby enhance the probability of winning trades.

Are you currently using probability tools to increase your own chance for success?

Trading Ideas And Systems: Keep It Simple, Smart GuyAugust 18, 2014 5:00 am6 commentsViews: 2071

One thing that I have discussed in numerous NAS Trading articles is the tendency for traders to believe that complex ideas and systems are inherently better or more desirable than simple, conceptual ideas or systems. Just taken at face value, this notion is completely understandable. After all, if an idea is fairly simple, how could it possibly be “secret” enough to add value to a trading program?

To switch things up, let’s take an example from the macro trading space. The move in the Japanese yen from late 2012 through the spring of 2013 was a windfall for some of the macro trading firms. Yet, to read many pundits, blogs, etc., this trade was “not good” because it was too obvious. The tone was almost that traders didn’t deserve large profits, because the trade was, “obvious to everyone.”  Yet, did these pundits themselves make money? Of course not. Partly as a consequence, they miss the entire point: It is not about how you look or how impressive you sound, it is about what you do and when you do it. In trading, that is all that matters. Saying something was “obvious” but not acting on it is worthless. At first such punditry hardly seems worth commenting on, yet to me it is significant. It provides a textbook example of why some highly intelligent people struggle with the markets. They believe trading is an intelligence test or even an ego contest. The idea that a simple idea could be lucrative if acted upon promptly and with discipline shatters their egocentric notion of how the markets should work.

Here in Chicago there is a tremendously successful doughnut shop called “The Doughnut Vault”. It sells a few varieties of doughnuts and coffee out of a very small space. On most days there is a line that stretches around the corner, and this has been the case since it opened a few years ago. The owner of this business owns several very successful full service restaurants which are far more complicated in concept and implementation. Yet, in an interview he stated that it was the doughnut shop that has

created his biggest financial success.

Trading is commerce. Our “regular customers” are not people lining up and waiting 45 minutes for expensive (and very good) doughnuts. The other side of our trade is most often other market participants who have immediate, urgent liquidity needs, such as those entering on a greedy impulse, or exiting due to fear of further loss (or even better, a margin call). In other cases, the other side of a trade might be a government looking to achieve a policy objective, or hedgers looking to lock in profits at their operating businesses. I wrote about this concept more extensively in this article (which is one of the first I wrote for this site… I think my style has improved since then). On further reflection, here are three simple questions that I think are useful:

Do you have liquidity when your counterparty does not? Do you have an understanding that your counterparty does not have? Does your counterparty have other (non-profit minded) objectives?

To get a real trading edge, I think you need at least one of these things to be true.

One idea that is reinforced over and over again in the classic strategy book, “The Book of Five Rings”, is that in a real martial situation, simple direct movements dominate complex, refined movements. Humorously (and I think particularly relevant to investors and traders) the author also states that complex, “sophisticated” or flashy moves mostly exist to sell new students on a particular school. Replace “students” with “new traders” or “investors” and the above should sound highly familiar. Here is a very simple trading idea (one of the simplest swing trading ideas I can think of) that is useful, yet with substantial room for improvement and development:

If today’s close is less than the close 6 days ago, buy (enter long) If today’s close is greater than the close 6 days ago, sell (exit long)

Here are the results in the S&P 500 futures, 1998 to present (1 contract per $50k account equity):

Here are the results using a look-back parameter of between 1 and 10 days with the above system:

In fairness “simple” is not really that simple. The right kind of simplicity, in the right amount at the right time, is what creates the genius of a great artwork, a great business or great trading strategy. If you liked this article, you might like some of my other trading strategy articles (click here). You can also join my mailing list here and find me on twitter here: @NASTrading.

– By Nat Stewart at NAS Trading. Original article appeared here.

Strategy Testing – Violating These Steps Will Damage Your AccountDecember 30, 2013 5:00 am0 commentsViews: 2248

One of the most rewarding experiences for a TradeStation trader is to pick up a performance report that proves their great strategy idea is indeed a profitable strategy. Strategy testing done properly, as is outlined in this article, can verify the efficacy of your trading strategy and give you confidence to start trading it. But be forewarned, strategy testing done improperly can lead you toward financial destruction.

Strategy testing done incorrectly can result in false hope in a losing strategy. A trader recently shared his experience of getting great results from strategy testing his idea, but after trading it live in the market, he was losing money every day. He was baffled about what he did wrong. Having gotten excellent results on his back-testing performance report, he wondered why his promising strategy was draining his trading account. The problem was he violated several of the proper steps necessary for reliable strategy testing.

With the knowledge of how to get an accurate performance report you will be able to trust your strategy in live trading and protect your trading account. In order to properly test a strategy, there are 5 main steps that are vital to follow; configure TradeStation, “in-sample data” testing, “out-of-sample data” testing, live forward testing on the simulator account, and real live trading execution.

Step 1: Configure TradeStation

Before you begin testing your data, you must configure TradeStation so that the data it pulls onto your performance report will be accurate. Follow these 3 critical items to properly configure TradeStation.

(a) In your TradeStation platform menu, go to “format symbol” and give TradeStation a starting and ending date to test. This historical date range is called the “in-sample data.” Do not include the most recent six months in this “in-sample data.” The most recent six months is called the “out-of-sample data,” and it will be used later during your “out-of-sample data” testing step.

(b) Next, in your TradeStation platform menu, go to “format strategy” and select “properties for all.” Now select the “general” tab and enter the commissions and slippage (be as realistic as possible, or estimate too high if you are not sure). If this step is skipped, then the strategy testing performance report will be meaningless. If this is not done you might have a good looking performance report equity curve, but as soon as you enter the commissions and slippage figures the equity curve can reverse into an underwater equity curve.

(c) The last configuration step is under “properties for all” under the “general” tab. Look in the bottom left section called “strategy testing resolution.” Check the “look-inside-bar back-testing” option and then select the smallest time frame available for your chart style to make the strategy testing more closely resemble live data. When strategy testing, TradeStation uses the open, high, low, and closing data, thus the larger the time frame bar, the more distorted the strategy testing performance report can be. This “look-inside bar back-testing” option will make the computer do a lot more strategy testing calculations. This may really slow down your performance report generation, so please be patient. For an accurate performance report you must use the “look-inside bar back-testing” option.

These configuration steps are critical to getting an accurate performance report, so be sure this is completed precisely before continuing. Once TradeStation has been configured correctly, you can begin testing your strategy.

Step 2: In-Sample Data Testing (also called “Back testing”)

You are now ready to start testing your strategy idea. We will begin with testing the “in-sample data” that you set up for testing during the configuration steps. Begin with bringing up a TradeStation performance report. Right now I have a performance report in front of me that I will refer to, but you will be looking at your own performance report to analyze your own numbers. This is what we will be referring to in the steps below. There are 7 sub-steps to “in-sample data” testing, as follows:

First, look at how many trades the strategy made. To reduce strategy testing errors, where error is defined by [ error = 1 / Square Root (Number Trades In Test) ], you want at least 400 trades to reduce the margin for error to 5% in your strategy testing results. At 100 trades you have a 10% margin for error. The greater the number of inputs in your strategy that you optimize, the greater the number of trades you need to keep from over optimizing your strategy.

Also look at how many times the strategy traded on average per day. The more often a strategy trades the more profit it can generate.

In the performance report that I am looking at, it traded 397 trades in the last 3 1/2 months, averaging 5.3 trades per day.

Second, look at the “Average Trade Amount.” It needs to be large enough that slow order fills and/or larger than normal slippage does not kill the profitability of the strategy.

In my report the “Average Trade Amount” is $162.32. The commissions and slippage amount as defined in the set up steps is already subtracted in this performance report.

65% of the time this strategy trades 1 contract.35% of the time this strategy trades 3 contracts.10% of the time this strategy trades 5 contracts.

Third, look to see if the “Profit Factor” and “Ratio Average Win-Average Loss” are both above 1.5 and the percentage of winning trades around 45% or better

This strategy had a “Profit Factor” of 1.83.This strategy had a “Ratio Average Win-Average Loss” of 2.28 (2.28 means breakeven is around 28% “Percent Winning Trades”)On this strategy the “Percent Winning Trades” was 44.58%.

Fourth, look at the trade list page and assess the profit run ups and draw downs column. Notice how many trades made money and how much money they made before the trade exit occurred. Looking at what amount of money was made in relation to the profit run up and draw down, you want to know if managing the trades could generate more profits. The example used here shows that a good percentage of trades made much higher profits than where the automated exit points occurred.

Fifth, look at the three draw down (DD) numbers. I like to see the largest number at 15% or less of the “Total Net Profit” and the “Max DD” at 5% or less of the “Total Net Profit” (these numbers tell about the draw down risk level during your trades).

Total Profit – $64,440Peak to Valley DD – $8,960 is 13% of Total ProfitClose to Close DD – $7,120 is 11% of Total ProfitMax DD – $3,420 – 5% of Total Profit

Sixth, Looking at the “Largest Losing Trade” on the report, I like to see 5% or less of the “Total Net Profit.” In my report the “Largest Losing Trade” that occurred was $2,580 which is 4% of “Total Net Profit.”

Seventh, I review the length of time in the average trade. Does the average time in a trade comply with the golden rule of trading; “cut your losses quickly and let your profits run?” You will also want to see if the strategy is built using only profit exits (no real stop loss exits). It might have a nice looking report, but it could show a messed up ratio between average bars per winning trade verses average bars per losing trade if there are no stop loss exits. Here are my average bars:

Average bars per winning trade 7.24Average bars per losing trade 3.51 bars

This strategy complies with the golden rule of trading. Notice how it cuts losses quickly, at an average of 3.51 bars, and lets the profits run for an average of 7.24 bars.

So what does this all mean? It means this strategy has passed the historical strategy testing phase of strategy testing.

Step 3: Out-of-Sample Data (also called “Walk Forward Testing”)

Once you have tested your “in-sample data” and have determined that your strategy is worthy of continued testing, you can now test your strategy against the “out-of-sample data.” If you have not yet tested your “in-sample data, do that before proceeding.

To test the “out-of-sample data” we use the most recent 6 months of data available that you reserved in step 1(a). In step 1(b) of this article, we talked about configuring TradeStation and covered entering commissions and slippage and using the “look-inside-bar back-testing” option, which must be used to run any performance report used in your strategy testing. Be sure you have configured TradeStation correctly before moving on.

Go into “format symbol” and change the date range to include ONLY the “out-of-sample data” date range that was NOT used during the strategy testing on “in-sample data.” This is referred to as testing on the “out-of-sample” data.

Begin with bringing up a TradeStation performance report on the “out-of-sample” data and review all the items that we discussed in step 2 above on this “out-of-sample” performance report. The closer it performs to the Step 2 “in-sample data” performance report, the more robust the strategy is. This suggests that the results were not from curve fitting and you have a good chance of having a viable strategy. This “out-of-sample” date range test is much more important than the strategy testing step on the “in-sample-data” for finding a successful strategy. It is a good idea to test multiple different “out-of-sample” date ranges, which is called “Walk Forward Analysis.”

Robustness: Perry J. Kaufman stated, “Practically speaking, a robust trading strategy is one that produces consistently good results across a broad set of parameter (input) values applied to many different markets tested for many years.”

If the strategy fails during this “out-of-sample data” test, do NOT optimize using your reserved “out-of-sample data.” This would defeat this vitally important step in strategy development. You can go back to your strategy and fix it, or else drop it and develop a new strategy idea.

One caveat – if your strategy is capitalizing on a certain market condition, like the current volatility, and then you “out-of-sample data” test a non-volatile date range, it may not perform well, However in our next phase of testing, “ Live Forward Testing,” it could prove to be successful since we are still in a volatile market. You must understand why your strategy works, under what market conditions it performs well, and in what market conditions it does not perform well.

Now that you have tested your “out-of-sample” data and your strategy is promising, you are ready to live forward test your strategy on the simulator account.

At this point you have configured TradeStation so your performance report will be accurate, you have tested your “in-sample data” and your “out-of-sample” data and your strategy still looks great. Now you are ready to live forward test your strategy on the simulator account.

Step 4: Live Forward Testing on the Simulator Account

During your live forward testing on the simulator, you want to verify that the live data feed entries and exits are similar to historical entries and exits. After you have made live data trades for a day, save the live trade list. Now reload this same chart so the strategy recalculates based on the historical for this same day. Record the historical trade list and compare the live entries and exits to the historical entries and exits. Are they the same or at least similar? Do you understand the differences and the impact your “Live Data” test says about your strategy?

Only by monitoring the program daily can performance be seen under real “live” market conditions. Continue Live Forward Testing on the simulator until you are totally comfortable that your strategy works on live data. Real time results will often be less profitable than your historic results. The key question is does the real time testing show that you have a profitable strategy that is worth trading?

Step 5: Real Live Trading Execution

Once you have done your due diligence and are comfortable with your strategy results on the simulator, you are ready to trade live. Since you are trading a brand new strategy with real money, start with a significantly reduced position sizing risk of 1/4 of 1% of your account equity at risk per your stop loss point per trade. Continue trading with minimal risk until you have verified everything is working properly within your new strategy during live order executions.

Once your strategy is making money in the live market, slowly over time begin to increase your position sizing risk. Move your risk upward from 1/4 of 1% toward 1%. Continue trading at 1% risk until you have several weeks to months of consistent trading performance. If you want to be aggressive and use more, you can continue to increase toward the maximum 3% of your account equity at risk per trade.

If you follow the 5 steps as outlined in this article, you will now be able to confidently strategy test any strategy idea that you have. Keep this article for reference so the next time you are inspired with a great idea, you will be able to prove it out, protect your TradeStation trading account, and have confidence in live trading your strategy.

Click on this link to learn how to quickly and accurately strategy test any trading idea you might have.

— by Mark David Johnson from http://www.customizedtrading.com. Mark David Johnson is a full time TradeStation Trader and TradeStation EasyLanguage Programmer for the past 6 years. He has been involved in programming for over 26 years and involved in trading for over 16 years.

What Is Walk Forward Optimization?December 9, 2013 5:00 am6 commentsViews: 1712

One of the biggest issues with system development is that many trading strategies do not hold up into the future. This could be due to several reasons:

The system is not based on a valid premise Market conditions have changed in a dramatic way that invalids the theoretical

premises on which the system was developed The system has not been developed and tested with a sound methodology. For

instance, (a) lack of robustness in a system due to improper parameters, and (b) inconsistent rules and improper testing of the system using out-of-sample and in-sample data

There are several approaches and methodologies to asses robustness and increase likelihood of positive real life trading performance including:

Peak avoidance Limited number of optimizable parameters 3D surface charting Monte Carlo analysis Data scrambling Walk forward & walk backward analysis

One of the most robust ways of testing the reliability of a trading system and making sure the program will have the highest likelihood of performing well in real live trading is to use walk forward optimization (WFO), a method first described in the book “Design, Testing, and Optimization of Trading Systems” by Roberto Pardo.

What is walk forward analysis?

Walk forward analysis is the process of optimizing a trading system using a limited set of parameters, and then testing the best optimized parameter set on out-of-sample data. This process is similar to how a trader would use an automated trading system in real live trading. The in-sample time window is shifted forward by the period covered by the out-of-sample test, and the process is repeated. At the end of the test, all of the recorded results are used to assess the trading strategy.

In other words, walk forward analysis does optimization on a training set; tests on a period after the set and then rolls it all forward and repeats the process. We have multiple out-of-sample periods and look at these results combined. Walk forward testing is a specific application of a technique known as Cross-validation. It means taking a segment of data to optimize a system, and another segment of data to validate. This gives a larger out-of-sample period and allows the system developer to see how stable the system is over time.

The picture below illustrates the walk forward analysis procedure. An optimization is performed over a longer period (the in-sample data), and then the optimized parameter set is tested over a subsequent shorter period (the out-of-sample data). The optimization and testing periods are shifted forward, and the process is repeated until a suitable sample size is achieved.

In order to demonstrate the concept we will perform in this article a walk forward optimization on a volatility breakout trading system (VBO). For the test we will use the German DAX futures, NinjaTrader, CQG historical 1-minute data, and we will assume 3 points of slippage for each R/T trade to cover trading frictions.

The process consists of three main steps:

1. Define in-sample and out-of-sample periods2. Define a robust parameters area3. Execute the walk forward

Definition of the in-sample and out-of-sample periods

We will choose as in-sample 1/1/2001 to 12/31/2009 for system design and in-sample optimization and 1/1/2010 to 12/31/2012 as out-of-sample period to evaluate the in-sample optimization robustness and execute the walk forward. We will then use a 3:1 ratio for the WFO (walk forward optimization):

Optimize 2007 to 2009 and verify performance out-of-sample in 2010 Optimize 2008 to 2010 and verify performance out-of-sample in 2011 Optimize 2009 to 2011 and verify performance out-of-sample in 2012

Define the robust parameters area in the in-sample period

In this section we will define the “robust area” of the system parameters. We will optimize only 3 system parameters:

Lookback period of the fast average Lookback period of the slow average Volatility filter

Other system parameters that we will not optimize are:

Start time: 09:00 (GMT+1) End time: 22:00 (GMT+1) Last trade: 18:00 (GMT+1) Stop risk: 2% Maximum trades per day: 3 Exit on close: True

Optimization of the charting resolution

As we can see the average SQN (system quality number) tends to decrease as we increase the time period of the chart in minutes. We will choose 14 minutes for all simulations going forward.

For more info on SQN:http://www.traderivar.com/2010/03/sqn-trading-system-quality-number.html

The NinjaTrader code for the system quality number can be downloaded here:http://www.vbosystems.info/download.html

 Definition of the robust parameter area for the averages

Via a 3D surface chart we can identify the robust parameter area as follows:

Slow average: 12 to 30 minutes Fast average: 330 to 500 minutes

We define “robust” a parameter surface area that does not have major peaks or valleys and has generally a good performance.

Definition of the robust parameter for the volatility filter

As we can see the robust area for the filter is between 0.55 and 0.70 when the SQN is slowly increasing.

Now that we have identified the robust parameters area it is worth performing a full in-sample optimization to see how the system would have performed between 2001 and 2009.

The system generated a net profit of $120,000 between 2001 and 2009 with a profit factor of 1.56, performing 756 trades with an average of 41% trades profitable. The systems exhibits certain desirable characteristics such as a high win/loss ratio of 2.28.

Walk forward optimization

As anticipated we will now proceed with a walk forward optimization.

Step 1: We’ll optimize between 2007-2009 and find the best parameters.

The best parameters for the 2007-09 period are:

Fast average: 12 Slow average: 410 Filter: 0.55

 

We apply these parameters to the out-of-sample period in 2010 with the following results:

 

Net profit: $12,300DD: $9,000% profitable: 43%Profit factor: 1.38

Step 2: We’ll optimize between 2008-2010 and find the best parameters.

The best parameters for the 2008-10 period are:

Fast average: 20 Slow average: 500 Filter: 0.70

We apply these parameters to the out-of-sample period in 2011 with the following results:

Net profit: $27,900DD: $7,450

% profitable: 43%Profit factor: 1.61

Step 3: We’ll optimize between 2009-2011 and find the best parameters.

 

The best parameters for the 2009-11 period are:

Fast average: 20 Slow average: 420 Filter: 0.55

We apply these parameters to the out-of-sample period in 2012 with the following results:

Net profit: $17,540DD: $7,300% profitable: 41%Profit factor: 1.58

Conclusions

In this article we have shown how to perform a walk forward optimization on an intraday mechanical system. The walk forward results are in line with the in-sample results and this builds confidence in the robustness of the strategy.

For more information on algorithmic trading strategies please visit: http://www.vbosystems.info

— By Amon Licini of VbO Systems. VbO Systems is a developer of 100% automated trading systems coded in NinjaTrader that can be auto-traded on almost all asset classes. Amon Licini, the founder of Vbo Systems, has been a private trader for 15 years and a senior manager with various corporates in Italy. Amon’s main trading interests lie in the area of volatility and open range breakouts for intraday systems. He

lives in Milan with his wife and 2 children and loves traveling when he’s not developing new systems. Amon has a degree in mechanical engineering from the Polytechnic University of Milan.

Risk-Based Position SizingSeptember 30, 2013 5:00 am11 commentsViews: 4390

Let’s start with a thought experiment. Let’s say we have two traders who are each given identical trading systems to execute on the SPDR S&P 500 ETF (SPY). Because both systems are identical copies of an automated trading system, both trading systems generate the same buy/sell signals, use identical stop loss and trailing stop parameters. These two trades are also going to start trading on the same day, with the same starting account size.  In essence let’s pretend we have two traders that have identical trading circumstances. At the end of the trading period, we would expect them to have the same account balance, right? But at the end of the trading period, reality has made fools of us. One trader generated nearly twice as much net profit. How can this be? Reviewing the trading results for both traders you can see both traders took the same signals. Both traders have the same win/loss ratio and even the same number of stop-out trades. But there is one difference. One trader used a simple mathematical formula to determine how many shares to buy. This formula utilized a risk-based metric to calculate the number of shares thus, help to protect the trader from risk. His account grow at a much faster rate.

Position sizing is an integral part of both system trading and discretionary trading alike. It answers the question, how many shares or contracts should I buy? Too often this question is answered by nothing more than an educated guess. Too many traders position size their trades by not accounting for risk in determining the number of units to buy. In doing so you could be sabotaging your results, your system and maybe your career as a system trader! By tying your trade size to a risk metric you are acting defensively in the face of a dynamic market. Managing risk should be a top priority of any good trader. Thus, using a risk-based metric to help you mathematically determine how to size your trade is vitally important.

For most people they simply use a fixed number of shares or dedicate a fixed dollar amount for a trade. Let’s think about this for a moment. When a trading system generates a new buy signal you could risk all of your cash on that one signal. If you win you might double your money very quickly. Yet if you lose, your account it at zero. It’s probably not a good idea to bet all your account on a single trade, right? Most people understand this extreme example and thus, will not bet their entire trading equity on a single trade. It’s way too foolish. So, how much do you bet? Maybe 90% of your account? Or maybe you decide 100 shares is about right. What many system developers fail to see is how their trade size affects the risk and reward of the given trade. As you risk more, the payoff can be greater. However, risk too

much and you can be quickly wiped out by a string of losing trades. There exists a theoretical optimal balance between the risk and reward based upon the given trading system. Your job is to find a near optimal level that is comfortable for you as a trader.

While talking about the nuances on finding an optimal position sizing method for a given trading system is well beyond this article, I’m going to demonstrate the difference between using a risked-based position sizing algorithm vs. simply buying a fixed number of shares. Buying the fixed number of shares will represent what most people do (not taking into account a risked based metric to properly size their trades for the given market conditions).  Does using a risk based really help? Let’s find out.

Our Trading System

We will be using a simple strategy model for our demonstration on position sizing algorithms. I chose a simple RSI based system since most strategy traders have experience with this type of setup. I based our testing on daily SPDR S&P 500 ETF (SPY) data going back to 2/1/1993. Trades are executed on a daily bar and all trades are long only. Calculations are performed at the end of the trading day and orders are placed at the open of the next trading day. In all cases, we assumed a starting capital of $50,000. $20 for commission and $.02 per share for slippage was accounted for each round trip.

Go Long When:2-Period RSI is below 10

Exit When:

2-Period RSI > 70

Below is an example of the strategy trading on the daily chart of SPY over the past couple of months.

Method 1 Fixed Shares

The fixed-share method is a non-risk-based method. In this case we simply buy the same number of shares (200) for each signal.

Shares To Buy = 200

Method 2 Fixed Dollar

The fixed-dollar method is a non-risked based method. In this case we simply dedicate the same dollar amount for each signal. In this case we are going to use 50% of our starting equity to each signal which comes out to be $25,000 per trade.

Shares To Buy = $25,000 / Current Price

Method 3 Percent Risk

The percent-risk method is a risked based method. In this case we determine a fixed percentage of our equity (2%) to risk on each signal. This dollar amount is then divided by the dollar amount you are willing to risk on a individual trade. If our trading system had a known stop value we would use this value in our calculation.

However, since our demo trading system has no stop value we are simply going to estimate a dollar number based upon the security’s price. Let’s use 5% of ETF price as the amount to risk. For example, let’s say SPY is trading at $100 and we are willing to risk five percent. Thus, we are risking $5 ($100 * .05 = $5 ). As the amount we risk climbs, we reduce the number of shares to purchase.

Shares To Buy  = (2% of Total Equity) / ( 5% of  Current Price)

Method 4 Percent Volatility

The percent-volatility method is a risked based method. In this case we determine a fixed percentage of our equity to risk on each signal. For our example we are going to risk 2%. We then take this dollar amount and divide it by a multiple of the security’s 10-day average true range. In our case we are using 3 times the 10-day average true range. In other words, we are dividing our risk capital by the average amount the security moves within 3 days. As volatility increases we reduce the number of shares to purchase.

Shares To Buy = (2% of Total Equity) / 3*(10-Day Average True Range)

Conclusion

In this article I wanted to demonstrate the difference between non-risked based position sizing methods (Fixed Shares and Fixed Dollar) and risk-based methods (Percent Risk and Percent Volatility). As you can see our demo trading system generates improved trading performance with the risk-based positions sizing methods. You can see this with the increase with the Profit Factor scores as well as the increased profit per trade. Most traders simply use the non-risk based methods or guess the number of shares. Such a strategy may not be optimal for your trading system. It will be important to test different position sizing methods to find out what works best. 

While this is far from an in-depth look at position sizing and how it can be used to improve your trading, this should highlight the impact a position sizing model can have on your system. 

Account Undercapitalization is the Primary Cause For Trader FailuresSeptember 2, 2013 5:00 am1 commentViews: 741

— By Michael Harris, Price Action Lab

About 11 years ago, I gave a presentation in a conference in New York City as an invited speaker. The audience was mostly comprised of inexperienced traders. A good part of my presentation was on the subject of risk of ruin and proper account capitalization.

After I explained what consecutive losers are, how they contribute to drawdown and that getting 4, 5 or even 7 consecutive losers is normal in trading, I wrote down this simple formula that determines proper account capitalization:

Trading account minimum capital = Amount risked per trade / Percent risk

The amount risked per trade is usually determined by the stops. The percent risk determines what percentage of the account is risked on each trade. For example, if a swing trader of ES futures risks $500 on each trade and each time only 2% of the account should be at risk, then the required trading account equity is equal to $500/0.02 or $25,000.

Let us assume now that the trader starts with $5,000 in the account. The trader knows that this is a small amount to start with and an attempt is made to make the stops as tight as possible without affecting a lot the profitability of the trading method. The improved trading method risks only $350 on each trade instead of $500. After rearranging the above equation  we get

Risk percent = Amount risked per trade/Trading account equity

In the case of the new example with the improved stop, the risk percent turns out to be $350/$5,000 = 0.07 or 7%. This means that 7% of the account is risked on each trade. Does this sound like a lot?

It sounds like way too much to experienced traders but often too little to newcomers or gamblers. This is because inexperienced traders always hope for a good streak of winning trades first to act as a cushion before a drawdown arrives. This happens rarely experience (and math) says and a drawdown arrives first. With 7% risked on each trade a 5K account will be reduced to $3478 after 5 consecutive losers and to roughly 3K after 7 consecutive losers. If the trader cannot cover the margin requirement then any open positions will be liquidated. Reducing the probability of ruin requires a smaller risk percent, like 2% for example, and in that case the proper account equity size is determined by the original formula and it is equal to $350/0.02 or $17,500. The difference in the two accounts is a respectable 250% or in other words, the trader with the 5K account is undercapitalized by a factor of 2.50.

These are some of the facts I tried to explain 11 years ago with many more examples and details that included determination of account size for futures trading systems and taking into consideration drawdown estimates. (More information can be found in one of the books I have authored, which are all out of print by the way). Shortly after I finished my presentation, someone from the organizers approached me and told me that a group of individuals rushed to her office and complained that I was discouraging them from getting involved with trading because none of them had more than 5K to start with and they got scared after they attended my presentation. That did not come as a surprise to me. I expected that type of a reaction.

What happened during that interesting period of the markets, which included a rally of the Dow Jones to all-time highs, the dot-com boom and the bear market that followed, is that many traders after failing a few times to make it in swing or position trading due to their low capitalization, they entered the day trading arena. They either figured it out themselves or were directed there by those who realized that the only way to keep the capitalization requirements low and at the same time avoid the risk of ruin was to lower the amount risked per trade down to a few ticks. Thus, when the ES futures started trading most day traders risked a few ticks per trade, like ann equivalent of $5o to $100. Some even tried and are still trying, 1 tick stops, or $12.5. That certainly lowers the capitalization requirements. For example, for a 4 tick stop, or $50, and 4% risk per trade the required account equity is just $1,250. What is the catch?

The catch goes by the name volatility.   At that level of trading there is a lot of noise, or random walk, generated by all the market participants. This neutralizes any small trading edges after a while because of the commission effect. If commission paid is about 10% of the average winner for scalpers (traders who chase a few ticks), for example, and the winning edge is only 5%, or 55% winners, then it is very hard to make money. The only way out is by increasing the edge substantially, or decreasing the risk to reward ratio, something that we all know is very hard to do in those timeframes. Thus, the majority of scalpers are doomed. Only those with a substantial edge and low risk to reward ratio survive.

Then, there is the issue of HFT in the equity markets. In those markets, the flood of ambitious, poorly capitalized traders was dealt with fast order placing that essentially creates an additional commission effect by front-running orders and thus increasing the demand for even larger winning edges to levels beyond what is achievable. Scalping the equity markets is considered dead at this point and most active traders have increased their timeframes to beyond 5 minutes, some to 30-minute intervals or to even longer.

Proper trading account capitalization is very important but it cannot help those with an edge smaller than the commission wealth-out effect on their account. Update (04/23/2011): this is a post with   specific examples of the impact of commission rate   on profitability coupled   with   account capitalization . It shows that depending on starting capital and commission rate, risk of ruin is possible regardless of the presence of an edge

Who Needs the Trading Day? I do

Posted on February 5, 2011

A study last month in the Bespoke blog concluded that the simple strategy of buying on the close and selling on the open of next day would have significantly outperformed buy and hold since the launch of SPY ETF. The Bespoke results were published in several other blogs, for example here. According to the study, the simple strategy return would have been 378% as opposed to a return of 193% for the buy and hold strategy. Surprisingly enough, they admitted that the return for buy and hold did not include SPY dividends.  This is a curious assumption but this was not the main problem of the study. The conclusions of that study is an example of how one can be fooled by simplified analysis and extraordinary results.                      

Before I start I must say that one thing I learned in the last 23 years I have been developing trading systems for qualified traders, corporations and funds is not to trust any analysis, any analysis/backtesting program and any results published anywhere, even in the most prestigious journal. The only results I trust are those I derive myself using realistic assumptions and taking into account all the major factors that can influence the calculations. Having said that, I find not including dividends in the buy and hold return calculations quite unreasonable. Long term buy and hold investors target value and dividends. Before I discuss some more serious issues with the Bespoke study, let us first see how the inclusion of dividends changes the buy and hold return calculated.                             

The study calculated a buy and hold return of 193%. This would have been the return if someone had bought  and held SPY. This number did not include dividends. The impact of dividends, which is of course substantial, can be estimated fairly accurately by using the adjusted close of January 23, 1993, the day SPY was launched, instead of the actual close. This number can be found here and it is $32.13. The actual close was $43.94. Let us go over the calculations:                              

Using the actual close:                             

Return = 100 × (close of 02/03/2011 – close of 01/23/1993)/close of 01/23/1993 = 100 × (130.78 – 43.94)/43.94 = 197.6%        

Without reinvestment of dividends, the calculation of the return is as follows:     

Return = 100 × (close of 02/03/2011 – adj. close of 01/23/1993)/close of 01/23/1993 = 100 × (130.78 – 32.13)/43.94 = 224.5%                           

With reinvestment of dividends, the maximum possible return may be approximated as:                       

Return = 100 × (close of 02/03/2011 – adj. close of 01/23/1993)/adj. close of 01/23/1993 = 100 × (130.78 – 32.13)/32.13 = 307.6%                              

Thus, the difference is significant and ranges from 27% up to 100%. But this is not where the problems with the Bespoke study ended. Actually, not accounting for dividends was the least of the problems. The study did not even account for

commissions and this is much more important than not including dividends.                              

Commissions paid per transaction or on a per share basis can significantly impact the profitability of a trading strategy. As a matter of fact, many simple strategies turn quite unprofitable when commissions are included. The material that follows is quite revealing.                              

Let us first try to reproduce the Bespoke results using their simple strategy of buying SPY on the close and selling on the open of next day. The following graph shows how a $10,000 account would have grown. These results are in agreement with the Bespoke study. The numbers are a bit different because my study included data up to 02/03/2001:                              

Buy SPY on the close, sell on the next open - No commissions

 

It may be seen from the above graph that a $10,000 account grew to about $50,000. The actual return was 385%. This was a compound return, meaning that the cumulative equity was used in calculating the number of shares to purchase. Thus the number of shares was not constant but increased or decreased based on available equity.                              

Now, this is quite interesting. Below is a graph of the equity of a $10,000 account with commission $7 per trade. This is the average commission charged by many retail online brokerages:                              

Buy SPY on the close, sell on the next open - $7 commission per trade

 

 Two things: not only the equity dropped straight down on a perfect straight line but the account was totally ruined by 04/08/1996,  almost 3 years after trading began. This happened because, as a general rule, equity growth in stock trading is a function of commission rate and account size. The edge of the simple strategy was very small and when commissions were added it disappeared. This is how naive trading strategies are hit when reality is taken into account.                              

Below is a table that shows the effect of a $7 commission per trade for various values of starting equity. It may be seen from the table that trader accounts of up to $50,000 were totally ruined. The corresponding date of ruin is also shown:  

Staring Capital Drawdown % Ruin date Return %

10,000 100 04/08/1996 –

20,000 100 07/15/1996 –

50,000 100 03/15/2009 –

75,000 46.64 – 42

100,000 38.43 – 128

We may see from the table that an investor with $50,000 in her account paying $7 commission per trade would have lost everything by the middle of March, 2009 following this simple strategy of buying the SPY on the close and selling on the open, when a buy and hold investor would have made a substantial return in contrast. It would take a large starting capital to make respectable returns and overcome the commission wealth-out effect.                              

At this point someone could argue that $7 per trade is a lot and there are brokerages that charge much lower commission. This is what was done next. A commission rate of 1/2 cent per share was used and the graph below shows the resulting equity curve for $10,000 of starting capital:                              

Buy SPY on the close, sell on the next open - $0.005 commission per share

 

It may be seen from the graph above that the return in this case was 193.3% (It happened to be equal to the buy and hold return of the Bespoke study with no commission included). Not even near the buy and hold return that included dividends.

As a matter of fact, the buy and hold strategy that took account of the dividends outperformed this simple trading strategy with an additional return up to 100% on the starting capital depending on how dividends were reinvested.                              

Now, let us come to another point of the Bespoke study. It is true that the simple strategy of buying SPY on the close and selling at the open of the next day would have produce positive returns, depending on commissions and although much lower than the buy and hold strategy, as opposed to the strategy of buying at the open and selling on the close, which would have been a loser but this cannot drive the conclusion that day trading was an inferior strategy. First, day traders do not buy on the open and sell on the close. This is a naive strategy that most traders know it doesn’t work. Intraday traders buy and sell several times during the day based on a variety of methods that are beyond the scope of this article. Second, it just happened that this simple strategy in the Bespoke study worked for SPY. It does not work for other markets and as such it is the result of survivorship bias. There was no basis for choosing this strategy back in 1993.                              

The message from the Bespoke study, the author of which I must thank for motivating this article, is that as far as trading the markets, if an analysis looks too good to be true, it is definitively too good to be true.                              

Have a good trading week!