The Big Picture: what is the best way to test Forex Trading Strategies? Quick research on this topic shows that the main focus is on Strategy Testing Software. Though it is very important, the software is only a part of the whole picture. In comparison, little attention is paid to the problem of the testing methodology. However it is crucial in getting objective estimates of the quality of the trading strategy.
In classic approach, the goal of testing is to manage the risk and provide numerical estimates of quality. This is achieved by assessing the probabilities of failures, based on the results of carefully selected test scenarios, which are to be run during available timeframe. We apply these classic principles to Forex Strategies testing, aiming at managing risk by defining appropriate test strategy and developing test scenarios.
Risk Management. Purpose of testing is reducing the risk and impact of possible failures. More testing normally means less risk. The risk can be virtually eliminated for simple systems, when essentially all possible inputs can be tested. For Forex Market, the situation is principally different. As the markets are unpredictable, after any amount of testing there will still be infinite number of “untested inputs”. Hence, any successful test is no guarantee of the future performance.
Does this mean the Forex strategy testing is completely useless? In our opinion, this question is too theoretical – regardless of the correct answer.
Of course, the aim of reducing risk via testing must remain. However, we can’t obviously demand that strategy testing would manage trading risks in all situations. So, we need to revisit definitions of failures and, more importantly, testing success criteria. For example, we can measure success by identifying certain conditions when trading strategies can work. This can be done by evaluating probabilities. So, if for at least some parameter areas we can estimate the risk being under 50%, this can be a measure of success.
Test Strategy is another fundamental element of classic testing. Proper strategy ensures balance between risk level and efforts needed to achieve it. Unpredictability of forex markets makes test strategy even more important: as we remember, any testing still can’t give us 100% test coverage. On the other hand, testing multitude of strategy parameters against big amount of market data is already complex and time-consuming task. So, the test strategy is responsible for selecting essential Test Scenarios so that the trustworthy quality level can be achieved in reasonable time.
To make risk evaluation easier, we need to simplify. One approach is to evaluate strategy risk independently of market risks. This is in line with the observation that Trading Strategies fail more often when market behavior changes (high market risk), rather than in “stable” market conditions (low risk). The advantage of such approach is that for the cases of low market uncertainty, we can use all well-known classic testing techniques to get quantifiable quality estimates.
Test Scenarios. Classic test scenarios include most common use cases (positive) and most probable anticipated failures (negative). Our approach is in using price action patterns as Test Scenarios. Even though the Forex Market is unpredictable, patterns can clearly be observed there. Elliot Waves are one of the well-known examples of market patterns. We try to identify the patterns best suited as our Test Scenarios by analyzing performance of the Trading Strategies in real market conditions.
For example, trending markets is a positive scenario. All trends are similar in the sense of clear overall market direction, so consequently the market risk is low. Negative scenarios are those where market risks are high – e.g. when the market conditions change, or where Trading Strategies are observed to consistently fail. The principal quality evaluation criteria for negative scenarios are recognizing them as early as possible and minimizing losses.
For the positive scenarios, main focus of testing is on optimizing the parameters of the trading strategy. For the trending market example above, the test scope consists of the set of the trends of various shapes and sizes, for different timeframes. Each trend is unique – e.g. pullbacks are all of different shape and magnitude. In this case, quality of the strategy is essentially that of algorithm and its implementation. Consequently, such testing provides principally more valuable results than back fitting to the markets on a random time interval.
Main Techniques How To Test Forex Strategies
Most widely used technique is “backtesting”, when the strategy is run against the historical Forex Market data either manually or, for the Automated Trading System, with the help of some tool.
Another option is “forward testing“, which is using the strategy in Live Forex Trading. This approach might look attractive at first sight. However, the principal problem is that such testing can only be done in real time, and not any faster. To say nothing of the cost incurred, of course.
“Paper Trading” differs from forward testing in that the trades are executed not on the Live Forex Account, bun on a Demo one. This will obviously save from money losses. However, this method is the least effective. Not only is it as slow as forward testing, but in the most cases the market quotes sent by Demo Trading Servers are not good enough for testing. Though both might come from the same data feed, but a Demo server usually sends only part of the quotes that the Real Trading Server sends.