How to test your trading system for stability
There is no lack of information on the Internet these days, rather, on the contrary, unverified, incorrect information creates a whole problem for traders. People lose a lot of time and energy trading systems that do not deserve the effort expended on them. And so you decided to check the popular TS found on the network, which is considered very profitable among traders. This article will help you thoroughly test your system for stability and make sure that the resulting robot will withstand all market storms.
What does it mean? For our trading system, this means continuing to trade effectively under different market conditions, adapting to their changes. Such a system should contain a clear and rigorous trading logic, while flexibly adapting to any market conditions, and its parameters should not be too rigid. In other words, your system must be robust.
What is strength?
The figure below shows the tardigrades. This is the most living creature on Earth, which will survive even the end of the world. These distant relatives of crayfish and insects could survive in outer space and multiply there under complete zero gravity, without food and water. They are not afraid of lethal doses of radiation, the fall of large asteroids, explosions of supernovae and gamma-ray bursts.
The strength of a trading system is its ability to remain effective in different markets and under different market conditions. There are several types of vehicle strengths: working period strength, seasonal strength, market phase strength, tool strength, optimization strength, parameter strength, portfolio strength. Further we will consider everything in detail.
Market Phase Strength
Strictly speaking, there are two types of market phases - general and strategic. The general type is determined by the presence of a trend and the level of volatility of the instrument.
If the trading system undergoes tests at any stage of the market, it can be considered strong with respect to the phases of the market. This is the most important type of strength - because if the system works well in a trend, but merging everything earned in a flat, then in such a system there will be little sense. In the picture below you can see the six conditional phases of the market, for which it is worth conducting tests of your system.
The second type of market phases is strategic. It is associated with the foreign and domestic policies of those countries that form a currency pair and its influence is sometimes very great. A commonplace example is the Swiss franc, which has changed dramatically after the Swiss National Bank significantly lowered interest rates and abandoned the exchange rate limit of 1.20 per euro, which he introduced in September 2011 in an attempt to prevent deflation and further appreciation of the currency.
Ideally, of course, the system should not lose money in any of the market phases, but this rarely happens. Therefore, the maximum task here is to lose relatively little on any of the phases. Does it mean that if your system is not stable in market phases, is it worth abandoning it? Of course not, because there are a lot of systems designed specifically for trading in a particular phase. The most important thing in researching the effectiveness of your trading system regarding market phases is to determine the phases in which it is highly undesirable to trade in the future and refrain from trading during such periods or switch to trading systems that are more suitable for the current phase.
Seasonal strength is the ability of a system to remain equally effective regardless of the seasonal effects that occur in the markets. In principle, it was possible to include seasonal sustainability in the stability group by market phases, but I highlighted it separately.
Seasonal effects recurring from year to year certainly exist in the markets. They are most often associated with economic phenomena, the natural behavior of people, and features of state economies. For example, due to the fact that oil is used in the United States for energy, demand for it increases significantly in the winter, that is, when severe frosts occur in North America. Oil is really closely linked to the dollar, which means that all seasonal oil anomalies are reflected in the dollar, and, as a result, in most currency pairs.
But seasonality is also less short-term. Sometimes in trading statistics there are very interesting anomalies, for example, very low system performance on Mondays or at certain hours of the day. Also, often the unsatisfactory operation of the system occurs at the beginning or end of the month.
In most cases, all these anomalies are easily explained. For example, markets are more active at certain times of the day and less active at other times, which is associated with the opening and closing of certain trading floors around the world. You all know that in the Asian trading session, trading is the most calm. Or, for example, everyone knows that in the first half of the first Friday of the month, trading is especially quiet due to the approaching Non-Farm Payrolls.
All these patterns can be easily tracked in myfxbook or any offline program for analyzing statistics. Your task in this case is to identify such points and take them into account in future work. It should be remembered that seasonal effects are unstable and, over time, they appear or disappear. This is due to the fact that market participants are constantly trying to use certain seasonal patterns in their favor and this, of course, affects the overall picture.
For example, from the middle of 2016, night scalpers worked very well, but at the moment we are observing quite high volatility for this time of day, which makes the work of night scalpers not so effective. A little more time will pass and the effectiveness of applying such strategies will completely disappear and people will stop using them. Then the market, quite likely, after some time will again discover this inefficiency and again people will rush to use it until everything repeats in a circle.
The trading system remains stable over the period if it is trading efficiently on different timeframes. There can be two options - either our strategy works fractally, or simply when the period decreases, it remains insensitive to market noise.
The concept of "fractal" has many different meanings, but in this case I mean self-similarity. That is, when a certain figure or pattern is laid at the base of the system, which works equally well at any period. An example is the Elliott waves - it is believed that they can be used on any period.
In general, there are very few such trading systems, so most often the second option is more suitable for us, when the system is insensitive to noise. It is believed that the lower the period, the more noise and more difficult to trade. Therefore, each trading system has a certain minimum threshold, a minimum timeframe, on which the system still remains quite effective.
And our task here is to determine this minimum period and trade on it, because the smaller the period, the more trading opportunities, more transactions and higher profits. In addition, along with a decrease in the timeframe, the stop loss level also decreases, which means that it is possible to manage risks more flexibly and drawdowns will be lower and shorter. If your system works equally well on H4 and on H1, but on M30 it is already bad, you definitely need to choose H1.
But here, as elsewhere, it is important to stop in time. If your system trades well and is below the H1 period, for example, at M15, you need to be especially careful, because when testing at such short periods there are many factors that can significantly distort the final result.
If your system is not durable in timeframes, this is not very scary. It is only important to determine the period of work at which the system is as durable as possible to the phases of the market.
The trading system is strong in instruments when it shows positive results in a wide range of trading instruments. The fact that your system trades equally well in EURUSD, and USDCHF, and AUDUSD, and USDJPY means that you have found global market inefficiency. And she, like a huge diamond, is as durable as rare. In fact, most strategies do not have this type of strength, working effectively on some tools and earning almost nothing or even losing on others.
Therefore, instead of trying to create a universal system suitable for any tool, it is worth focusing your efforts on the search for inefficiency on a specific tool. Let you have a better set of systems, each of which is great for a couple of currency pairs, than one trading so-so, but at all.
A trading system is considered robust for optimization when its parameters on the forward test remain within the parameters obtained during the optimization period. Surely many of you tried to optimize the advisers. It often happens that a particular Forex robot shows excellent results during the optimization period, but to find suitable settings that pass the forward test period is a problem.
In this context, all we can do is always use forward testing to avoid fitting to the history and make sure that the system parameters on the forward test do not deviate much from the system parameters during the optimization period. If the system does not pass the forward test in any way, you should refuse to use it on a specific currency pair.
For a more thorough solution to this problem, they came up with such a technique as Walk Forward Optimization. Its essence is to break the whole story into pieces in a certain way, as shown in the figure:
That is, we carry out optimization on piece A, then conduct a forward test, repeat on the second piece B, and so on. The main goal of this intricate exercise is to check how the system behaves in an unknown “territory”. If the system in all pieces of the forward tests showed statistics similar to those obtained during optimization, then the system is robust by optimization and the probability of fitting its parameters to the history is quite small.
A vehicle is considered strong in terms of parameters if a small change in system parameters (within 10-20%) does not lead to fatal consequences. In other words, if you changed the period of the moving average in your system from 24 to 20 and it leaked the deposit, then it cannot be considered strong in terms of parameters.
If this happens, then such a vehicle will be very sensitive to fitting settings to the story. And you should not use such a strategy - there is a high probability of losses in a real account.
The practical application of this knowledge is as follows. You can easily run off the optimization of each system parameter and see how much its change affects the results. After identifying all such parameters, carry out their general optimization and look at the result. If most of the results (from 50-70%) turned out to be profitable, then everything is fine. If most of the presets are merged, then most likely this system is too sensitive to changes in settings and it is worth either trying to modify it or throwing it away.
When working with MetaTrader 4, on the Optimization Graph tab, you can switch to the Two-Dimensional Surface mode and evaluate the distribution of two selected values:
In the figure above, a system with good strength in selected parameters. Each rectangle is a ratio of two parameters. The higher the profit on the set of settings, the deeper the rectangle has a green color. The results are distributed throughout the area quite smoothly. When you change the settings, the change in profitability also happens quite smoothly.
When looking at this surface, it is worth perceiving it as a topographic map. The greener the color, the higher the rectangle relative to zero yield. Here's what it might look like in 3D:
The Net Return axis - total profit, is the height. The other two axes are optimized parameters, in this case, moving average periods. The highest points on this surface graph from the previous example would look like single rectangles of deep green color. And this is the optimization surface of a bad system. And here is a good system:
Here the peaks no longer have such an acute shape, but are more evenly distributed in space. Flat peaks also speak of such.
In this context, your main task is to check whether all peaks on the optimization surface of your system are flat. Otherwise, it threatens to re-optimize and fit the market.
There is another, more reliable, but also more time-consuming method, proposed by Van Tharp. Its essence is reduced to the calculation of the so-called quality number of the system, determined by the formula:SQN = Squareroot (N) Average (of the N Profit & Loss) / Std Dev (of the N Profit & Loss), where:
Average (of the N Profit & Loss) - the average deal in the system,
Std Dev (of the N Profit & Loss) - standard deviation from the average transaction,
N is the total number of transactions in the test.
Thus, for each combination of the two studied parameters, you need to calculate the SQN and identify the average SQN. It is believed that the value should be at least 2.5. You should get something like this if you use a plate to analyze the results:
Using the color formatting of the cells, one can see how smoothly the results are distributed, which combinations of parameters are optimal and how stable the system is in general to changes in the selected parameters.
A little trick - in order not to read this parameter manually, you can calculate it in the OnTester () function, and then, after optimization, take ready-made values from the Optimization Results tab from the OnTester Result column.
With this article, I introduced you to the main types of stability or strength of trading systems. If you follow these recommendations, your strategies will show the best results, or at least you will significantly reduce the likelihood of losses. The described strength criteria are far from the only ones for selecting a profitable vehicle, by the way.
In truth, this is only one of the criteria, but it is the most important. He will not help you select the best and most profitable strategy. But, guided by all the tricks in the article, you will be able to select the most stable, the most tenacious vehicles that, like slow-moving, will have the maximum possible margin of safety and will delight you with profit for a very long time.
Good luck and see you soon!