This is a short article covering the dangers of curve fitting strategies designed to trade the forex market using automated software.
To design a forex robot, we start off with a set of trading parameters that we believe have some reasonable chance of returning a profit over time. We then backtest those ideas/parameters against historical data.
Evaluating these automated trading system ideas manually is very time and energy consuming. I use a variety of system building tools that identify many thousands of possibly profitable forex robots per day. After the initial identification, the strategies are then automatically evaluated in real-time to get an idea of their success in backtesting. In the rest of the article I’ll show how this avoids the dangers of curve fitting strategies.
This isn’t an easy process. There are many trips, tricks and traps along the way to finally discovering an automated trading strategy with a good chance of performing profitably in the future.
How to Curve Fit a Strategy, And How Not to…
One of the first and worst mistakes a forex robot designer can make is to do with curve fitting strategies to their ideas. To understand this better, let’s take a look at how we test strategy parameters against historical data.
If you take a set of data from date A to date Z, and run endless backtests to find which strategy performed best between those two dates, you have basically curve fitted your results. This is because you will be retroactively fitting the strategy to the data. In other words, you have tested against all the available data, and discarded those strategies that did not give a good result. But if you test against ALL the available data to begin with, you will eventually come up with a set of strategies that by chance, just happened to perform well against that past data.One of the first and worst mistakes a forex robot designer can make is curve fitting strategies to their ideas.Click To Tweet
So, you might get an equity curve from one of these supposedly successful strategies that looks something like this:
Then you go and place this system on a live chart, and in most cases discover that it fails to perform into the future. The way to overcome this is to split your data into evaluation data and verification data. More on this below.
This is the results from a moving average strategy. The blue equity curve at the left shows results from backtesting against a full dataset. The red curve to the right shows the disastrous results in live testing.
The First Step in Building Robustness into Forex Robots:
Instead, in trying to develop a robust strategy that will perform into the future, you should split your data into two sections. In the first section you evaluate the parameters of your strategy against the data from say, A to K.
If the strategy parameters perform well against this data, you then verify its performance against the data from L to Z. In effect, you are pretending that the second dataset is future data. This gives us more confidence in selecting the final strategies to take into live testing. There are however, a couple more steps along the way to a robust strategy, and I will deal with those in future posts.
And for those of you who wondered what happened to the AFX Robots Club, it is getting very close now. I have been toying with two options:
1) A Forex Signals Service, and
2) A Members Club, where each month, the members get a free forex robot that has proven itself in test over the past six months. I am currently trialling around 60 robots, and under this scenario each month, members would get the best performing robot over the past test period.
For those of you who are interested in such a service, please give me an indication in the comments section below of your preference for either 1) or 2). Also, if you can give a rough indication of the price you would be prepared to pay for your preferred service each month.
Take Care & Trade Well,