Of Backtesting

The joys of backtesting. In my last post I described the framework I am utilizing for autotrading. Once the strategy is coded inside the framework, I can choose to run it in live mode, or I can choose to run it thru previous price quotes in order to get a historical perspective on it's performance. This is helpful to reveal bugs more than anything else. People will often "tweak" there strategies in order to gain good backwards testing performance. Unfortunately, this is also how many are scammed with the inevitable forex bots, and pretty graphs showing beautiful equity curves. Suffice to say, tweaking for the past is an exercise in futility, and borders on dangerous. While it is enjoyable to see a strategy perform on an unknown dataset, I have successfully run robots that don't backtest well. The reason of course is that the past is different from the future; which is of course different from today. The only thing that matters is how the robot is performing now, in realtime.

That's not to say backtesting has no place. As I said, it helps to reveal bugs, and can identify or disprove your thesis on the core of the strategy. For example, if I code a trend following system, I would want to backtest it on a dataset containing trends. This will help me to see if my idea on how to idenitfy a trend is correct (and correctly coded). The performance of the robot on this dataset is a secondary indicator, but again is most useful for determining proper code. If a robot wipes the account of otherwise experiences severe drawdown my strategy doesn't call for, it usually means there is a bug somewhere. Typically you can no more always be wrong, than always be right. Either one is a cause for investigation. Or you found the grail. Heh. My money is on a bug!