# backtesting

This is the final post in our 3-part Back to Basics series. You may be interested in checking out the other posts in this series: Part 1: An Introduction to Algorithmic Trading Part 2: How to Succeed at Algorithmic Trading We've also compiled this series into an eBook which you can download for free here. Nearly all research related to algorithmic trading is empirical in nature. That is, it is based on observations and experience. Contrast this with theoretical research which is based on assumptions, logic and a mathematical framework. Often, we start with a theoretical approach (for example, a time-series model that we assume describes the process generating the market data we are interested in) and then use empirical techniques to test the validity of our assumptions and framework. But we would never commit money to a mathematical model that we assumed described the market without testing it using real observations, and every model is based on assumptions (to my knowledge no one has ever come up with a comprehensive model of the markets based on first principles...

In the first Mean Reversion and Cointegration post, I explored mean reversion of individual financial time series using techniques such as the Augmented Dickey-Fuller test, the Hurst exponent and the Ornstein-Uhlenbeck equation for a mean reverting stochastic process. I also presented a simple linear mean reversion strategy as a proof of concept. In this post, I’ll explore artificial stationary time series and will present a more practical trading strategy for exploiting mean reversion. Again this work is based on Ernie Chan's Algorithmic Trading, which I highly recommend and have used as inspiration for a great deal of my own research. Go easy on my design abilities... In presenting my results, I have purposefully shown equity curves from mean reversion strategies that go through periods of stellar performance as well as periods so bad that they would send most traders broke. Rather than cherry pick the good performance, I want to demonstrate what I think is of utmost importance in this type of trading, namely that the nature of mean reversion for any financial time series is constantly changing. At times this dynamism can...

In the first part of this article, I described a procedure for empirically testing whether a trading strategy has predictive power by comparing its performance to the distribution of the performance of a large number of random strategies with similar trade distributions. In this post, I will present the results of the simple example described by the code in the previous post in order to illustrate how susceptible trading strategies are to the vagaries of randomness. I will also illustrate by way of example my thought process when it comes to deciding whether to include a particular component in my live portfolio or discard it. I tested one particular trading system on a number of markets separately in both directions. I picked out three instances where the out of sample performance was good as candidates for live trading. The markets, trade directions and profit factors obtained from the out of sample backtest are as follows: USD/CAD - Short - Profit Factor = 1.79 GBP/USD - Long - Profit Factor = 1.20 GBP/JPY - Long - Profit Factor = 1.31 Next, I estimated the performance of...

Picture this: A developer has coded up a brilliant strategy, taking great care not to over-optimize. There is no look-ahead bias and the developer has accounted for data-mining bias. The out of sample backtest looks great. Is it time to go live? I would've said yes, until I read Ernie Chan's Algorithmic Trading and realised that I hadn't adequately accounted for randomness. Whenever we compute a performance metric from a backtest, we face the problem of a finite sample size. We can't know the true value of the performance metric, and the value we computed may or may not be representative of this true value. We may have been simply fooled by randomness into thinking we had a profitable strategy. Put another way, was the strategy's performance simply due to being in the market at the right time? There are a number of empirical methods that can be used to address this issue. Chan describes three in his book mentioned above, and there are probably others. I am going to implement the approach described by Lo, Mamaysky and Wang (2000), who simulated...