# backtesting

Anyone that's been around the markets knows that the monthly release of the United States Department of Labor's Non-Farm Payrolls (NFP) data can have a tremendous impact, especially in the short term. NFP is a snapshot of the state of the employment situation in the US, representing the total number of paid workers, excluding farm employees and public servants. We know your barn is hiding a giant mining station, Rick The release of the monthly NFP data typically causes large swings in the currency markets, even when the results are in line with estimates. Here, we are interested in exploring potential seasonal effects around the release of this data. For example, does price tend to drift prior to the release? If so, which way? For this analysis, we'll explore the EUR/USD exchange rate. To set up this research problem, we need to know that NFP is released on the first Friday of the month at 8:30am ET - usually. If the first Friday is a holiday, NFP is released the following Friday. These sorts of details can make seasonal analysis...

In this post, we are going to construct snapshots of historic S&P 500 index constituents, from freely available data on the internet. Why? Well, one of the biggest challenges in looking for opportunities amongst a broad universe of stocks is choosing what stock "universe" to look at. One approach to dealing with this is to pick the stocks that are currently in the S&P 500 index. Unfortunately, the stocks that are currently in the S&P 500 index weren't all there last year. A third of them weren't there ten years ago... If we create a historical data set by picking current S&P 500 index constituents, then we will be including historical data for smaller stocks that weren't in the index at that time. These are all going to be stocks that did very well, historically, or else they wouldn't have gotten in the index! So this universe selection technique biases our stock returns higher. The average past returns of current SPX constituents is higher than the average past returns of historic SPX constituents, due to this upward bias. It's easy...

In the first three posts of this mini-series on pairs trading with Zorro and R, we: Implemented a Kalman filter in R Implemented a simple pairs trading algorithm in Zorro Connected Zorro and R and exchanged data between the two platforms In this fourth and final post, we're going to put it all together and develop a pairs trading script that uses Zorro for all the simulation aspects (data handling, position tracking, performance reporting and the like) and our Kalman implementation for updating the hedge ratio in real-time. You can download the exact script used in this post for free down at the very bottom. Let's go! Step 1: Encapsulate our Kalman routine in a function Encapsulating our Kalman routine in a function makes it easy to call from our Zorro script - it reduces the call to a single line of code. Save the following R script, which implements the iterative Kalman operations using data sent from Zorro, in your Zorro strategy folder: ###### KALMAN FILTER ####### delta <- 0.0001 Vw <- delta/(1-delta)*diag(2) Ve <- 0.01 R <- matrix(rep(0,...

This Kalman Filter Example post is the first in a series where we deploy the Kalman Filter in pairs trading. Be sure to follow our progress in Part 2: Pairs Trading in Zorro, and Part 3: Putting It All Together. Anyone who's tried pairs trading will tell you that real financial series don't exhibit truly stable, cointegrating relationships. If they did, pairs trading would be the easiest game in town. But the reality is that relationships are constantly evolving and changing. At some point, we're forced to make uncertain decisions about how best to capture those changes. One way to incorporate both uncertainty and dynamism in our decisions is to use the Kalman filter for parameter estimation. The Kalman filter is a state space model for estimating an unknown ('hidden') variable using observations of related variables and models of those relationships. The Kalman filter is underpinned by Bayesian probability theory and enables an estimate of the hidden variable in the presence of noise. There are plenty of tutorials online that describe the mathematics of the Kalman filter, so I won't...

Chart patterns have long been a favourite of the technical analysis community. Triangles, flags, pennants, cups, heads and shoulders.... Name a shape, someone somewhere is using it to predict market behaviour. But, we need to find out if there is a grain of truth or reliability in these patterns. Can attempts to objectively measure these patterns, such as with the Frechet distance, really give you a hint as to the future direction of the market? Or should you stick to an approach we know makes money? ah, I see a blue star pattern on my chart... a good omen. The problem is that such an approach is inherently subjective since price action almost never matches perfectly with the idealized version of price patterns you see in every beginner's guide to trading. It is up to you, the individual, to determine whether a particular chart formation matches closely enough with a particular pattern for it to be considered valid. This is quite tricky! It's very difficult to codify a trading system based on their use. By extension, it is difficult to test...

When tinkering with trading ideas, have you ever wondered whether a certain variable might be correlated with the success of the trade? For instance, maybe you wonder if your strategy tends to do better when volatility is high? In this case, you can get very binary feedback by, say, running backtests with and without a volatility filter. But this can mask interesting insights that might surface if the relationship could be explored in more detail. Zorro has some neat tools that allow us to associate data of interest with particular trading decisions, and then export that data for further analysis. Here's how it works: Zorro implements a TRADE struct for holding information related to a particular position. This struct is a data container which holds information about each trade throughout the life of our simulation. We can also add our own data to this struct via the TRADEVAR array, which we can populate with values associated with a particular trade. Zorro stores this array, along with all the other information about each and every position, as members of the TRADE struct....

In an ideal trading universe (free from backtesting bias), we’d all have a big golden “causation magnifying glass”. Through the lens of this fictional tool, you’d zoom in and understand the fleeting, enigmatic nature of the financial markets, stripping bare all its causes and effects. Knowing exactly what causes exploitable inefficiencies would make predicting market behaviour and building profitable trading strategies a fairly cushy gig, right? If you’re an engineer or scientist reading this, you are probably nodding along, hoping I’ll say the financial markets show some kind of domino effect for capitalists. That you can model them with the kinds of analytical methods you’d throw at a construction project or the petri dish. But unfortunately, trying to shoehorn the markets into formulas is a futile exercise... like stuffing Robot Wealth’s frontman Kris into a suit. Since the markets aren’t strictly deterministic, this makes testing your new and exciting strategy ideas a bit tricky. We’d all love to know for sure whether our ideas will be profitable before we throw real money at them. But, since you can’t realistically apply...

I recently read Gary Antonacci's book Dual Momentum Investing: An Innovative Strategy for Higher Returns with Lower Risk, and it was clear to me that this was an important book to share with the Robot Wealth community. It is important not only because it describes a simple approach to exploiting the "premier anomaly" (Fama and French, 2008), but because it is ultimately about approaching the markets with a critical, inquisitive mindset, while not taking oneself too seriously. I think we can all do with a dose of that sometimes. Gary's style is unique: this is the work of a free and critical thinker who is not afraid to question the status quo. While articulately drawing from a range of sources, from Shakespeare to Bacon and Einstein to Buffet (even Thomas Conrad's 1970 book Hedgemanship: How to Make Money in Bear Markets, Bull Markets and Chicken Markets While Confounding Professional Money Managers and Attracting a Better Class of Women, which has got to be the greatest title in the history of trading books), Gary comes across as playful and slightly eccentric (which is wonderfully refreshing...

This is the final post in our 3-part Back to Basics series. You may be interested in checking out the other posts in this series: Part 1: An Introduction to Algorithmic Trading Part 2: How to Succeed at Algorithmic Trading We've also compiled this series into an eBook which you can download for free here. Nearly all research related to algorithmic trading is empirical in nature. That is, it is based on observations and experience. Contrast this with theoretical research which is based on assumptions, logic and a mathematical framework. Often, we start with a theoretical approach (for example, a time-series model that we assume describes the process generating the market data we are interested in) and then use empirical techniques to test the validity of our assumptions and framework. But we would never commit money to a mathematical model that we assumed described the market without testing it using real observations, and every model is based on assumptions (to my knowledge no one has ever come up with a comprehensive model of the markets based on first principles...

In the first Mean Reversion and Cointegration post, I explored mean reversion of individual financial time series using techniques such as the Augmented Dickey-Fuller test, the Hurst exponent and the Ornstein-Uhlenbeck equation for a mean reverting stochastic process. I also presented a simple linear mean reversion strategy as a proof of concept. In this post, I’ll explore artificial stationary time series and will present a more practical trading strategy for exploiting mean reversion. Again this work is based on Ernie Chan's Algorithmic Trading, which I highly recommend and have used as inspiration for a great deal of my own research. Go easy on my design abilities... In presenting my results, I have purposefully shown equity curves from mean reversion strategies that go through periods of stellar performance as well as periods so bad that they would send most traders broke. Rather than cherry pick the good performance, I want to demonstrate what I think is of utmost importance in this type of trading, namely that the nature of mean reversion for any financial time series is constantly changing. At times this dynamism can...