This post comes to you from Dr Tom Starke, a good friend of Robot Wealth. Tom is a physicist, quant developer and experienced algo trader with keen interests in machine learning and quantum computing. I am thrilled that Tom is sharing his knowledge and expertise with the Robot Wealth community. Over to you, Tom.
Unlike most other businesses, algorithmic trading has the advantage of giving you almost instant feedback on how good you are in your business. For anyone who is numerically inclined this is a very attractive proposition. I have seen articles written about this subject but they have never really addressed a lot of the issues I have come across on my journey. In this post I would like to talk about this a little as an inspiration or perhaps a deterrent for all the people who read this and consider making money that way.
Nothing could be more amazing, a system that runs by itself and consistently spits out cash to finance prolonged stays in Bali, South America or with your mom if that’s what you’re after.
However, as you may have guessed, it ain’t that simple, even for the most hardened trade-savvy – math-wiz or programming genius. First of all, algo-trading is the multi-disciplinary by nature. This makes it both a lot of fun and challenging at the same time. You have to be fairly clued up in areas such as computer science, maths, data analysis, and some finance although I found that people who understand the first few subjects usually learn the finance bit very quickly.
So you want to build your own algo trading system?
Perhaps you might think that a lack of knowledge in some of these areas can be resolved by knowing the right people and teaming up with them. It may, but from my experience the odds of success aren’t great. I have personally met three guys with extensive computer knowledge that have teamed up with institutional traders to develop trading ideas into automated systems – none of them worked out.
Why is this so? In my experience, traders often work with technical analysis and respond to trading signals, which to them are “obvious” buys and sells and they often make good money from that. The problem here is that their awareness is very fine-tuned to see the relevant signals that have potential and they are almost blind to the ones that are there but obviously make no sense. In their view, the signals they use work 100%. A computer, however, does not have the ability to discern signals that make sense and the ones that don’t, it just executes regardless. As a result, the trader thinks that his computer guy made mistakes in the program and the programmer thinks that the strategies of the trader are no good, and finally they go their own way.
Take a typical institution, a quant develops a prototype of his system in, say MATLAB, and the IT guy converts it into C++ to be executed on the fund’s platform. A well-designed algo contains a lot of subtleties that the developer probably won’t understand exactly and tiny mistakes can completely kill the performance of the system.
These are common scenarios and I have had my fair share of that too. I’m all up for team work but the reality is that before you can outsource parts of your project you have to understand it yourself really well.
Now, say you come from a quantitative background and you read all the books, did the exercises, ran a bunch of backtests and you are ready to go. You start to build your own platform and realise that there are a number of show-stoppers such as getting good market data, historical data, broker connections and the like. These problems are more significant than you think. The easiest solution is to get a Bloomberg connection but it comes at a significant cost. Also, a typical Bloomberg terminal is not really built to run sophisticated number-crunching algos. How about Interactive Brokers? Yes, it can work but their API is far from perfect and their historical data are annoyingly full of errors and their market data have a lot of false signals too, which can be detrimental to your strategy. All this can be sorted out but trust me, it’s not easy. The little details create the complexity and it takes time to even learn about all the pitfalls that may come up.
But here’s the thing: if this stuff was easy, I would have never got into it because complexity creates opportunity. I like learning and I enjoy solving problems so this is a lot of fun. In the process I learned more than during my entire physics PhD … how cool is that?
How to start depends on your skills and experience, but here are some recommendations.
First of all, find the lowest commission broker you can. Commissions are the single biggest killer of an otherwise good strategy. One basis point (1 percent of 1 percent) does not sound like much but each extra basis point of commission that is added to your trade brings you into entirely different trading regimes. More commission means you have to wait longer to exit, your strategies are riskier and less consistent.
Second, don’t try to automate everything right away. Semi-automated strategies are actually really cool. What that means is that your machine supplies you with the signals and you carry out the execution by hand (provided that you are not executing 46 trades per second). To implement good automated execution strategies is actually quite tricky. What do you do when your trade does not get filled? Or it gets partially filled? Or you cannot exit at the price you want? Or you see that it might get filled soon, so is it ok to wait for a little while? Or do you want to cancel straight away. If you have done this by hand before it is much easier to develop something that accounts for all the scenarios you might encounter.
With semi-automated systems, if you stoically follow what your strategy tells you to do, it can work well, since you don’t have to worry about also watching the market at the same time. We all know the guys with 6 screens in front of them trying to make sense of what happens in the market – I have never seen a more anxious, sleep-deprived and adrenaline-ridden group of people than those. Often, they don’t even have a strategy and just react to what they think is rational.
Whilst developing your system you will realise that even the best fully automated system in the world will not allow you to lie on the beach, sip cool drinks, and relax while the trades tick away and make you money. Say, your strategy does actually run well and generates profit, but suddenly your internet cuts out or your server has a power surge. Does your system stall or smoothly reconnect and carry on? Do you have a backup server that runs in sync and can take over (not trivial)? Can you somehow close all your open positions through an alternative internet connection? Or maybe you do it the Warren Buffet way and leave your positions open for the next 24 years. What if your strategy experiences a large drawdown? Is it expected or does it go beyond the acceptable limit and you have to cut losses? Have you pre-determined at what point you do that?
This takes us to considerations in how you design your system. My journey and probably yours, will start with a simple design that just does what it’s meant to do but quite slowly and inefficiently. You move on to run bits in parallel, introduce complex event processing and end up running the different parts of your trader via sockets on external servers that exchange messages asynchronously. I’ve iterated through all of those myself and learned a decent amount of computer science along the way. If you are not already an experienced developer I highly recommend starting simple, rather than building a big system right away. By the way, that was one of my first humbling mistakes – oh well, we all learn. By now, I’ve built a wide variety of different systems from simple to complex and each one helped me to understand different aspects of the field.
It is also handy to backtest simulated execution, since some strategies significantly suffer when slippage, market impact and other factors are taken into account. Often, the tests for trading signals, portfolio and execution can be run separately, which simplifies the process and makes it easier to analyse and understand.
A big stumbling block, especially for people coming from computer science and finance, is the processing and statistical interpretation of the backtest data they have gathered. It isn’t trivial and it can take years of experience to get good at that. Data tell stories, learn to find those stories in the numbers.
If you have made it all the way to here and you are at the beginning of your journey you may feel a bit overwhelmed but think about it this way: there are all those amazing things you can learn on your way that will eventually enlighten you. If it was easy, everyone would do it.
Conclusion
If you ask me what is the most important aspect you need in order to get this to work, I’d say it’s vision. Vision of what you actually want to achieve. This, more than anything else will pull you in the right direction. In my honest opinion, it’s not the smartest people that succeed with this, it’s the visionaries. If you’re not sure what I’m talking about I suggest you have a look at the books by the likes of Charles Haanel, Napoleon Hill and Wallace D. Wattles. Whether you are a scientist, financial analyst, software engineer or hairdresser, you will have large gaps in your knowledge and it takes determination to fill them. So, go ahead, build you vision, put your sleeves up and get started!
Great post, thanks Tom! Truly motivating.