The Financial Hacker’s Cold Blood Index

Posted on Oct 30, 2015 by Kris Longmore
5 comments
327 Views

This post builds on work done by jcl over at his blog, The Financial Hacker. He proposes the Cold Blood Index as a means of objectively deciding whether to continue trading a system through a drawdown.
I was recently looking for a solution like this and actually settled on a modification of jcl’s second example, where an allowance is made for the drawdown to grow with time. The modification I made was to use the confidence intervals for the maximum drawdown calculated by Zorro’s Monte Carlo engine rather than the maximum drawdown of the backtest. The limitation is that the confidence intervals for the maximum drawdown length are unknown – only those for the maximum drawdown depth are known. I used the maximum drawdown length calculated for the backtest and considered where the backtest drawdown depth lay in relation to the confidence intervals calculated via Monte Carlo to get a feel for whether it was a reasonable value.
Below is a chart of the minimum profit for a strategy I recently took live plotted out to the end of 2015, created using the method described above. Using the 50th percentile maximum drawdown shows that my strategy could still be underwater at the end of the year and still be well within its profit expectation. If I use the 90th percentile maximum drawdown to pull out of this strategy (that is, I want to be 90% sure that my strategy has failed), the chart shows that I should expect even less of my strategy.
example of DD percentiles One of the very useful aspects of this sort of analysis is that it serves to temper the expectation associated with taking a strategy live. Personally, I know that after all the research and development that goes into developing a strategy, it is all too easy to let oneself think that the money will just roll in as soon as you go live. This approach brings the developer back to reality in terms of profit expectations, but it will also provide the confidence to continue trading while you should. In short, it brings objectivity to the forefront, which of course is a cornerstone of any statistical or quantitative approach to the markets.
I like jcl’s Cold Blood Index a lot. In a nutshell, it describes the similarity of a current drawdown to the backtest drawdown profile by examining multiple windows of the backtest balance curve, with each window equal in length to the length of the drawdown of interest (probably the current drawdown if trading live, but this approach is also useful for considering hypothetical drawdowns prior to going live).
The Cold Blood Index is more meaningful than the approach I used, but it does suffer from a similar drawback – being a function of the balance curve in the backtest. If you accept that the balance curve obtained in the backtest is one possible outcome amongst an infinite number of possibilities due to the random nature of individual trade results, then the result obtained with the Cold Blood Index is likewise one possible outcome amongst many.
(As an aside, this condition can be validated by examining the autocorrelation of the backtest returns series with various lags using either Zorro, R or some other statistical tool. Nearly every strategy I’ve experimented with, and definitely the ones that I’ve taken live, have all shown statistically insignificant return autocorrelations).
The Cold Blood Index could be improved by incorporating the algorithm into a Monte Carlo routine that creates many unique balance curves by randomly sampling the backtested trade results, and running it separately on each curve. I was fortunate enough to receive advice from jcl on how to accomplish this using Zorro, and my implementation is reproduced below. Most of the code is jcl’s, I have simply added the loop in which the balance curve is shuffled before being subject to jcl’s Cold Blood Index algorithm. I also added code to store the Cold Blood Index value generated in each loop iteration in an array and to calculate the 5th and 95th percentile values.

To compare methods for pulling out of a strategy, I looked at the results obtained from my original method (Method 1), the Cold Blood Index calculated from the backtest balance curve (one-sample Cold Blood) and the 5th percentile Cold Blood Index calculated from 4,000 randomly sampled balance curves (resampled Cold Blood). These results are for a drawdown of 55 days duration that started immediately upon strategy initiation (these results are for a real strategy that I am currently trading live. 55 days equates to a trading time that extends to the end of the year from the date that I initiated it. I am curious to investigate what sort of drawdown I should endure at the end of the year).

Drawdown($) Method 1 %ile One-sample Cold Blood 5th%ile Resampled
Cold Blood
-60.18 50 6.8 5
-68.79 60 5.5 4
-76.28 70 4.9 3.5
-92.00 80 3 2.3
-111.85 90 1 1
-202.08 100 0 0

The results are interesting. The additional accuracy provided by resampling the balance curve appears to be worthwhile. For example, if my drawdown was $60, I would pull out based on this method (assume that I pull out once the 5th percentile of the Cold Blood Index hits 5% or less.). Using the one-sample Cold Blood Index, I would stay in until my drawdown was more than $78. Using method 1, I may choose to stay in until almost $112 was lost.
The capital allocated to this strategy was only $500, so the resampled Cold Blood Index saved me 3% of my capital over the one-sample Cold Blood Index and over 10% of my capital over the method I was originally using.

Conclusions

  1. If we are committed to a quantitative approach to trading, an objective measure of strategy performance is required in order to decide whether to pull out or continue through a drawdown.
  2. The Cold Blood Index is a significant improvement over existing techniques such as modelling acceptable drawdown as a function of backtest performance measures.
  3. The resampled Cold Blood Index provides additional accuracy by considering the random nature of individual trade results and empirically estimating its true value from a large number of samples. This method would break down in the case of serially correlated returns, but otherwise has the potential to preserve more of the trader’s capital than other methods.

(5) Comments

mariano
November 1, 2015 at 10:37 pm

awesome work!
I was wondering it the cold blood Index could be used as well in the backtest somehow. I mean, some part of the data is used for backtesting and the last part of it is used as “live trading” . In that way the cold blood Index would check the balance curve of the backtest until that point that we consider the backtest as live trading.
The point would be reduce the DD. It could be used as equity curve trading. It is just an idea, not sure if this make sense at all since the CBI is trying to tell us if the strategy is working in real trading. And in this case it would be used to improve the strategy

November 2, 2015 at 10:08 pm

Hey Mariano, thanks for reading!
I think you could use the CBI in your backtest. The application that springs to mind is to apply it to your out of sample test and compare the results with the in-sample test. Ideally you would see some level of stability in the CBI across the training and test sets if your model wasn’t over-fit. Some level of degradation is probably unavoidable, but a model that is neither under-fit nor over-fit would show comparable results. Of course, degradation could also be a result of the strategy simply expiring during your out of sample test. Either way, it would be a sign that the strategy wasn’t quite ready to go live.
Of course there are numerous other performance metrics to compare across the training and test set results, but I see no reason why the CBI shouldn’t be added to the mix. The additional insight gained would be valuable.

mariano
November 3, 2015 at 3:37 am

I see it more clear now. Thanks for the nice explanation and congratulations for the blog!
I´ll look forward to read more of your researchs.

[…] The Financial Hacker s Cold Blood Index [Robot Wealth] This post builds on work done by jcl over at his blog, The Financial Hacker. He proposes the Cold Blood Index as a means of objectively deciding whether to continue trading a system through a drawdown. I was recently looking for a solution like this and actually settled on a modification of jcls second example, where an allowance is made for the drawdown to grow with time. The modification I […]

November 18, 2015 at 4:44 am

[…] The Financial Hacker’s Cold Blood Index [Robot Wealth] […]

Leave a Comment