*Algorithmic Trading*and realised that I hadn’t adequately accounted for randomness. Whenever we compute a performance metric from a backtest, we face the problem of a finite sample size. We can’t know the true value of the performance metric, and the value we computed may or may not be representative of this true value. We may have been simply fooled by randomness into thinking we had a profitable strategy. Put another way, was the strategy’s performance simply due to being in the market at the right time?

- Constructed random strategies that mirror the trade distribution and frequency of the original strategy
- Ran the random strategy 5,000 times and constructed a histogram of the profit factor
- Compared the profit factor of the strategy against the distribution of randomly obtained profit factors

//Randomly generate entries to match a strategy's simulation period, number of trades and trade duration #define ASSETLOOP while(asset(loop("GBP/JPY", "GBP/USD", "USD/CAD"))) #define TradesLong AlgoVar[0] #define TradesShort AlgoVar[1] function getOpt(string param) { switch (Asset) { case "GBP/JPY": switch (param) { case "strategyTradesLong" : return 114; case "strategyTradesShort" : return 0; case "strategyDuration" : return 4; } case "GBP/USD": switch (param) { case "strategyTradesLong" : return 128; case "strategyTradesShort" : return 0; case "strategyDuration" : return 4; } case "USD/CAD": switch (param) { case "strategyTradesLong" : return 0; case "strategyTradesShort" : return 113; case "strategyDuration" : return 4; } } } function tradeRandom() { var StrategyTradesLong = getOpt("strategyTradesLong"); //Number of trades in the actual strategy var StrategyTradesShort = getOpt("strategyTradesShort"); int StrategyDuration = getOpt("strategyDuration"); //Average trade length in the actual strategy strategy ExitTime = StrategyDuration; var NumOpen = NumOpenLong + NumOpenShort; if (random() > 0) { //prevents trades being taken on the same bars each run of the strategy if(NumOpen == 0 and TradesLong < StrategyTradesLong and random() > 0) { //may need to modify this line to get correct random trade frequency enterLong(); TradesLong += 1; } if (NumOpen == 0 and TradesShort < StrategyTradesShort and random() < 0) { //may need to modify this line to get correct random trade frequency enterShort(); TradesShort += 1; } } } function run() { set(LOGFILE); BarPeriod = 1440; StartDate = 2010; //Set start and end dates to match those from the acutal strategy's simulation EndDate = 2014; LookBack = 0; if (is(INITRUN)) TradesLong = 0; if (is(INITRUN)) TradesShort = 0; ASSETLOOP while(algo(loop("RAND"))) { if(Algo == "RAND") tradeRandom(); } }

I ran this strategy 5,000 times and extracted the profit factor of each run using this Unix script that I ran with Cygwin.

#!/bin/bash i=0 while [ $i -lt 5000 ] do cd "C:/~/Zorro" ./Zorro -run random_trades cd "C:/~/Zorro/Log" sed -n 56p random_trades.txt >> RandomTrades_GBPJPY.txt sed -n 57p random_trades.txt >> RandomTrades_GBPUSD.txt sed -n 58p random_trades.txt >> RandomTrades_USDCAD.txt i=`expr $i + 1` done

This feels like a crude approach, but it let me move fast, which counts for a lot. In order to use this on a random strategy with different assets, the lines extracted from the performance report would need to be modified accordingly. Zorro can also be programmed to run a certain number of times using the `NumTotalCycles`

parameter. The profit factor metric can then be calculated and stored in a histogram.

In the next post, I’ll explore some results obtained with this approach.

**References**

Chan, Ernest, *Algorithmic Trading*, 2013, John Wiley and Sons

Lo, Andrew, Mamyskey, Harry and Wang, Jiang, 2000, *Foundations of Technical Analysis: Computational Algorithms, Statistical Inference, and Empirical Implementation*, Journal of Finance, Volume IV, Number 4.

Hi Robot Master,

Nice post on the application of random portfolios for strategy evaluation.

In your example application, you say that you pick the optimal strategy from a range of strategies based on out of sample performance. If this is the case, then you need to compare the statistics of this optimal strategy to the histogram of the ‘optimal’ profit factor.

A simple way to do this is on each of your 5000 runs, have the same number of random strategies running as in your original portfolio of strategies from which you choose from. Then for each of your 5000 runs, pick the optimal random strategy and record its profit factor. This is the correct distribution to compare your actual optimal strategy’s profit factor against.

See David Aronson’s 2011 textbook on technical analysis or look up White’s (2000) Reality Check method (see Hansen (2005) for an improved version).

Thanks Emlyn! One limitation with the approach I documented above is that in order for it to work, one has to assume that no data mining bias has taken place. This would be almost impossible to achieve in reality.

If I understood you correctly, picking the optimal random strategy from each of the 5000 runs would be equivalent to Wihte’s bootstrap method? This seems like a much simpler implementation than keeping track of all the tested strategy variants and detrending and sampling from their equity curves, as is my understanding of White’s Reality Check.

Even if it is not precisely equivalent, I can see that your suggestion is a more robust benchmark to use than the one I wrote about. Thanks for sharing it!

It looks like it does not work. That the message from my Zorro platform:

‘tradeRandom’ undeclared identifier

.

A1 compiling………..

Error 055: GBP/JPY no 2010 history (History\GBPJPY.t6)

Error 055: No bars generated

That’s odd…the

`tradeRandom`

is defined before`main`

in the code. Maybe try declaring the function as`void`

rather than`function`

– perhaps the latter no longer works in recent Zorro version.