Backtesting: The method to check your strategy

• Backtesting is a simulation of the past.
• You take the data from the past and use it to predict what would have happened if you had used a certain trading strategy.
• A trading system is a set of rules that tells you when to buy and sell.
• A backtest is a way to see if the system would have made money in the past.
• You do this by pretending you had the system before and using it to buy and sell stocks.
• If it made money, then it’s probably a good system.

Backtesting is a simulation of the past. You take the data from the past and use it to predict what would have happened if you had used a certain trading strategy.

It’s like I am running a race and I want to know how fast I would have run if I had run it yesterday. The problem with backtesting is that it can help justify just about any strategy. 

Forb example, let’s say that we are trying to figure out if a certain strategy would have worked during the housing bubble of 2005-2007.  If we use backtesting to do this, we will find that it worked extremely well.  We can then say, “See! This strategy works! I can make money with it!”. 

A trading system is a set of rules that tells you when to buy and sell. A backtest is a way to see if the system would have made money in the past. You do this by pretending you had the system before and using it to buy and sell stocks. If it made money, then it’s probably a good system.

If you’re going to make a model, you have to test it out first, to see if it works.

What is backtesting?

A backtest is when you try out a strategy with past data to see if it will make money. You do this by putting fake money in your account and pretending to trade with it. You can then see if your strategy made money or lost money. If you want to make a real account, you can do that too, but it is more risky because you could lose real money.

You make a fake history, and see how your strategy would have performed in that fake history. The concept of backtesting is used in a variety of disciplines and for a variety of purposes. The term is also used to refer to the simulation of strategies that are intended to be used in the future, although this usage is not universal. The concept of backtesting applies in fields such as physics, biology, computer science, finance, and marketing.

In physics, for example, backtesting helps determine the validity of physical models. For biology, it determines the validity of biological models such as genetic algorithms. In finance, traders use it to determine the viability of a financial strategy. With marketing, it helps determine the viability of a marketing campaign. 

For finance and economics fields, backtesting is commonly used in portfolio theory to test whether a proposed portfolio has desirable properties such as being risk-efficient. It can also be used to test whether an existing portfolio is efficient. It can also test whether a financial instrument will have desirable properties under certain conditions. For example, some traders use it to test whether a particular derivative will be profitable under certain market conditions.

If you can define it, you can backtest it. 
As long as you can backtest it, you can trade it. 
If you can trade it, you can make money from it.

This course is clearly the best one to get your backtesting skills up and running : Backtesting 101.
Check it out if you’d like to level up your backtesting (and trading) skills.

How to proper backtest?

The backtest must use data from a relevant time period of a duration that reflects a variety of market conditions. In this way, one can better judge whether the results of the backtest represent a fluke or sound trading. 

The historical data set must include a truly representative sample of stocks, including those of companies which eventually went bankrupt or were sold or liquidated. The alternative, including only data from historical stocks that are still around today, will produce artificially high returns in backtesting. 

The backtest should consider all trading costs, however insignificant, as these can add up over the course of the backtesting period and drastically affect the appearance of a strategy’s profitability. Traders should ensure that their backtesting software accounts for these costs. 

Out-of-sample testing and forward performance testing provide further confirmation regarding a system’s effectiveness and can show a system’s true colors before real cash is on the line. Good correlation between backtesting, out-of-sample and forward performance testing results is vital for determining the viability of a trading system. To understand why the statement is true, one must understand what it means to “backtest” a strategy. 

The goal of backtesting is to see how well a strategy would have performed in the past.

10 rules for backtesting trading strategies

1. Don’t test a strategy developed during a bear market in a bull market.
2. Don’t test a strategy developed with a small universe in a large universe. 
3. Never test a strategy developed with low volatility in high volatility. 
4. Don’t test a strategy developed with a low average number of bars held in a high average number of bars held.
5. Don’t test a strategy developed with high exposure in low exposure. 
6. Avoid testing a strategy developed with high average gains and low average losses in low average gains and high average losses.
7. Don’t test a strategy developed with high annualized return in low annualized return. 
8. Don’t test a strategy developed with custom commission amounts, round lot sizes, tick sizes, margin requirements, interest rates, slippage assumptions, position-sizing rules, same-bar exit rules, trailing stop settings and much more in standard settings used by most other people who trade the same instruments you trade and use the same broker you use.
9. When you over-optimize your strategy, it can become very hard to understand. This means that if you need to change it, or if you want to try something different, it can be very hard to understand what you are doing.
10. Sometimes, the market changes, and what used to work before doesn’t work anymore. If you want to be a successful trader, you have to adapt to the changing market conditions. You can’t just rely on past performance of a system. Paper trading is the only way to know if a system works or not.

Differences between backtesting and forward testing

When you have a system that makes money, but you are not trading it, you are not making money. When you are not making money, you should trade your system. When you are trading your system, and it is losing money, you should stop trading it. If your system is losing money when you are trading it, stop trading it. If your system is making money when you are trading it, keep trading it.

You need to follow the system’s logic exactly, or you won’t know if the system is any good. It’s like baking a cake. If you put too much of one ingredient, or too little of another, it won’t work. You need to measure everything exactly, or you’ll get a bad cake.

Drawbacks of backtesting

Better understand the risks

“You have to use data that you did not use to build your strategy, so that you are not cheating. Otherwise, you will be biased and the results will be worthless”. “I got it” he said. I was stunned. He understood it immediately.
I told him that he had just learned one of the most important lessons in trading, and he should remember it for the rest of his life. It’s not that difficult to explain technical concepts in plain language, if you really try.  This is what I have been trying to do for the past two years. I am just not sure I am doing a good job of it. I hope this post will help others to learn from my mistakes.

If you’re testing your strategy with data you’ve used to develop it, then you’re using data that has a bias, and the results of your backtest will be biased. The same goes for using the same data set for both training and testing. The only way to avoid this bias is to use a different set of data to test your strategy than the one you used to develop it. To test this, I developed a simple strategy and back-tested it on several different data sets. I then compared the results from each of these tests to see if they were similar or not. They were all similar, which means my strategy works regardless of what data I use to test it. This is a good sign that my strategy is likely to work in real-time markets.

It depends on the data you choose

When you backtest a strategy, you test it with data used to create the strategy. So if you backtest a strategy that uses the last 10 years of price data, you are using data from the last 10 years to test a strategy that uses the last 10 years of price data. This means your backtest will always look good.

If you want to make sure your strategy is actually good, you have to test it with data from a different time period than the one you used to create it. For example, if you use the last 10 years of price data to create a strategy, then test it with data from the 20th century, it will probably look bad. But if you test it with data from the 21st century, it will probably look good. If your backtests are similar in both time periods, then they are probably valid.

Conclusion

Backtesting is a way to test a strategy, by looking at the past. The results of the backtest will show you how your strategy would have performed in the past, under certain conditions. It can help you see if your strategy has potential or not.

To go into more details about how to do a proper backtest, you should check:

1. Our article about the best backtesting platforms
2. The best course available right now about backtesting : Backtesting 101
   Created by Steve Burns from NewTraderU (more than 380 000 followers on Twitter so we knows what he does), this 18-part course covers all you should know to kickstart your backtesting trading journey. From data mining to beating “Buy and Hold”, it also covers emotions & stress, selection bias, market constituents, curve fitting and many more. Don’t miss it!
   Click here to signup for this course today!

• portfoliovisualizer.com
• risk.net
• investinganswers.com
• quantinsti.com
• value-at-risk.net
• quantstart.com