In my 36 Obvious Investment Truths, number 24 stated:
24. Most backtests work better on a spreadsheet than in the real world because of competition, taxes, transaction costs and the fact that you can’t backtest your emotions.
Anyone in the investment business knows there’s no such thing as a bad backtest (although the recycle bin is filled with deleted Excel files that didn’t work quite as well).
The fact that more investors are utilizing a quantitative approach using the weight of historical evidence is a step in the right direction. What most investors passed off as alpha or outperformance in the past can now be explained by different risk factors or quantitative screens.
The active management fund industry isn’t a huge fan of the fact this is getting out. Now that the flood gates are open things like smart beta funds and their ilk are becoming de facto active management for many investors.
There are some downsides to the move to quant for investors who don’t understand how all of this works:
- Backtesting will always invite data-mining, over-optimization and the assumption of correlation where there is no causation.
- In an effort to fight the last war many investors create quant models to show you exactly how to invest in the previous market environment. Rarely do they have models for the next one.
- The fund industry knows which buttons to push to get you to invest. Typically that involves marketing funds with the best track records. That will now shift to include new funds with the best backtested results, as well.
- If you torture the data enough it will tell you exactly how to beat the market in the past. The future is a different story.
Research Affiliates, themselves a quantitative fund manager, recently performed some research that shows how difficult it can be to translate backtested results into successful real time performance. They showed that the outperformance that’s shown in the typical smart beta backtest virtually disappears once it’s translated into a live fund.
They looked at 125 different smart beta indices that various ETFs are based on. Here are their findings:
We find that prior to launch the indices tend to have superior performance relative to a market-capitalization-weighted benchmark, with outperformance peaking about six months ahead of the launch date. The outperformance seems to be extremely strong over the three-year period ahead of the launch. After the indices officially launch, however, their performance relative to the S&P 500 Index appears to hover around the base line, exhibiting virtually none of the outperformance demonstrated before they were live.
The visual does a nice job showing the before and after:
Size is the enemy of outperformance but so is popularity. Once investments become popular you always get the weak hands entering the fray and those investors typically follow a last-in-first-out selling strategy. There are few barriers to entry in the markets in terms of investment strategies these days so it’s harder than ever to keep a long-term edge over the competition.
While part of this comes down to a classic performance chase by fund management companies (they know what sells), there are a few other considerations when thinking about translating backtested results into a live portfolio:
Mean reversion. Nothing works always and forever. Above average performance is bound to even out eventually because of competition or the cyclical nature of the markets. Cycles are a fact of life in the markets. What makes the investor’s job difficult is the fact that those cycles don’t operate on a set schedule.
The sales cycle. Very few investors are in search of solid backtested results that have looked terrible lately. Recent outperformance is more comforting than looking for a diamond in the rough that has underperformed. One reason investors chase performance is that it feels like a safer bet. Underperformance is a tough sell.
Understand the ‘why.’ Every strategy needs a basis in reality for why it should continue to work beyond a good looking backtest. Investors spend plenty of time thinking through the ‘what?’ when putting together a portfolio but very little time on the ‘why?’. What deals more with tactics while why is more philosophical.
The biggest problem for investors is never finding a good backtest. That’s the easy part. The hard part is passing the front test by understanding your parameters, filters, and guidelines enough to be able to implement it in the real world and stick with it when it’s out of favor.
As Michael Batnick likes to say, “The worst 10 year period of any backtest is the next 10 years.”
If and when that happens it’s not your backtesting skills that come into question but your mental fortitude. In a backtest you know exactly how long each period of poor performance lasts. No one knows how long these things will last outside of the backtest.
Translating investment research into the real world is challenging because you can’t model human psychology.
Source:
Live from Newport Beach. It’s Smart Beta! (Research Affiliates)
Further Reading:
10 Things You Can’t Learn From a Backtest
*******
