Torturing Historical Market Data

I received a few comments from some statistical-minded readers on my post about what happens to stocks after periods of above average performance. Because I used some rolling average return figures I was told I was looking at the data the wrong way.

What they failed to realize is that there is no right or wrong way to look at historical data, only your interpretation of it. Here are some other random thoughts on using historical market data when making investment decisions:

There’s no such thing as right or wrong data, just better or worse. Stock market data looks spotless when you just see the performance numbers, but looks can be deceiving. There’s no way one can expect historical data to be all encompassing going back 100-200 years when it comes to earnings numbers, stock-splits, dividends paid or even the inclusion of all companies in an index (such as small caps or failed companies).

Most investors run back-tests under the assumption that the information would have been available to act on at the time. That’s simply not the case. Don’t you think investors would have made different decisions 50 or 60 years ago if they had the amount of data and computing power we have today? Saying you could have killed the market based on your back-test doesn’t hold up if you’re not using only data that existed at that time. The old saying is that if you torture the data long enough, it will admit to anything seems to apply here.

Data is useless without common sense. Some of my favorite quants that I read on a regular basis are Jim O’Shaughnessy, Wes Gray, Meb Faber and Patrick O’Shaughnessy. While I appreciate their quantitative research, what really stands out to me about each of their approaches is the fact that they acknowledge the behavioral side of the equation, as well. Numbers don’t mean anything if you can’t recognize their possible limitations or the fact that investors don’t always utilize them correctly.

Yeah but… Market data always comes with caveats. Show me a return series or valuation metric based on past data and I’ll find you someone that can play devil’s advocate: This data is skewed by the tech bubble. What if you take out the financial crisis? Yeah, but what about the 1970s? What if you lengthen the time horizon? What about Japan? 

If we can’t agree on the past how are we ever going to agree about the future? Sometimes I wonder why there are so many intelligent investors out there that fail. The easiest explanation is that people can’t agree on anything. If investors can’t even agree on how to interpret historical data, there’s no way we’re all going to agree about what’s going to happen in the future. Context often determines investor attitudes and opinions. It’s why some succeed while others fail.

It’s never going to be easy. Even if we had 5,000 years of data it still wouldn’t be a large enough sample size because investors would continue to make their own biased judgments of what that past data means to them. Every cycle is unique, so using historical market data works better as a way of defining risks, not as a way of knowing exactly what’s going to happen next.

Having a firm grasp on financial history is essential for investors, but the interpretation of that data is where things always get tricky.

Further Reading:
On the merits of being a financial historian

Subscribe to receive email updates and my monthly newsletter by clicking here.

Follow me on Twitter: @awealthofcs

Full Disclosure: Nothing on this site should ever be considered to be advice, research or an invitation to buy or sell any securities, please see my Terms & Conditions page for a full disclaimer.