The Tyranny of Benchmarking

A number of years ago the firm I worked for went through a lengthy review process of the compensation policies for our investment team. Management was trying to make sure our pay was aligned with industry averages but specifically, they were overhauling our bonus pool.

A compensation consultant was brought in. The process took a few months of back and forth. Finally, a plan was hatched.

I wasn’t a huge fan of the solution. Our bonuses were tied to investment performance but the benchmarks used left much to be desired. In this instance, we were given a handful of investment performance related benchmarks, the majority of which were of the risk-adjusted variety.

I’m not a huge fan of risk-adjusted return measurements. There’s a time and place for everything, but my experience working with money managers and asset allocators alike is that these measures are often used to confuse the end investor or move the goalposts after underperformance occurs. We used a subset of risk-adjusted return measures that, in my mind, were low hurdles to beat.

On the one hand, this was a good thing for me because it meant it was easier for me to get a larger bonus. On the other hand, setting up our bonus pool in this manner had to have an impact on the way people thought about running the portfolio.

My biggest problem with this compensation system is that it really had nothing to do with the company’s overall mission. I’ve never been a fan of using index or risk-adjusted benchmarks as a way to judge a portfolio’s performance because the only true sign of financial success is whether or not you’re on track to achieve your stated goals.

Incentives drive the world in many ways so getting the right compensation system in place can have a huge impact on people’s actions. Measurement can help an organization better achieve its goals but it can also lead to unintended consequences. Performance bonuses or fees are typically seen as aligning interests of all parties involved but there are downsides when the wrong measures are used.

There was a story this week about Amazon warehouse employees in the U.K. who allegedly felt so pressured to perform their duties that they often peed in bottles instead of going to the bathroom because they were so worried about losing their jobs. I’m guessing these employees had certain quotas to fulfill.

On this week’s EconTalk podcast, Russ Roberts interviewed Jerry Muller about his book, The Tyranny of MetricsMuller laid out a number of examples that shows how setting the wrong type of measurement systems can lead to suboptimal results.

When test scores were dropping for young people in school, the government mandated that school systems needed a certain amount of children to pass standardized tests to meet minimum requirements. This led many schools to simply focus on helping the students get better at standardized tests and practicing for them more often, whether that actually helped them learn more or not.

Muller also discussed how politicians often promise they will clean up crime in certain cities to get (or stay) elected. Promotions for higher-ups within police departments are often attached to major crime statistics, so there are incentives from a number of parties to see these stats fall. It’s really difficult to cut down on actual crimes being committed but they can cut down on the official reporting of crimes by gaming the numbers (I believe my first experience with this was watching The Wire). Studies have shown the simplest way to cut down on major crime figures is to classify felonies as misdemeanors or focus more attention on easier to solve cases.

This example from Muller was even crazier:

Some years ago the national health service in Britain had a lot of complaints about how waiting times to be admitted to the hospitals were too long. So they declared that hospitals would be penalized if the waiting time to be admitted to the hospital was four hours or more. So some of the hospitals did the following: when they had patients coming in by ambulance and they knew that the wait was going to be more than four hours, they would have the ambulance circle around the hospital until they could admit the patients within four hours. Which sounds kind of amusing at first until you think about the fact that there were then patients sitting at home waiting to get picked up by those ambulances.

Obviously, benchmarks can be helpful in terms of setting expectations, opening the lines of communication, and figuring out where potential problem areas lie. But too often these days many organizations and individuals are so tethered to their stated benchmarks that they’ve forgotten about the judgment and common sense aspects involved in measuring performance.

Performance isn’t always about providing answers. In most cases, performance measurement can help identify the right questions to ask so you understand whether you’re on the right path or need to make a course correction. You also must be able to look past the actual end results and try to understand what’s contributing or detracting from your performance. Performance attribution can be more telling than the performance itself if you know where to look.

Unfortunately, a large number of investment organizations, funds, and investors either pay too much attention to benchmarks (think closet indexing) or not enough (think ridiculously low hurdle rates or unsuitable benchmarks).

Benchmarking can be useful when it’s set up with good intentions, realistic expectations, and a system of checks and balances to ensure it’s incentivizing the desired behavior, not just a set of outcomes.

But relying exclusively on benchmarks in lieu of good old-fashioned judgment and common sense can be a mistake if you don’t understand the unintended consequences.

Further Reading:
The Unintended Consequences of Innovation

Now here’s what I’ve been reading this week:

 

This content, which contains security-related opinions and/or information, is provided for informational purposes only and should not be relied upon in any manner as professional advice, or an endorsement of any practices, products or services. There can be no guarantees or assurances that the views expressed here will be applicable for any particular facts or circumstances, and should not be relied upon in any manner. You should consult your own advisers as to legal, business, tax, and other related matters concerning any investment.

The commentary in this “post” (including any related blog, podcasts, videos, and social media) reflects the personal opinions, viewpoints, and analyses of the Ritholtz Wealth Management employees providing such comments, and should not be regarded the views of Ritholtz Wealth Management LLC. or its respective affiliates or as a description of advisory services provided by Ritholtz Wealth Management or performance returns of any Ritholtz Wealth Management Investments client.

References to any securities or digital assets, or performance data, are for illustrative purposes only and do not constitute an investment recommendation or offer to provide investment advisory services. Charts and graphs provided within are for informational purposes solely and should not be relied upon when making any investment decision. Past performance is not indicative of future results. The content speaks only as of the date indicated. Any projections, estimates, forecasts, targets, prospects, and/or opinions expressed in these materials are subject to change without notice and may differ or be contrary to opinions expressed by others.

The Compound Media, Inc., an affiliate of Ritholtz Wealth Management, receives payment from various entities for advertisements in affiliated podcasts, blogs and emails. Inclusion of such advertisements does not constitute or imply endorsement, sponsorship or recommendation thereof, or any affiliation therewith, by the Content Creator or by Ritholtz Wealth Management or any of its employees. Investments in securities involve the risk of loss. For additional advertisement disclaimers see here: https://www.ritholtzwealth.com/advertising-disclaimers

Please see disclosures here.