The other day, a wealth advisor colleague called to relate his conversation with a prospective client who had questioned the academic basis of our evidence-based investment philosophy.

Specifically, she doubted the validity of the statement that most active managers persistently underperform their risk-adjusted benchmark. Her response was: “Of course they outperform, that’s what they get paid for. If they didn’t, no one would pay them.”

From her perspective, it might have seemed painfully obvious to believe this was the case. After all, investors are rational and would not pay for poor performance. Yet the evidence shows she could not have been more wrong.

In 1998, when Charles Ellis’s book, “Winning the Loser’s Game,” was first published, only about 20% of actively managed funds were generating statistically significant alphas. About 80% were not. That’s why Ellis called active management a loser’s game—a game that’s possible to win, but with odds so poor that it’s not prudent to try.

Thus, like all loser’s games (such as buying lottery tickets or playing slot machines), the surest way to win is to not play. With investing, that means using passively managed vehicles, such as index funds.

If the poor odds that Ellis describes aren’t convincing enough, as my co-author Andrew Berkin and I demonstrate in our book, “The Incredible Shrinking Alpha,” today the percentage of actively managed funds generating statistically significant alpha is less than 2%. We explain why this trend, in which active managers experience a persistently declining ability to generate alpha (that is, to outperform risk-adjusted benchmarks), is virtually certain to continue.

That, in turn, raises an interesting question: Why did this prospective client possess such a strong belief that active managers actually outperformed?

In Love With A Loser’s Game

One of the greatest frustrations for me, and one of the great anomalies in finance, is that, given the overwhelming amount of evidence against active management and in favor of passive investing, a majority of investors keep playing a loser’s game. I offer four explanations for this phenomenon.

First, the education system has totally failed the public. Unless they obtain an MBA in finance, it’s highly unlikely that investors have taken a single course in capital markets theory. Without the appropriate knowledge, how can investors determine whether an active or a passive strategy is the right one?

Unfortunately, many obtain their “knowledge” about investing from the very institutions—Wall Street and the financial media—that don’t have their interests at heart, because the winning strategy for them is when investors play the game of active investing.

Wall Street needs investors to trade frequently and to pay the high fees imposed by actively managed funds. The media needs investors to “tune in.” The result is that most investors are unaware of the historical evidence.

Getting Properly Educated

Second, despite the importance of the issue, the public seems unwilling to invest the time and effort to overcome the failings of the education system. Instead of exploring the appropriate resources to become informed, such as books by William Bernstein, John Bogle or myself that lay out the historical evidence on active versus passive management, they would rather watch some reality TV show or cable financial news to hear the latest guru’s forecast.

There’s an all-too-human need to believe there is someone out there who can predict the future and thus protect us from uncertainty.

The third explanation is what we might call the Lake Wobegon effect—the need and/or desire to be above average. In my long experience, this seems especially true of certain high net worth investors.

They want to feel special, particularly when it comes to investments. They want to be members of the “in crowd,” with access to products to which the hoi polloi does not. That can offer a feeling of prestige and sophistication. In other words, they want more than just returns from their investments.

Wall Street plays on that need. You hear the repeated lie that goes something like this: “Indexing is a good strategy, but it gets you average returns. You don’t want to be average. We can help you do better than that.”

The truth is that indexing doesn’t get you average returns; it gets you market returns. And because it does so with lower costs and greater tax efficiency, by definition, you earn above-average returns—as long as you maintain the discipline to stay the course. This is about the only guarantee there is in investing.

Why Do Investors Ignore The Evidence?

Why do so many investors continue to play a game that has such poor odds of winning? Kathryn Schulz, author of “Being Wrong,” provides us with some fascinating insights to help explain this phenomenon, one that allows Wall Street to transfer tens of billions of dollars every year from investors’ pockets to their own.

She explains that most of us go through life assuming we are “basically right, basically all the time, about basically everything” and that “our indiscriminate enjoyment of being right is matched by an almost equally indiscriminate feeling that we are right.” Schulz then continues: “Occasionally, this feeling spills into the foreground as we make predictions or place bets” (or make investments).

She explains that, often, this confidence is justified as we “navigate day-to-day life fairly well.” This suggests that we’re right about most things. Schulz’s book is all about the opposite of that. It’s about being wrong and what happens when our convictions collapse around us. When that happens, we often feel foolish and ashamed, as error is often associated with “ignorance, psychopathy and even moral degeneracy.”

Schulz’s observations can help us understand why that prospective client believed what she did—it would be too painful to her self-image to admit her ignorance and mistaken belief.

Mistakes Happen To Other People

Schulz noted that we tend to view mistakes as things that happen to others. Yet we feel it is implausible they’ll happen to us. She believes this is because “our beliefs are inextricable from our identities” and because “we’re so emotionally invested in our beliefs that we are unable or unwilling to recognize them as anything but the inviolate truth.”

She also notes that “we tend to fall in love with our beliefs once we have formed them” (such as the belief that active management is the winner’s game), and that explains why “being wrong can so easily wound our sense of self.” It explains why we experience cognitive dissonance—the uncomfortable feeling and/or anxiety we feel when someone disproves a long-held belief. It also explains why we ignore evidence, even when it is compelling, and why we resist change.

Schulz writes that this view—that others make errors but not us—is the greatest error of them all, as “wrongness” is a vital part of how we learn and change.

In other words, even smart people make mistakes—for instance, I used to be an active investor. However, once smart people learn that a behavior is a mistake, they revise their ideas and change their ways. This is what separates them from fools, who keep repeating the same mistakes while expecting different outcomes.

Among the many insights Schulz offers is that our ability to forget our mistakes is keener than our ability to remember them. During her research, Schulz met a lot of people who told her that she should interview them, as they make mistakes all the time. Yet when asked to provide a specific example of their mistakes, they were hard-pressed to come up with any. The inability to remember mistakes leads to overconfidence, which in turn can lead to other mistakes, especially investment mistakes—such as taking too much risk and failing to diversify—which can be very expensive.

Yet Another Explanation

The fourth and final explanation for the anomaly that most individual investors use actively managed funds comes from social psychologists Carol Tavris and Elliot Aronson, authors of the wonderful book, “Mistakes Were Made (But Not by Me).”

They write: “Most people, when directly confronted with proof that they are wrong, don’t change their point of view or course of action but justify it even more tenaciously. Politicians, of course, offer the most visible, and often tragic, examples of this practice. … We even stay in an unhappy relationship or merely one that is going nowhere because, after all, we invested so much time it making it work.”

Tavris and Aronson explain: “Self-justification has costs and benefits. By itself it’s not necessarily a bad thing. It lets us sleep at night. Without it we would prolong the awful pangs of embarrassment. We would torture ourselves with regret over the road not taken or over how badly we navigated the road we did take. We would agonize in the aftermath of almost every decision.… Yet mindless self-justification, like quicksand, can draw us deeper into disaster. It blocks our ability to even see our errors, let alone correct them. It distorts reality, keeping us from getting all the information we need and assessing issues clearly.”

Investors, both individual and institutional, who rely on the past performance of active managers, and rankings such as Morningstar’s star ratings, hire managers, eventually firing most of them, and then repeat the process.

They do so without ever asking themselves: What should I do differently in the selection process so I don’t repeat the mistake I made last time? In a triumph of self-justification, they end up behaving in a way that has been described as the definition of insanity—they repeat the same thing over and over again and expect a different outcome.

Tavris and Aronson note that “none of us can live without making blunders. But we do have the ability to say: ‘This is not working out here. This is not making sense.’ To err is human, but humans then have a choice between covering up or fessing up. The choice is crucial to what we do next. We are forever being told that we should learn from our mistakes, but how can we learn unless we first admit we make any?”

Admitting Mistakes Can Be Freeing

Unfortunately, admitting mistakes is a difficult hurdle for many. A great example of this behavioral problem is a clever insight from Lord Molson (a 20th century British politician) into his own behavior: “I will look at any additional evidence to confirm the opinion to which I have already come.” This leads to the well-documented problem of confirmation bias, which we have already discussed.

Perhaps the saddest part is that we miss out on just how freeing it can be to admit a mistake. Tavris and Aronson told a story about a friend who was sent to traffic court and heard excuse after excuse for running red lights or making illegal U-turns. This friend became so fed up with it, he stood in front of the judge when it was his turn and said, “I didn’t stop at a stop sign. I was entirely wrong and I got caught.” The entire room burst into applause.

In 1994, only about 3% of individual investor inflows into mutual funds went to index funds. By 1999, as more individual investors became aware of the evidence on the poor performance of actively managed funds, that figure had reached almost 40%.

Today the net figure (funds going into passively managed funds and out of active ones) is more than 100%. Actively managed funds are no longer just losing market share, as they have been for about 25 years, they are now losing net assets as well.

For example, in March 2017, $31.1 billion flowed into passive equity strategies, while investors pulled $18.6 billion out of actively managed equity. Most active funds are dead men walking; they just don’t know it yet.

Conclusion

If you’ve hired and fired fund managers and/or advisors who advocate active strategies, or have been using active strategies on your own and have underperformed appropriate risk-adjusted benchmarks, perhaps it’s time you fessed up and started playing the winner’s game that is passive investing.

By doing so, it’s highly likely you will increase your odds of achieving your financial goals—assuming you also have the discipline (like Warren Buffett) to stay the course, adhering to your well-thought-out plan (assuming, of course, you have one).

In fact, in his 1996 Berkshire Hathaway annual report, Buffett himself advised investors: “Most investors, both institutional and individual, will find that the best way to own common stocks is through an index fund that charges minimal fees. Those following this path are sure to beat the net results (after fees and expenses) delivered by the great majority of investment professionals.”

Sadly, not enough investors have heeded his advice. But it’s never too late.

By Larry Swedroe, Director of Research, The BAM Alliance

This commentary originally appeared September 1 on ETF.com