What Is the Second Generation of Behavioral Finance, and What Does It Teach Us About Market Efficiency?

Editor’s Note: Meir Statman is a longtime advisor to the Wealthfront investment team and a foremost expert in the field of behavioral finance. His new book, “Behavioral Finance: The Second Generation” is published by the CFA Institute Research Foundation and can be downloaded for free at this link.

My new book, “Behavioral Finance: The Second Generation,” presents the second generation of behavioral finance. The first generation, starting in the early 1980s, largely accepted standard finance’s notion of people’s wants as “rational” wants—restricted to the utilitarian benefits of high returns and low risk. That first generation commonly described people as “irrational”—succumbing to cognitive and emotional errors and misled on their way to their rational wants. 

The second generation describes people as “normal.” It acknowledges the full range of people’s normal wants and their benefits (utilitarian, expressive, and emotional),  distinguishes normal wants from errors, and offers guidance on using shortcuts and avoiding errors on the way to satisfying normal wants. People’s normal wants include financial security, nurturing children and families, gaining high social status, and staying true to their values. People’s normal wants, even more than their cognitive and emotional shortcuts and errors, underlie answers to important questions in finance, including questions about saving and spending, portfolio construction, asset pricing, and market efficiency.

What does the second generation of behavioral finance teach us about market efficiency?

The efficient market hypothesis is at the center of standard finance and many believe that behavioral finance refutes it. Indeed, many believe that refutation of the efficient market hypothesis is behavioral finance’s most important contribution. This issue becomes confused, however, when people fail to distinguish between two versions of efficient markets and their corresponding efficient market hypotheses: the price-equals-value efficient market hypothesis and the hard-to-beat efficient market hypothesis. And it remains a mystery why so many investors believe that markets are easy to beat. 

Price-equals-value efficient markets are markets where investments’ prices always equal their intrinsic values, and the price-equals-value efficient market hypothesis is the claim that investment prices always equal their intrinsic values. The intrinsic value of an investment is the present value of its expected future payments, such as dividends, discounted by an expected return determined by a correct asset pricing model. 

Hard-to-beat efficient markets are markets wherein some investors are able to beat the market consistently, earning abnormal returns over time, but most are unable to do so. Abnormal returns are returns exceeding the returns one would expect based on a correct asset pricing model. 

Both standard and behavioral finance provide evidence refuting the price-equals-value efficient market hypothesis, but their evidence generally supports the hard-to-beat efficient market hypothesis. Behavioral finance also explains why so many investors believe that markets are easy to beat when, in fact, they are hard to beat.

Price-equals-value efficient markets are impossible to beat because abnormal returns come from exploiting discrepancies between prices and values. Such discrepancies are absent in price-equals value efficient markets. But hard-to-beat efficient markets are not necessarily price-equals-value efficient markets. It might be that substantial discrepancies between prices and values are common, implying markets are far from price-equals-value efficiency, but discrepancies are hard to identify in time or difficult to exploit for abnormal returns. As I often say, markets are crazy, but this does not make you a psychiatrist. 

Who beats a hard-to-beat market?

Hard-to-beat markets are not impossible to beat. Investors possessing exclusively or narrowly available information find it hard but not impossible to beat the market consistently. 

Investors possessing exclusively and narrowly available information are typically investment professionals. Yet typical investment amateurs — and many investment professionals — possess nothing more than widely available information, and thus find it impossible to beat the market consistently. Rather, investment amateurs are actually beaten by the market far more often than they beat it. Indeed, investors possessing exclusively available and narrowly available information gain their market-beating returns by emptying the pockets of investors who attempt to beat the market with widely available information alone.

Amateur investors who attempt to outperform the market can be do-it-yourself investors who buy an undiversified handful of stocks and trade them frequently or they can hire professional investors, such as managers of active mutual funds, to invest for them. Many fund managers possess narrowly available information and some may have exclusively available information, that is, information beyond the widely available, enabling them to generate abnormal returns. Yet investors in active funds are not assured of abnormal returns unless fund managers share these abnormal returns with them.

Economists Jonathan Berk and Jules van Binsbergen found that active mutual fund managers do beat the market before fees, generating abnormal returns, but, on average, they do not share their abnormal returns with their investors. Managers of active mutual funds generated an average annual $3.2 million of abnormal returns in each fund, yet they kept the entire $3.2 million as fees, delivering to their investors returns equal, on average, to returns earned by investors in passive index funds.

Others have concluded, however, that active fund managers’ fees exceed the above-market returns generated. That is, they deliver to their investors average returns lower than index fund returns. The economist Burton Malkiel (Wealthfront’s Chief Investment Officer) wrote that “Managed funds are regularly outperformed by broad index funds, with equivalent risk.”

Why do amateur investors persist in attempts to beat the market? 

Why do amateur investors, possessing nothing more than widely available information, persist in their attempts to beat the market as do-it-yourself investors or as investors in active mutual funds? Some of these investors are misled by cognitive and emotional errors. Others follow their wants.

Framing errors are prominent among cognitive errors. Amateur investors regularly frame trading by analogy to surgery, driving, cabinetmaking, or plumbing. A plumber improves her work as she does more of it, fixing more leaks. Framed by analogy, an investor improves his performance as he trades more often, increasing his profits. Framing trading by analogy to plumbing is flawed, however, because pipes and fittings do not compete against the plumber, inducing her to choose the wrong fitting. But a trader always faces a competing trader on the other side of his trade, sometimes inducing him to choose the wrong trading strategy. 

Active traders possessing only widely available information often commit another framing error: framing their returns relative to zero rather than relative to the market return (the return they could have earned by investing in a low-cost diversified index fund). A 15% annual return sounds excellent, but it is inferior when an index fund delivers 20%. 

Moreover, amateur investors tend to form general impressions of their returns rather than calculate them. This leads them into the trap of confirmation errors, whereby they count gains that confirm their image as winners, while overlooking losses that disconfirm that image. Indeed, a study of amateur investors, members of the American Association of Individual Investors (AAII), found that they overestimated their own investment returns by an average of 3.4% relative to their actual returns, and they overestimated their own returns relative to those of the average investor by 5.1%.

Some amateur investors commit representativeness errors known as the belief in the law of small numbers. A sample of 10 might well be sufficient for general conclusions about the quality of meals at a particular restaurant. This is because there is relatively little randomness in the quality of meals at any particular restaurant. Seven bad meals out of 10, or even two bad meals out of three, might well be all we need for a general conclusion that it is best to forego that restaurant. Moreover, we lose little when we forego that particular restaurant when, in truth, its meals are generally good. There are hundreds more restaurants to explore.   

Yet a sample of 10 trades, or even 100, is too small to reach general conclusions about the quality of a trader or the validity of a trading strategy. This is because there is much randomness in trading outcomes. A lucky trader possessing only widely available information might enjoy a streak of wins when trading against traders possessing only widely available information. That lucky trader might even win sometimes when trading against traders possessing exclusively or narrowly available information. And unlike the case of restaurants, amateur traders are likely to forego substantial amounts when they conclude, in error, that they can beat the market.

Amateur investors are also susceptible to availability errors, whereby they form estimates and opinions based on easily available information. Social interactions make information easily available, misleading investors into trading when fellow investors disclose their winning trades, but withhold their losing ones.¹ And available fake news misleads investors who misperceive it as truthful. A study of stock promotions employing fake news articles revealed that many are parts of “pump-and-dump” schemes operated by corporate insiders who profit by trading with investors ignorant of the falsity of the news.

Many amateur investors are drawn into active investing by cognitive errors, but some are also drawn into it by wants for expressive and emotional benefits. 

Wants for expressive and emotional benefits are evident among Dutch investors who agreed with the statement “I invest because I like to analyze problems, look for new constructions, and learn” and the statement “I invest because it is a nice free-time activity” more than they agreed with the statement “I invest because I want to safeguard my retirement.” These wants are evident among German investors who enjoy investing who traded twice as much as other investors.  And these wants are evident among the quarter of American investors who said that they buy stocks as a hobby or because it is something they like to do. 

A Fidelity survey of amateur traders revealed that 54% enjoy “the thrill of the hunt,” 53% enjoy learning new investment skills, and more than half enjoy sharing trading news with family and friends.“ This research confirms the obvious satisfaction traders receive when generating cash from their activities, but it also highlights their desire to learn new skills and to share, teach, and mentor others,” said James C. Burton, president of Fidelity’s retail brokerage business.

Conclusion

Commercials by investment companies, like commercials by vitamin companies, cater to wants for winning or, at least, for being above average, whether in running or investing. The announcer in a commercial for vitamin stores denigrates “average” athletic ability. The crowning achievement of average, he says in disdain, is “everyone gets a trophy.” It shows a boy in bed, whispering in his dream, “Average is good…” “No,” says the announcer, “Average is average. You can beat it.”

A commercial by an investment company denigrates index funds as average. It concludes with a man standing on a stage as a sign he created lights up: “Why invest in average?” And an advertisement by another investment company asks “If passive investing was called ‘Don’t Try,’ would you still be interested?”

Yet when it comes to investing, average is pretty good. Amateur investors can choose between average returns by passively investing, matching the market by well-diversified low-cost index funds of ETFs, or below-average returns by actively investing, attempting to beat the market. Average is pretty good when the alternative is below average. 

This post is adapted from Meir Statman’s “Behavioral Finance: The Second Generation.”

¹ Han, Bing and Hirshleifer, David A. and Walden, Johan, Social Transmission Bias and Investor Behavior (January 17, 2018). Rotman School of Management Working Paper No. 3053655. Available at SSRN: https://ssrn.com/abstract=3053655 or http://dx.doi.org/10.2139/ssrn.3053655