Why You Systematically Overestimate Your Knowledge and Abilities

Why You Systematically Overestimate Your Knowledge and Abilities: Overconfidence Effect

My (Rolf Dobelli) favorite musician, Johann Sebastian Bach, was anything but a one-hit wonder. He composed numerous works. How many there were I will reveal at the end of this chapter. But for now, here’s a small assignment: How many works do you think Bach composed? Choose a range, for example, between one hundred and five hundred, aiming for an estimate that is 98 percent correct and only 2 percent off.

How much confidence should we have in our own knowledge? Psychologists Howard Raiffa and Marc Alpert, wondering the same thing, have interviewed hundreds of people in this way. Sometimes they have asked participants to estimate the total egg production in the United States or the number of physicians and surgeons listed in the Yellow Pages of the phone directory for Boston or the number of foreign automobiles imported into the United States, or even the toll collections of the Panama Canal in millions of dollars. Subjects could choose any range they liked, with the aim of being wrong no more than 2 percent of the time. The results were amazing. In the final tally, instead of just 2 percent, they proved incorrect 40 percent of the time. The researchers dubbed this amazing phenomenon the overconfidence effect.

The overconfidence effect also applies to forecasts, such as stock market performance over a year or your firm’s profits over three years. We systematically overestimate our knowledge and our ability to predict—on a massive scale. The overconfidence effect does not deal with whether single estimates are correct or not. Rather, as Taleb puts it, “it measures the difference between what people actually know and how much they think they know.” What’s surprising is this: Experts suffer even more from the overconfidence effect than laypeople do. If asked to forecast oil prices in five years’ time, an economics professor will be as wide of the mark as a zookeeper will. However, the professor will offer his forecast with certitude.

The overconfidence effect does not stop at economics: According to Taleb, 84 percent of Frenchmen estimate that they are above-average lovers. Without the overconfidence effect, that figure should be exactly 50 percent—after all, the statistical “median” means 50 percent should rank higher and 50 percent should rank lower. In another survey, 93 percent of the U.S. students estimated to be “above average” drivers. And 68 percent of the faculty at the University of Nebraska rated themselves in the top 25 percent for teaching ability. Entrepreneurs and those wishing to marry also deem themselves to be different: They believe they can beat the odds. In fact, entrepreneurial activity would be a lot lower if the overconfidence effect did not exist. For example, every restaurateur hopes to establish the next Michelin-starred restaurant, even though statistics show that most close their doors after just three years. The return on investment in the restaurant business lies chronically below zero.

Hardly any major projects exist that are completed in less time and at a lower cost than forecasted. Some delays and cost overruns are even legendary, such as the Airbus A400M, the Sydney Opera House, and Boston’s Big Dig. The list can be added to at will. Why is that? Here, two effects act in unison. First, you have the classic overconfidence effect. Second, those with a direct interest in the project have an incentive to underestimate the costs: Consultants, contractors, and suppliers seek follow-up orders. Builders feel bolstered by the optimistic figures, and through their activities, politicians get more votes. We will examine this strategic misrepresentation later in the book.

What makes the overconfidence effect so prevalent and its effect so confounding is that it is not driven by incentives; it is raw and innate. And it’s not counterbalanced by the opposite effect, “underconfidence,” which doesn’t exist. No surprise to some readers: The overconfidence effect is more pronounced in men—women tend not to overestimate their knowledge and abilities as much. Even more troubling: Optimists are not the only victims of the overconfidence effect. Even self-proclaimed pessimists overrate themselves—just less extremely.

In conclusion: Be aware that you tend to overestimate your knowledge. Be skeptical of predictions, especially if they come from so-called experts. And with all plans, favor the pessimistic scenario. This way, you have a chance of judging the situation somewhat realistically.

Back to the question from the beginning: Johann Sebastian Bach composed 1,127 works that survived to this day. He may have composed considerably more, but they are lost.

* Source: The Art of Thinking Clearly by Rolf Dobelli