False Prophets: Forecast Illusion
“Facebook to be number one entertainment platform in three years.”
“Regime shift in North Korea in two years.”
“Sour grapes for France as Argentinian wines expected to dominate.”
“Euro collapse likely.”
“Low-cost space flights by 2025.”
“No more crude oil in fifteen years.”
Every day, experts bombard us with predictions, but how reliable are they? Until a few years ago, no one bothered to check. Then along came Philip Tetlock. Over a period of ten years, he evaluated 28,361 predictions from 284 self-appointed professionals. The result: In terms of accuracy, the experts fared only marginally better than a random forecast generator. Ironically, the media darlings were among the poorest performers; and of those, the worst were the prophets of doom and disintegration. Examples of their far-fetched forecasts included the collapse of Canada, Nigeria, China, India, Indonesia, South Africa, Belgium, and the EU. None of these countries has imploded.
“There are two kinds of forecasters: those who don’t know, and those who don’t know they don’t know,” wrote Harvard economist John Kenneth Galbraith. With this he made himself a figure of hatred in his own guild. Fund manager Peter Lynch summed it up even more cuttingly: “There are 60,000 economists in the U.S., many of them employed full-time trying to forecast recessions and interest rates, and if they could do it successfully twice in a row, they’d all be millionaires by now…. As far as I (Rolf Dobelli) know, most of them are still gainfully employed, which ought to tell us something.” That was ten years ago. Today, the United States could employ three times as many economists—with little or no effect on the quality of their forecasts.
The problem is that experts enjoy free rein with few negative consequences. If they strike it lucky, they enjoy publicity, consultancy offers, and publication deals. If they are completely off the mark, they face no penalties—neither in terms of financial compensation nor in loss of reputation. This win-win scenario virtually incentivizes them to churn out as many prophecies as they can muster. Indeed, the more forecasts they generate, the more will be coincidentally correct. Ideally, they should have to pay into some sort of “forecast fund”—say, $1,000 per prediction. If the forecast is correct, the expert gets his money back with interest. If he is wrong, the money goes to charity.
So what is predictable and what is not? Some things are fairly simple. For example, I have a rough idea of how many pounds I will weigh in a year’s time. However, the more complex a system, and the longer the time frame, the more blurred the view of the future will be. Global warming, oil prices, or exchange rates are almost impossible to foresee. Inventions are not at all predictable because if we knew what technology we would invent in the future, we would already have invented it.
So, be critical when you encounter predictions. Whenever I hear one, I make sure to smile, no matter how bleak it is. Then I ask myself two questions. First, what incentive does the expert have? If he is an employee, could he lose his job if he is always wrong? Or is he a self-appointed guru who earns a living through books and lectures? The latter type of forecaster relies on the media’s attention so, predictably, his prophecies tend to be sensational. Second, how good is his success rate? How many predictions has he made over the past five years? Out of these, how many have been right and how many have not? This information is vital, yet often goes unreported. I implore the media: Please don’t publish any more forecasts without giving the pundit’s track record.
Finally, since it is so fitting, a quote from former British prime minister Tony Blair: “I don’t make predictions. I never have, and I never will.”
* Source: The Art of Thinking Clearly by Rolf Dobelli