The Dubious Efficacy of Doctors, Consultants, and Psychotherapists: Regression to Mean
His back pain was sometimes better, sometimes worse. There were days when he felt like he could move mountains, and those when he could barely move. If that was the case—fortunately it happened only rarely—his wife would drive him to the chiropractor. The next day he felt much more mobile and recommended the therapist to everyone.
Another man, younger and with a respectable golf handicap of 12, gushed in a similar fashion about his golf instructor. Whenever he played miserably, he booked an hour with the pro, and, lo and behold, in the next game he fared much better.
A third man, an investment adviser at a major bank, invented a sort of “rain dance” that he performed in the restroom every time his stocks had performed extremely badly. As absurd as it seemed, he felt compelled to do it: Things always improved afterward.
What links the three men is a fallacy: the regression-to-mean delusion.
Suppose your region is experiencing a record period of cold weather. In all probability, the temperature will rise in the next few days—back toward the monthly average. The same goes for extreme heat, drought, or rain, Weather fluctuates around a mean. The same is true for chronic pain, golf handicaps, stock market performance, luck in live, subjective happiness, and test scores. In short, the crippling back pain would most likely have improved without a chiropractor. The handicap would have returned to 12 without additional lessons. And the performance of the investment adviser would also have shifted back toward the market average—with or without the restroom dance.
Extreme performances are interspersed with less extreme ones. The most successful stock picks from the past three years are hardly going to be the most successful stocks in the coming three years. Knowing this, you can appreciate why some athletes would rather not make it on to the front pages of the newspapers: Subconsciously they know that the next time they race, they probably won’t achieve the same top result—which has nothing to do with the media attention, but with natural variations in performance.
Or take the example of a division manager who wants to improve employee morale by sending the least motivated 3 percent of the workforce on a course. The result? The next time he looks at motivation levels, the same people will not make up the bottom few—there will be others. Was the course worth it? Hard to say, since the group’s motivation levels would probably have returned to their personal norms even without the training. The situation is similar with patients who are hospitalized for depression. They usually leave the clinic feeling a little better. It is quite possible, however, that the stay contributed absolutely nothing.
Another example: In Boston, the lowest-performing schools were entered into a complex support program. The following year, the schools had moved up in the rankings, an improvement that the authorities attributed to the program rather than to natural regression to mean.
Ignoring regression to mean can have destructive consequences, such as teachers (or managers) concluding that the stick is better than the carrot. For example, following a test, the highest-performing students are praised and the lowest are castigated. In the next exam, others students will probably—purely coincidentally—achieve the highest and lowest scores. Thus, the teacher concludes that reproach helps and praise hinders: a fallacy that keeps on giving.
In conclusion: When you hear stories such as: “I was sick, went to the doctor, and got better a few days later” or “the company had a bad year, so we got a consultant in, and now the results are back to normal,” look out for our old friend, the regression-to-mean error.
* Source: The Art of Thinking Clearly by Rolf Dobelli