Years ago, when our kids were young and we spent nearly every summer weekend boating all over northern Michigan, I remember how dependent we were on the weather. Starting early in the week, I remember hanging onto every word of the TV weatherman’s forecast, switching networks if we didn’t like what we heard — as if that would somehow change things.
Over time, I also realized how often they were wrong — and it bugged me that forecasters harbor no sense of accountability when their “mostly sunny” forecast on Wednesday is swept away by a steady downpour come Saturday afternoon.
Are northern Michigan meteorologists the only trained experts whose predictions consistently fall through?
Not at all, thanks to some fascinating research by University of Pennsylvania social science Professor Phil Tetlock, who, while serving on the National Research Council in 1984, was asked to respond to political experts’ dire predictions that relations between the U.S. and Soviet Union were ready to collapse in the wake of Ronald Reagan’s infamous “evil empire” speech. The following year, when Mikhail Gorbachev became general secretary and everything changed, Tetlock was exasperated that none of the so-called “experts” who made these dire predictions admitted their mistake. In response, he created a study designed to hold experts’ feet to the fire.
He recruited 284 people who were designated experts — highly educated people (over half had PHD’s) who frequently offered advice on political or economic trends. Over 61% had been interviewed by the national media. They were asked to make predictions in their area of expertise. Economists were asked about future growth rates of GDP, for example, while political experts predicted which party would lose or retain control in upcoming elections. Tetlock stuck to fairly basic predictions, trying to create such clear questions that experts would have no excuses if they were wrong.
From the mid-80’s until 2003, he collected over 82,000 predictions, publishing his findings in a book called Expert Political Judgment: How Good is It? How Can We Know?
The results were astounding. Not only were the so-called “experts” way off, but in addition, according to Tetlock, even the best forecasters did worse than what the professor termed a “crude extrapolation algorithm,” which is a simple computation that takes the base rates and projects that the trends from the past few years will continue. (Example: If the GDP growth rate average 1.9% over the past four years, it will continue to grow at 1.9%). What’s more, those with higher levels of education did no better.
And even more interesting, the experts who made more media appearances (think high profile media outlets like CNN and Fox News) had the lowest performance! And further research has shown that a wide range of fields, from psychologists to lawyers, have a similarly horrible track record at making predictions.
In other words, when it comes to predicting the future, you’re likely to get no more accurate projections than a pimply-faced teenager with basic information and a calculator.
What do you do with this surprising finding? I can think of three major reasons why skepticism plays a positive role here:
Be prudently skeptical. Resist the urge to blindly trust “expert opinions” on world-shaping trends that dominate the news outlets and talk shows. Whether it’s climate change, the collapse of social security, or the rising cost of a college education, be prepared, but don’t overreact. Chances are, the future is not nearly as bleak — or bright — as the “experts” proclaim.
Understand the media’s underlying goal. When you’re watching pundits on CNN or Fox News prognosticate on some future trend, remember: the main objective of the media is not reporting the news but winning the ratings battle. The result: controversy, drama, and sensationalism take precedence over accurate, balanced news coverage. Perhaps that explains why some of the most successful people I know almost never watch the news!
Don’t be swayed by fear — or greed. If you’re starting your own business, launching a new career, or doing anything that involves risk, remember: risk is unavoidable. Sometimes you need to trust your instincts and plunge forward. On the other hand, don’t be swayed by over-optimism (think late 90’s tech stocks). Generally, your grandparents had it right when they said, “If it sounds too good to be true, it probably is.”
What about you? Does this revealing surprise you? Does it change the way you interpret the opinions of experts? I’d really like to know!