Learning How to Think - Kristoff - NYTimes.comSorry, I really do want to see a physician with expertise. I don't buy the extreme view that expertise doesn't matter.... The expert on experts is Philip Tetlock, a professor at the University of California, Berkeley. His 2005 book, “Expert Political Judgment,” is based on two decades of tracking some 82,000 predictions by 284 experts. The experts’ forecasts were tracked both on the subjects of their specialties and on subjects that they knew little about.
The result? The predictions of experts were, on average, only a tiny bit better than random guesses — the equivalent of a chimpanzee throwing darts at a board.
“It made virtually no difference whether participants had doctorates, whether they were economists, political scientists, journalists or historians, whether they had policy experience or access to classified information, or whether they had logged many or few years of experience,” Mr. Tetlock wrote.
Indeed, the only consistent predictor was fame — and it was an inverse relationship. The more famous experts did worse than unknown ones. That had to do with a fault in the media. Talent bookers for television shows and reporters tended to call up experts who provided strong, coherent points of view, who saw things in blacks and whites....
Mr. Tetlock called experts such as these the “hedgehogs,” after a famous distinction by the late Sir Isaiah Berlin (my favorite philosopher) between hedgehogs and foxes. Hedgehogs tend to have a focused worldview, an ideological leaning, strong convictions; foxes are more cautious, more centrist, more likely to adjust their views, more pragmatic, more prone to self-doubt, more inclined to see complexity and nuance. And it turns out that while foxes don’t give great sound-bites, they are far more likely to get things right.
This was the distinction that mattered most among the forecasters, not whether they had expertise. Over all, the foxes did significantly better, both in areas they knew well and in areas they didn’t.
I do, however, accept that fame is a reasonably reliable measure of incompetence. I'm a firm believer in the power of humility, which is probably a proxy for "self-doubt".
So I'd take a physician with both expertise and humility, and maybe more self-doubt than most would like. Spare me the famous physician. As a former scholar of evidence-based medicine, I can empirically confirm that the "opinions of the great" are frequently worthless.
The same thing, by the way, goes for CEOs. If they're famous enough to appear on a magazine cover, be suspicious.
Steve Jobs being an exception - but there's only one of him.
Ok. Bill Gates too.
Oh, wait, maybe I'm wrong?
But my self-doubt means you should trust me ...
1 comment:
I wouldn't say Tetlock is dissing the value of experts in political forecasting. Rather, he's saying that the people who are most frequently considered to be experts--the hedgehogs--really aren't. If you want expert medical treatment, go to a good doctor. If you want the best political forecasting you can get, forget the soundbite spewing idiots and hire a few foxes.
Post a Comment