Sunday, November 02, 2008

Experts failing - how to get better guidance for economics, health care and everything

Somewhere in my library, I have an older book that compiled all the different ways experts have been wrong over the decades. Now that I've accumulated more personal entropy I have little need of the book; I've lived through cycles of abandoned medical fads endorsed by panels whose "expertise" was exceeded only by their egos.

Today, of course, we're specifically wondering about expert economists. The NYT proposes Groupthink (Wikipedia) as a villain ...
Economic View - Challenging the Crowd in Whispers, Not Shouts - NYTimes.com

... In his classic 1972 book, “Groupthink,” Irving L. Janis, the Yale psychologist, explained how panels of experts could make colossal mistakes. People on these panels, he said, are forever worrying about their personal relevance and effectiveness, and feel that if they deviate too far from the consensus, they will not be given a serious role. They self-censor personal doubts about the emerging group consensus if they cannot express these doubts in a formal way that conforms with apparent assumptions held by the group...
As an expert in having opinions, I'll respond. Groupthink is probably a contributing factor, but it's not the whole story. Happily we know of a fix for the bigger problem. Unhappily, Newt Gingrich blew away one of our best examples.

First some history, then the lessons.

For much of the 20th century medical consensus was achieved the way economics panels work today. A group of "experts", typically involving quite a bit of Harvard, met and pontificated. These groups lacked neither confidence nor ego, but their accuracy left a lot to be desired.

After 80 years or so of this, we evolved something that's now called "evidence based medicine", but had another label in the 80s (sorry, the labels blur together). In the 1980s we even had a federal agency that took an "evidence based" approach to recommendations, one founded in science and reason rather than reputation and ego. It was known then as the Agency for Health Care Policy and Research (AHCPR).

Problem is, the AHCPR's expert panels came up with recommendations that were very different from the old-style "experts". That was bad enough, but these recommendations affected payment. Orthopedic surgeons, in particular, were furious when an AHCPR guideline recommended far less back surgery. Gingrich (remember him) zeroed out funding for the AHCPR (great story in the link).

Yes, even then the Republican War on Reason had begun. Remember that the next time you hear Newt pontificate. For all of his intellect and pride, he began our long fall.

Of course the AHCPR panel was right. Now, fifteen years later, even orthopedic surgeons agree with the original recommendations. As far as I can tell, the IPCC took a similar approach to their recommendations.

So expert panels, done right, can be very effective. They also acquire very powerful enemies, and without a supportive political environment they will be destroyed.

The AHRQ rose from the ashes of the AHCPR, but it was never again so bold. It has largely hued to the conventional approach to "expert panels", though it's clear that the leadership would like to return to the days of reason.

If Obama wins, they will.

So history teaches us that the conventional approach to "expert consensus" is deeply flawed. Maybe it's the groupthink, maybe it's the way the "experts" are chosen, but the AHCPR lesson tells us that there's a better way. There's an approach that combines a rational process with open discussion and a certain measure of humility about what we don't know.

It would be insane to dispense with expert consensus all together. That way lies climate change denialisism, vaccine autism and other triumphs of unreason. We can, however, do far better than the old style of "medical experts" and the current practice of "economics experts".

We can study when expert panels work and when they fail, and codify and continuously improve the best practices. We can develop an empirical and informed science of expertise.

Or we could keep blowing up the world economy, bake the planet too, and leave our children wandering in the rubble.

We do have a choice. Two days from now in fact. Reason, or the dark ages. Pick the one you like ...

1 comment:

  1. I don't have alot of information on the political history of the medical industry, but I do believe that we are going in the wrong direction.
    While I believe the government has a duty to the country to have some regulation, I think this has been done to the destruction of "reason". How does a bureaucrat in Washington decide the health for all. One size fits all does not work for "health" care. The rights of individuals have been treaded on and, in many cases, have been completely taken away. I believe that FDA preemption, which has taken away the freedom of redress, is one example of this. Have expenses gone down? No. In fact, expenses have gone up dramatically. Another fact that the media was reporting when the housing crisis began and is conveniently ignoring now, is that all of the foreclosures were due to medical expenses.
    Also, the government does have some obligation to get the expenses into control. Medicaid is a spectacular failure - why would we expand this to everyone? While we do expect some protection from the government, why do they think that gives them to take away personal freedom from people? As it is now, in order to get help from Medicaid, a person must give up his house - making that person a ward of the state forever. Who profits from this? It's certainly not the individual nor the public.
    Finally, I believe while it is again, the duty to provide some medical help, I'm against the government or socialist insurance or whatever to completely pay for medical expenses. I believe that everyone has the right and the obligation to pay some of the expenses. If we have some connection to the expenses, the expenses will go down. If the government has the right to force us into all of the expectations of the medical industry, we are doomed. As I'm understanding Medicaid now, the average 60+ yr old is receiving 29 prescriptions per year. Are we protecting them by paying for all of the drugs the medical industry can supply? Should we mandate medical tests, drugs and procedures for everyone, ignoring health risks, personal freedom, and then ignore the inevitable financial destruction and predatory behavior?

    ReplyDelete