At last -- the study on how well studies hold up

CNN.com - Research: Third of study results don't hold up - Jul 13, 2005

When I started my practice in rural Michigan 16 years ago Family Physicians were (even then!) being "beat up" for not adopting new research quickly enough. After a few years of practice I experienced quite a few reversals in "best practice" (such as the U turn on Magnesium post-MI).

Back then I proposed doing a study that would take a number of journals from the 1980s and see how well the recommendations held up. I started doing some preliminary work, but my life took other directions. Once it became apparent I wasn't going to do the research myself I talked it up with friends and colleagues. They weren't too impressed.

Which is why I'm so pleased someone has done the work, and confirmed what I'd guessed back then:

CHICAGO, Illinois (AP) -- New research highlights a frustrating fact about science: What was good for you yesterday frequently will turn out to be not so great tomorrow.

The sobering conclusion came in a review of major studies published in three influential medical journals between 1990 and 2003, including 45 highly publicized studies that initially claimed a drug or other treatment worked.

Subsequent research contradicted results of seven studies -- 16 percent -- and reported weaker results for seven others, an additional 16 percent.

That means nearly one-third of the original results did not hold up, according to the report in Wednesday's Journal of the American Medical Association.

Now there's some empiric support for the practice of gnarly old docs who like to wait a few years before implementing the very latest research -- especially when the benefits of a new approach seem relatively modest.