Thanks Angela for a very good overview of the reproduciblility problem. It is the bane of science in far more fields than the social sciences. As both Science and Nature admitted over a decade ago, and as the NIH has confirmed, over 50% of all peer-reviewed published articles have no real validity. When I worked at a NIH-funded research institute it was common to see principal investigators doing animal studies using two control and two experimental animals, and then writing up the results using statistical techniques that were totally invalid. And then getting published in reputable journals.

Until there’s significant effort put into teaching researchers how to design proper experiments (double blind, placebo controlled, adequate sample sizes, controlling for variables, etc.) and into reproducing claimed experimental results, the problems will continue.

And sadly the mass media, with its addiction to simple-minded sensationalism, will always amplify the problems of misleading results via bogus headlines that claim “scientist cures cancer!” or “scientist finds cure for everything!” or “coffee leads to brain cancer but cures pimples!!”