Biocurious

/ a biophysics blog

Peer review is not to blame

Posted 23 March 2006 by PhilipJ under

This week’s Nature has another (and in my opinion, misguided) correspondence about scientific fraud and how to stop it. The premise is that big name authors are going to have an easier time getting papers accepted in big journals under the current referee system (where the referees know who the authors of a paper are), and that referees should instead be sent authorless papers in an effort to more objectively critique the work at hand. I think the double-blind refereeing process is a good idea for a variety of other reasons (people in the same field as you postponing your results getting published comes to mind), but I disagree entirely that this will in any way change the way we catch fraudulent science. Henning Bauch writes (subscription required),

In the most notorious recent cases (the Korean stem-cell work and Jan Hendrik Schön’s nanotechnology work), the peer-review system must be said to have failed, as the frauds were unveiled by people from outside the immediate process. Were the referees the weakest link, and were both the editors and the referees blinded by the aura of the authorships?

I’m going to quote André from a few posts ago, because I wholeheartedly agree with his interpretation:

Peer review can weed out crackpots and can enhance serious research by finding methodological or theoretical flaws and suggesting more or better controls, but it isn’t even really meant to catch someone like Hwang in the act. That must rely on one of modern science’s other safety checks: reproducibility.

In short, peer review is not how we are going to catch fraud. It is in reproducing and extending others’ work that discrepancies or outright fabrications of data are going to be revealed. As André said, if there are no obvious methodological or interpretational errors, and the proper controls have been carried out (even if the data is fabricated), there is little else a referee can complain about. Without doing the measurements themselves, it isn’t possible to know if the data has been fabricated, and it is inconceivable to expect referees to reproduce experimental results prior to recommending the paper be published or not. So no, I don’t agree that the peer review system must be said to have failed in either of these cases, since that isn’t what it is meant to catch to begin with.

  

  1. Xerxes    23 March 06    #

    Maybe I live in a small field, but I find it hard to believe that an informed reviewer would not easily be able to decifer the likely authors of a work. You could probably figure it out just by counting references. I know some people can discern the anonymous reviewers of their papers just by the writing style and pet peeves of the reviews they get back.


  2. PhilipJ    23 March 06    #

    Perhaps I do too, but I’m also willing to bet I could guess the author of a paper based on just a few simple parameters: type of optical tweezers used, system of study, etc.

    With that said, I still think the double-blind refereeing system makes more sense than having the referees in the know, but leaving authors in the dark.


Name
Email
http://
Message
  Textile Help