by PhilipJ on 23 March 2006
This week’s Nature has another (and in my opinion, misguided) correspondence about scientific fraud and how to stop it. The premise is that big name authors are going to have an easier time getting papers accepted in big journals under the current referee system (where the referees know who the authors of a paper are), and that referees should instead be sent authorless papers in an effort to more objectively critique the work at hand. I think the double-blind refereeing process is a good idea for a variety of other reasons (people in the same field as you postponing your results getting published comes to mind), but I disagree entirely that this will in any way change the way we catch fraudulent science. Henning Bauch writes (subscription required),
In the most notorious recent cases (the Korean stem-cell work and Jan Hendrik SchÃ¶n’s nanotechnology work), the peer-review system must be said to have failed, as the frauds were unveiled by people from outside the immediate process. Were the referees the weakest link, and were both the editors and the referees blinded by the aura of the authorships?
I’m going to quote AndrÃ© from a few posts ago, because I wholeheartedly agree with his interpretation:
Peer review can weed out crackpots and can enhance serious research by finding methodological or theoretical flaws and suggesting more or better controls, but it isn’t even really meant to catch someone like Hwang in the act. That must rely on one of modern science’s other safety checks: reproducibility.
In short, peer review is not how we are going to catch fraud. It is in reproducing and extending others’ work that discrepancies or outright fabrications of data are going to be revealed. As AndrÃ© said, if there are no obvious methodological or interpretational errors, and the proper controls have been carried out (even if the data is fabricated), there is little else a referee can complain about. Without doing the measurements themselves, it isn’t possible to know if the data has been fabricated, and it is inconceivable to expect referees to reproduce experimental results prior to recommending the paper be published or not. So no, I don’t agree that the peer review system must be said to have failed in either of these cases, since that isn’t what it is meant to catch to begin with.