Biocurious is a weblog about biology, quantified.

Most science papers are probably wrong

by PhilipJ on 31 August 2005

Via /., a NewScientist story stating that there is less than a 50% chance that a given scientific article is free from procedural or statistical errors.

This doesn’t surprise me. The signal to noise ratio in science is terrible. There are simply too many journals. The Nature Publishing Group themselves now publish some 50+ journals. The publish or perish mentality has diluted the value of publishing.

This also doesn’t terribly bother me. A group’s statistical analysis or procedure is there to read in their paper or supplemental material (or at least, it should be...). If I think the analysis has been carried out incorrectly, I’m free to publish a comment (and the original group gets a chance to reply to my comment, often giving further insight into why they think their analysis is correct).

Furthermore, science papers don’t have to be right all the time. Scientists build on the work of others (”...standing on the shoulders of giants…”), and if published results are incorrect, it will quickly become obvious to the variety of other people studying the same or similar systems (for example, the Jan Hendrik Schon fiasco). It is, as in the Schon case, when results are intentionally fabricated that we should get worried.

  Textile help