Academics Andre's Research Biocuriosities Books Graduate School History of Science Hot off the Press Igor's Research Interdisciplinarity Molecule of the Month Open Access Philip's Research Philosophy of Science Physics Physicsworld.com
Backreaction Ceclia's Blog at PHD Comics Cocktail Party Physics Cosmic Variance The Daily Transcript Easternblot Everyday Scientist The Evilutionary Biologist Freelancing Science The Futile Cycle Good Math, Bad Math iMechanica in singulo Incoherently Scattered Ponderings Juniorprof Klara Stefflova Life of a Lab Rat The Loom Metadatta Mixed States Morning Coffee Physics Not Even Wrong Notes from the biomass Notional Slurry OpenScience Project Pharyngula PLoS Blog Ponderings of a fool Recombinants The Sandwalk SciAm Observations ScienceBlogs Scientific Clearing House Shtetl-Optimized Three-toed Sloth Uncertain Principles What's New by Bob Park
PLoS Medicine has an editorial (freely accessible, as always) on the impact factor, and how it is a poor judge of a journal’s impact. For anyone unfamiliar with a the impact factor, Thomson Scientific calculates the impact factor as:
Journal X’s 2005 impact factor =
Citations in 2005 (in journals indexed by Thomson Scientific [formerly known as Thomson ISI]) to all articles published by Journal X in 2003–2004
divided by
Number of articles deemed to be “citable� by Thomson Scientific that were published in Journal X in 2003–2004
As you can imagine, there are some issues, not the least of which is: what exactly does Thomson deem a citable article? From the editorial,
During discussions with Thomson Scientific over which article types in PLoS Medicine the company deems as “citable,â€? it became clear that the process of determining a journal’s impact factor is unscientific and arbitrary. After one in-person meeting, a telephone conversation, and a flurry of e-mail exchanges, we came to realize that Thomson Scientific has no explicit process for deciding which articles other than original research articles it deems as citable. We conclude that science is currently rated by a process that is itself unscientific, subjective, and secretive.
Depending on whether Thomson included only original research articles the impact factor for PLoS Medicine weighed in at 11, but it was as low as three if all articles in the journal were deemed citable. An impact factor of 11 puts it among the higher cited journals around, while an impact factor of three, while still respectable, is considerably more pedestrian.
Some of us may be wondering, “who cares?” The editorial is quick to point out, however, that a journal’s impact factor comes into play all the time, everything from determining who gets tenure to where scientists should attempt to publish their own work. Everyone would like their articles to be read (and ultimately cited), and the impact factor is one way to try and find the right journal.
Or is it? Over at the new PLoS blog, Chris Surridge has his own take on the impact factor game, and mentions that the impact factor of a journal is really not a good judge of the potential impact a paper published in the journal is going to have. He says,
Problem is citations aren’t normally distributed across those papers making the power of that average to predict the likely citations of an individual paper very low. As a rule of thumb 80% of a journals impact factor is determined by 20% of the papers published.
So, what is the solution? So far there doesn’t seem to be any obvious alternative (though they do mention the Y factor which combines the impact factor and Google’s pagerank algorithm), and I suspect inertia is going to keep impact factors in the spotlight for a while yet. As a perfect example, even when a publishing house such as PLoS realises the near-useless quantitative value in the impact factor, it is still quoted as though it is meaningful on the PLoS Biology page.
Biocurious is written by Andre Brown and Philip Johnson, since 2005. Content of the weblog is licensed under a Creative Commons Attribution-Share Alike 3.0 License.
