The editors of Environmental Microbiology have published a year-end round-up of some of their more notable reviews. (pdf)

A couple that made me feel better about some of the things I’ve said:
“I suppose that I should be happy that I don’t have to spend a lot of time reviewing this dreadful paper; however I am depressed that people are performing such bad science.”

My personal favourite, because it mirrors how I feel today:
“The writing and data presentation are so bad that I had to leave work and go home early and then spend time to wonder what life is about.”

In the spirit of year-end cheer, let’s pull out some praise, too:
“Many spend much more time and space to say considerably less.”

It’s funny, isn’t it, that well-honed criticism seems so much sharper a weapon in attack than fullsome praise does in defence. It’s not just in peer review. I’ve recently been enjoying dipping in to a glorious collection of reviews of perfumes, published by Profile in the UK and Penguin in the States.

Here the rapier:
Desir deRochas Femme (Rochas) bleached rose
Thoroughly unpleasant fresh-rosy floral that whines like a dentist’s drill and hurts almost as much.

Here the back-handed compliment:
Deseo (Jennifer Lopez) coconut melon
Deseo is a clever mix of sauvignon blanc and Bailey’s, or in perfumery terms, Envy and Rush. Separately, its components would cause, respectively, a toothache and tooth rot. Together they work happily to produce a shortlived but superbly trashy fragrance for eighteen-year-old girls on the prowl.

And here the fullsome praise:
Le Feu d’Issey (Issey Miyake) milky rose
Whoever did this has that rarest of qualities in perfumery, a sense of humour…A reminder that perfume is, among other things, the most portable form of intelligence

Thanks to First Among Peers Mark Zip for the head’s up on peer review, and to Andrew Franklin for causing me to spend hours of my life buried in reviews of a product I don’t even use.

Tom Scott has produced a series of handy-dandy stickers to put on newspaper articles. Here are a couple of my favourites, but you can see the whole set here, and even print out your own set.

What I’d like Tom to get on to next is a set of stickers to help me quickly find the crap research in peer reviewed journals. He could start with “Warning: This paper was reviewed by someone who has applied for a job with the first author” and “Warning: Reprints of this article ordered by Big Pharma will account for 18 percent of the journal’s income this month”.

Thanks to CB for bringing Tom to my attention.

The paper must be important because it contains sentences such as: “A modest, but significant, relationship was found between this variable and the six-item index [of ‘dichotomously assessed condom use errors and problems’] (r = .25; P < 0.01) as well as the summative measure (r = .35; P < 0.01).” It apparently confirms the startling “Sloppy drunk = Condom slob = drippy dick” equation. Though for the nerds amongst us, the stats are quite a trick. Sex while high 2.78 times in two months, with a standard deviation of 5.5. Hmmmm.

I’d like to steal a line from Lee’s comment and suggest that we start a “Where do Bears Shit?” campaign, to try to get medical journals to reform their “peer review” process. The journals I review for regularly send me knuckle-draggingly appalling papers, together with on-line forms that require me to rate the paper’s statistical accuracy, and to assess the extent to which it meets ethical criteria (on a five-point scale, nautrally). But nowhere are reviewers asked to pronounce on whether a paper answers the universal research question: “So What?”

Frankly, a lot of the stuff I see is so bad that I would hesitate to consider its authors peers. I don’t hesitate to tell its authors, and a journal’s editors what I think. And yet when I look at comments by other reviewers of the same paper, I see that this is by no means the norm. Here are comments on a paper I reviewed recently. One of them is mine:

Reviewer 1: This paper… has no clear objective, is poorly written and completely overlooks most of the issues of potential public health importance.
Reviewer 2:The paper is generally well written and focused.
Reviewer 3: This is a well written article… The conclusion has some spelling errors.

The problems with the peer review system are legion. We feel under professional obligation to take on reviews, but there’s little incentive to invest time and energy in doing them well. There’s a “you scratch my back and I’ll scratch yours” thing going on, especially in journals that a) allow authors to recommend individuals as reviewers and b) have drifted away from anonymity for reviewers and authors. And reviewers are often reluctant to appear too negative in their comments to editors, because we tend to review for the journals we most want to publish in.

The other campaign suggested by Lee’s comments is the “Bloodsuckers” campaign. I’m thrilled at the growth of open access publishing of important scientific findings, and hugely supportive of institutions such as the Wellcome Trust, which are increasingly stipulating that any research they fund must be submitted to publications which do not charge for access. But it is shameful that it should be a private charitable trust that blazes this path. Most public health research is paid for by you and me and all the other long-suffering tax payers out there. We have a right to be able to see the results of our investment without paying again to the Bloodsuckers at Elsevier, LWW et al. So it’s exciting to see that the budget bill referred to in Needling the White House takes small steps towards ensuring open access. It requires all NIH-funded research to be posted on PubMed Central within a year of publication. A sort of open-after-a-bit access, but better than nothing. Thanks to purepedantry for bringing this to my attention.