Thanks to Lee Rudolph for bringing to our attention another spectacularly insightful piece of (taxpayer funded) research (see Lee’s comment on an earlier post).
The paper must be important because it contains sentences such as: “A modest, but significant, relationship was found between this variable and the six-item index [of ‘dichotomously assessed condom use errors and problems’] (r = .25; P < 0.01) as well as the summative measure (r = .35; P < 0.01).” It apparently confirms the startling “Sloppy drunk = Condom slob = drippy dick” equation. Though for the nerds amongst us, the stats are quite a trick. Sex while high 2.78 times in two months, with a standard deviation of 5.5. Hmmmm.
I’d like to steal a line from Lee’s comment and suggest that we start a “Where do Bears Shit?” campaign, to try to get medical journals to reform their “peer review” process. The journals I review for regularly send me knuckle-draggingly appalling papers, together with on-line forms that require me to rate the paper’s statistical accuracy, and to assess the extent to which it meets ethical criteria (on a five-point scale, nautrally). But nowhere are reviewers asked to pronounce on whether a paper answers the universal research question: “So What?”
Frankly, a lot of the stuff I see is so bad that I would hesitate to consider its authors peers. I don’t hesitate to tell its authors, and a journal’s editors what I think. And yet when I look at comments by other reviewers of the same paper, I see that this is by no means the norm. Here are comments on a paper I reviewed recently. One of them is mine:
Reviewer 1: This paper… has no clear objective, is poorly written and completely overlooks most of the issues of potential public health importance.
Reviewer 2:The paper is generally well written and focused.
Reviewer 3: This is a well written article… The conclusion has some spelling errors.
The problems with the peer review system are legion. We feel under professional obligation to take on reviews, but there’s little incentive to invest time and energy in doing them well. There’s a “you scratch my back and I’ll scratch yours” thing going on, especially in journals that a) allow authors to recommend individuals as reviewers and b) have drifted away from anonymity for reviewers and authors. And reviewers are often reluctant to appear too negative in their comments to editors, because we tend to review for the journals we most want to publish in.
The other campaign suggested by Lee’s comments is the “Bloodsuckers” campaign. I’m thrilled at the growth of open access publishing of important scientific findings, and hugely supportive of institutions such as the Wellcome Trust, which are increasingly stipulating that any research they fund must be submitted to publications which do not charge for access. But it is shameful that it should be a private charitable trust that blazes this path. Most public health research is paid for by you and me and all the other long-suffering tax payers out there. We have a right to be able to see the results of our investment without paying again to the Bloodsuckers at Elsevier, LWW et al. So it’s exciting to see that the budget bill referred to in Needling the White House takes small steps towards ensuring open access. It requires all NIH-funded research to be posted on PubMed Central within a year of publication. A sort of open-after-a-bit access, but better than nothing. Thanks to purepedantry for bringing this to my attention.