The product in question is Pro2000 gel, and the results of the first large trial on more than 3,000 women were reported yesterday at the Conference on Retroviruses and Opportunistic Infections. CROI is all a scientific conference should be (and all the biannual AIDS Circus is not), and you can see and hear every presentation online. The results of the Pro2000 study show that women using the gel were 30% less likely to become infected with HIV than women using a placebo, and a third less likely than women using nothing at all. The reason the researchers are not screaming about it more joyfully is that the results are “not statistically significant”. Meaning, in this particular case, that we can only be between 90 and 94% sure that the difference in infection rates were really the results of the gel, and not the results of pure chance.

This is just silly. If I told you that there was a 94% chance that the red car was a third more likely to crash than the blue car, which would you drive? Yet we’ve managed to establish a norm in the scientific community that only differences that have a 95% probability of not being due to chance can be trusted. For nerds, that means a “p value” of five percent or less is sacrosanct: (p <0.05) has become a talisman of good science. I’m not the first to remark that things can be significant without being statistically significant — economist Tim Hartford wrote a column on statistical significance and Guinness in the FT only last week.

Someone at the conference remarked that “none of us in this audience worship at the alter of the p value of point oh five” but in fact, many of us do. Another thing that researchers at CROI have been bending over backwards to do is to prove that people on ARVs don’t have more risky sex than people not on ARVs. (Aside: this completely misses the point about “behavioural disinhibition” — jargon for “Oh look! HIV won’t kill me! Let’s party!”. What matters is not so much what infected people do once they are on meds, what matters is what uninfected people do because they no longer see any visible connection between unprotected sex and death. Still, people feel the need to show that ARVs don’t make you screw more.) So when a group working in Uganda showed that people on ARVs were 70% more likely to have an extramarital partner than people not on ARVs, they were happy to worship at the alter of the p value of point oh five. In this case, the p value was 0.09 — in other words there was a greater than 90% chance that the differences were real, but researchers were able to say there were “no differences”. We worship from the underside of the alter, too. A larger study looking at ARVs, risky sex and HIV transmission found that unprotected sex was “significantly lower” in those on ARVs. In fact, 17% of those on ARVs reported unprotected sex compared with 19% of those not on ARVs. The difference may have been statistically significant, yes, but does it meet the most important test of significance, the “So What?” test? Almost certainly not.

Epi-rant over. The microbicide trial (and the fact that there is very low transmission from people on antiretrovirals to their partners in the two ARV studies I’ve just ranted about) wasn’t the only good news at CROI today. Giving monkeys antiretrovirals before exposing them to SHIV rectally worked pretty well, too, which bodes well for PrEP in humans. Disappointingly, though, it worked best when the drugs were given between a week and a day before exposure — ARVs taken just a couple of hours before exposure didn’t have much effect. Bang goes my dream of earning millions with an Ecstasy/ Viagra/ Tenofovir combination pill for big nights out. Maybe I’ll just have to settle down and get a real job.

Thanks to Roger for prodding me to spend my day at a virtual conference…