It’s a huge disappointment to those who were hoping that the pill-a-day-to-avoid-a-pill-a-day solution might drag us out of the despond that we’ve been in as we contemplate 2.7 million new HIV infections this year. That’s the same number as were newly infected when I started in this business 15 years ago. The only real change is in the cost of our failure: that’s increase over 70-fold.

The study, well conducted by my former employers Family Health International in four countries, is at odds with a study released last November, that showed that a daily dose of Truvada, a combination of two antiretorvirals in pill form, cut the risk of infection among gay men by over 40%. The earlier study did show that — surprise surprise — taking pills to prevent HIV doesn’t work unless you actually take your pills. Though virtually everyone said they took their pills, a later analysis of blood samples showed that wasn’t true. We don’t yet have that same analysis for the new trial in women (dubbed FemPrep), so although 95% reported taking their pills, it’s possible that real adherence was much lower. More than possible: likely. All women in the study were taking contraception, but there was a 9% pregnancy rate, apparently much higher in women on the pill than women on injectibles (we don’t yet have the actual numbers). That suggests that some women aren’t all that good at taking one pill a day (I’m one of them; thank God for implants), let alone two.

There’s a real possibility that antiretrovirals taken through the mouth and processed through the digestive tract aren’t as effective at preventing HIV from finding an entry point in the vagina as they are in the rectum, hence the difference between the trials in straight women and gay men. If that’s the case, it ups the ante for putting ARVs directly into your fanny (using fanny in the English sense!). We know from the Caprissa trials that antiretroviral microbicide works vaginally (though not necessarily rectally). But we only know that it works if you “shoot up” both before and after sex; in theory, at least, that is what was being tried. In practice, we know (well, strongly suspect) how unlikely it is that women will actually do that on a long-term basis. We now desperately need trials of a one-shot vaginal microbicide. Because for all the talk of “bio-medical solutions” the confusing results of recent HIV prevention trials remind us that most bio-medical solutions have a very strong behavioural component. Pills that “work” if you take them are no good if they make you feel so sick, so choked, or so fed up that you don’t take them.

One of the things that pleased me greatly about the FemPrep trials was that researchers made sure that the women who volunteered for the research knew about the disappointing results before the press or the scientific community did. There are more details from the study teams about how they interacted with participants on this interesting conference call, arranged by the ever-helpful AVAC network.

This post must end the way these posts seem always to end, with an underlining of the shockingly high rate of new infections in the study overall: five percent of women became infected, despite the fact that they were given female and male condoms, were regularly screened for other sexually transmitted infections and treated as necessary, and counseled up the wazoo. It’s a reminder of how badly suited the tools in our current toolbox are to the job of HIV prevention, and a caution about expecting much more from other behavioural interventions such as the use of pills or gels.

Not wanting to be always the purveyor of bad news, I was looking forward to today’s results from the Pro2000 microbicide studies. After hopeful results in an earlier trial, I’d convinced myself the gel would prevent HIV. But it doesn’t.

It’s very depressing news from a huge, well designed and well managed study of over 9,000 women across four countries. I’m fond of saying that if you torture the statistics enough they will confess to anything (as we saw in the recent vaccine trial in northern Thailand.) But however badly you twist the arms of this study, they’re not going to scream success. For the record:

* If you exclude the women who got pregnant or stayed in the trial more than the planned period of one year, there were 130 new infections among those who used the microbicide gel, and 123 in those who used the identical-feeling placebo. Calculated as new infections per 100 woman-years of exposure, that’s 4.5 for the gel and 4.3 for the placebo, making the microbicide 5% more risky. The statisticians are 95% sure that the true effect of the microbicide is somewhere between decreasing risk by18 % or increasing it by 34 %. In other words, we can’t make any claims at all that the product works.

* If you don’t exclude people who got pregnant and look at everyone in the trial for as long as they were enrolled, you get 145 new infections with the microbicide versus 143 without: 4.6 new infections per 100 years of exposure in both. No difference. None. The true effect using this analysis is somewhere between reducing the risk of HIV infection by 21% and increasing it by 26%.

* The researchers also looked at whether women who used the gel consistently were less likely to get infected than those who didn’t. They weren’t.

It is hard to measure consistent use of microbicides in these studies, but researchers compared three sources of information: what all participants reported during study visits, whether all participants brought back used applicators, and what a sub-sample of women recorded in their detailed sex diaries. All point in the same direction: around 90% of women used the gel most of the time. That in itself might be counted a triumph compared to some earlier studies. It means that if we could find something that actually works, women would be quite likely to want to use it. All eyes will now be on the CAPRISA study which is testing a vaginal gel that has antiretorvirals embedded in it.

For my own part, I’m feeling somewhat sheepish as well as disappointed. When the results of an earlier, smaller trial of Pro2000 microbicide showed that it reduced infection by 30%, I was dismissive, bordering on rude, about the tyranny of the statisticians who said the results were “not significant”, and that we needed more research before acting.

I remain impatient with scientists who want to delay any action until we have perfect data. In the field of public health we are often obliged to do the best we can with what we have; as long as policy-makers are prepared to change their approach as the data improve we can save valuable time and lives. But in this case, the caution was well placed. Mea culpa.

Once again, it is worth drawing attention to the most basic fact in this research: 30 years into the epidemic, in a population that had safe sex counselling up the wazoo as well as universal access to condoms and other services such as STI treatment, more than four in 100 women are still getting infected with HIV. Pro2000 may not work, but just urging people to use condoms doesn’t, either. We need to keep looking for something that does.

The product in question is Pro2000 gel, and the results of the first large trial on more than 3,000 women were reported yesterday at the Conference on Retroviruses and Opportunistic Infections. CROI is all a scientific conference should be (and all the biannual AIDS Circus is not), and you can see and hear every presentation online. The results of the Pro2000 study show that women using the gel were 30% less likely to become infected with HIV than women using a placebo, and a third less likely than women using nothing at all. The reason the researchers are not screaming about it more joyfully is that the results are “not statistically significant”. Meaning, in this particular case, that we can only be between 90 and 94% sure that the difference in infection rates were really the results of the gel, and not the results of pure chance.

This is just silly. If I told you that there was a 94% chance that the red car was a third more likely to crash than the blue car, which would you drive? Yet we’ve managed to establish a norm in the scientific community that only differences that have a 95% probability of not being due to chance can be trusted. For nerds, that means a “p value” of five percent or less is sacrosanct: (p <0.05) has become a talisman of good science. I’m not the first to remark that things can be significant without being statistically significant — economist Tim Hartford wrote a column on statistical significance and Guinness in the FT only last week.

Someone at the conference remarked that “none of us in this audience worship at the alter of the p value of point oh five” but in fact, many of us do. Another thing that researchers at CROI have been bending over backwards to do is to prove that people on ARVs don’t have more risky sex than people not on ARVs. (Aside: this completely misses the point about “behavioural disinhibition” — jargon for “Oh look! HIV won’t kill me! Let’s party!”. What matters is not so much what infected people do once they are on meds, what matters is what uninfected people do because they no longer see any visible connection between unprotected sex and death. Still, people feel the need to show that ARVs don’t make you screw more.) So when a group working in Uganda showed that people on ARVs were 70% more likely to have an extramarital partner than people not on ARVs, they were happy to worship at the alter of the p value of point oh five. In this case, the p value was 0.09 — in other words there was a greater than 90% chance that the differences were real, but researchers were able to say there were “no differences”. We worship from the underside of the alter, too. A larger study looking at ARVs, risky sex and HIV transmission found that unprotected sex was “significantly lower” in those on ARVs. In fact, 17% of those on ARVs reported unprotected sex compared with 19% of those not on ARVs. The difference may have been statistically significant, yes, but does it meet the most important test of significance, the “So What?” test? Almost certainly not.

Epi-rant over. The microbicide trial (and the fact that there is very low transmission from people on antiretrovirals to their partners in the two ARV studies I’ve just ranted about) wasn’t the only good news at CROI today. Giving monkeys antiretrovirals before exposing them to SHIV rectally worked pretty well, too, which bodes well for PrEP in humans. Disappointingly, though, it worked best when the drugs were given between a week and a day before exposure — ARVs taken just a couple of hours before exposure didn’t have much effect. Bang goes my dream of earning millions with an Ecstasy/ Viagra/ Tenofovir combination pill for big nights out. Maybe I’ll just have to settle down and get a real job.

Thanks to Roger for prodding me to spend my day at a virtual conference…

Here’s the real shocker in my opinion: Only one woman in 10 used the gel all the time. Overall, researchers estimated that women in the study used the gel in 44% of all sex acts. For years, we’ve been talking as if women were somehow better than men, somehow holier, somehow more likely to preserve their lives if given the chance. If only we had a “female controlled method” to prevent HIV and didn’t have to rely on those nasty men to use condoms, all would be well, we’ve lulled ourselves into believing. And then you give women a relatively unobtrusive product that may well protect them from HIV, and most of them don’t bother to use it most of the time.

Another disappointment was the sheer scale of infection. Over three women in every 100 became infected with HIV during the study, with no difference to speak of between the those who were assigned the candidate microbicide and those assigned a harmless but definitely ineffective gel. (Nerd Note: incidence was 3.7/100 pyo in the placebo group and 3.3/100 pyo in the microbicide group, p > 0.5). Let’s repeat that: 3.5 new infections per 100 women per year. That’s a higher rate of new HIV infections than we find among drug injectors in Bangkok, for God’s sake (and Bangkok is not exactly renowned for its effective HIV prevention for junkies). It is painfully high by any standards, and in one site the rate was close to six percent. But this isn’t what is going on in South Africa as a whole. This is what’s going on in a study where women were given fistfulls of condoms, and all were energetically urged to use them by carefully trained and highly motivated counselors. On top of that, they were screened for other sexually transmitted infections, and were treated with world-class medicines if need be, thus greatly reducing the likelihood that they’d get HIV if they did have sex with an infected partner. All the participants knew from the start that there was a 50:50 chance they might be using a “fake” microbicide, a gel which could not prevent HIV. In other words, these women had HIV prevention services that most women in South Africa could only dream of, even without the microbicide. As South Africa’s Treatment Action Campaign notes in its very sensible analysis, the rates in women without those services are likely to be higher still. That ought to be a real disappointment to everyone.

*Update*
This comment from Naomi Rutenberg, who heads the Pop Council’s HIV research programmes:

The news is disappointing and the context is dismaying. The good news is that we doubled condom use and made a serious dent in STIs for women in the trial. Also encouraging that there were fewer infections in Carraguard arm. It is important and good news that women who were in the trial ­benefited and were in no way harmed.

Carraguard is safe and we can add an antiretroviral and try again. Adherence is a challenge but 30% of women used it all or 3 of 4 times. There is most definitely a need and a gel certainly is attractive and feasible for some women. (We’re) also working on a vaginal ring.

For details about condom use, STIs, and product safety, see here. Admirably, the Population Council also posts a synopsis of the study protocol.