10/02/09

Significant progress in HIV prevention

Halleluliah! We’ve finally got something to be happy about in HIV prevention — a microbicide that cuts the risk of HIV infection by a third. You’d think everyone would be shouting for joy. But no, we’re bending over backwards to say we’re not sure it works.

The product in question is Pro2000 gel, and the results of the first large trial on more than 3,000 women were reported yesterday at the Conference on Retroviruses and Opportunistic Infections. CROI is all a scientific conference should be (and all the biannual AIDS Circus is not), and you can see and hear every presentation online. The results of the Pro2000 study show that women using the gel were 30% less likely to become infected with HIV than women using a placebo, and a third less likely than women using nothing at all. The reason the researchers are not screaming about it more joyfully is that the results are “not statistically significant”. Meaning, in this particular case, that we can only be between 90 and 94% sure that the difference in infection rates were really the results of the gel, and not the results of pure chance.

This is just silly. If I told you that there was a 94% chance that the red car was a third more likely to crash than the blue car, which would you drive? Yet we’ve managed to establish a norm in the scientific community that only differences that have a 95% probability of not being due to chance can be trusted. For nerds, that means a “p value” of five percent or less is sacrosanct: (p <0.05) has become a talisman of good science. I’m not the first to remark that things can be significant without being statistically significant — economist Tim Hartford wrote a column on statistical significance and Guinness in the FT only last week.

Someone at the conference remarked that “none of us in this audience worship at the alter of the p value of point oh five” but in fact, many of us do. Another thing that researchers at CROI have been bending over backwards to do is to prove that people on ARVs don’t have more risky sex than people not on ARVs. (Aside: this completely misses the point about “behavioural disinhibition” — jargon for “Oh look! HIV won’t kill me! Let’s party!”. What matters is not so much what infected people do once they are on meds, what matters is what uninfected people do because they no longer see any visible connection between unprotected sex and death. Still, people feel the need to show that ARVs don’t make you screw more.) So when a group working in Uganda showed that people on ARVs were 70% more likely to have an extramarital partner than people not on ARVs, they were happy to worship at the alter of the p value of point oh five. In this case, the p value was 0.09 — in other words there was a greater than 90% chance that the differences were real, but researchers were able to say there were “no differences”. We worship from the underside of the alter, too. A larger study looking at ARVs, risky sex and HIV transmission found that unprotected sex was “significantly lower” in those on ARVs. In fact, 17% of those on ARVs reported unprotected sex compared with 19% of those not on ARVs. The difference may have been statistically significant, yes, but does it meet the most important test of significance, the “So What?” test? Almost certainly not.

Epi-rant over. The microbicide trial (and the fact that there is very low transmission from people on antiretrovirals to their partners in the two ARV studies I’ve just ranted about) wasn’t the only good news at CROI today. Giving monkeys antiretrovirals before exposing them to SHIV rectally worked pretty well, too, which bodes well for PrEP in humans. Disappointingly, though, it worked best when the drugs were given between a week and a day before exposure — ARVs taken just a couple of hours before exposure didn’t have much effect. Bang goes my dream of earning millions with an Ecstasy/ Viagra/ Tenofovir combination pill for big nights out. Maybe I’ll just have to settle down and get a real job.

Thanks to Roger for prodding me to spend my day at a virtual conference…

This post was published on 10/02/09 in Pisani's picks, Science.

Send this post to a friend Send this post to a friend

7 comments

You can follow the comments on this post via this RSS feed.

Tags: , , , , , , , .

  1. Comment by Lindsey, 11/02/09, 02:35:

    Exciting news! I can’t wait to see where this all goes. Reducing transmission by 30% would be a miracle. I heartily agree with you on the point about statistics — statistically significant or not, that research should get out there now. Do it again soon, sure, but for now, people should have access to this information.

    (Aside: In the interests of my OCD, I should mention that I think you were referring to the statistical ‘altar’, not ‘alter’.)

  2. Comment by Roger, 11/02/09, 10:43:

    You know I can’t comment! I can only comment on the fact that I can’t comment!

  3. Comment by Roger, 11/02/09, 09:45:

    I can’t comment but I can explain the stats to the lay people:

    What HPTN035 showed is a 30% reduction in HIV infection in the group of women using the gel with a p-value of 0.1.

    In plain English this means that if your risk of being infected by HIV is about 1% then by using the gel, you have 90% of chance of reducing your risk by 30% that is down to 0.7%. A women who uses the gel still had 10% of chance of being infected as if she was not using the gel. As AVAC puts it:

    “The study data suggest that in this trial the group of women who used PRO 2000 had lower rates of HIV than groups of women who used the placebo gel, BufferGel or no gel at all. Many people will ask: how does this finding translate into benefit for an individual woman? This is an excellent question, and one of the reasons why follow-up research is needed. More information is needed to confirm the observed risk reduction in HPTN 035 and also to learn about patterns of gel use, numbers of sex partners, and rates of risk behaviour in different groups of women.”

    90% of chance to be efficient does not translate into 90% efficiency.

  4. Comment by Paul, 15/02/09, 07:48:

    I like your approach to complex problems. Very pragmatic. Science and theory take you so far, then the ground conditions of people’s actual situation and behavior has to be factored in. I trained as an engineer. Trained to take theory and calculations as far as they would go, then use experience to modify the result to something that would actually work. We call them ‘fudge factors’, ‘efficiency parameters’ etc. Scientists get paid if it looks good in a paper, engineers only get paid if it works in service. I think the frustration I detect in the post above puts you in the social engineer camp.
    I agree that a 30% reduction in transmission is significant, very significant, as I dont think we’re going to control HIV with a silver bullet. It is such a slippery little sod that changes its coat more times than Madonna in a concert, we are going to have to strangle the bugger in increments by reducing transmission. If it works…..do it!
    I was facinated by your book. By your easy acceptance of a whole range of sexual behaviour, which I can only begin to accept and come to terms with in my later years. By your honestly subjective approach where you interacted with the strata of society you were dealing with, rather than a distant observer like a satellite in orbit. Perhaps most of all by your respect for people that deserved respect, even though most of society denigrates them. That is whores, male, female and others.
    Cheers, Paul

  5. Comment by Lee Rudolph, 16/02/09, 05:20:

    “Perhaps most of all by your respect for people that deserved respect, even though most of society denigrates them.”

    Huh? Where was Elisabeth being so respectful of bankers?

    “That is whores, male, female and others.”

    Ooops. I should have kept reading.

  6. Comment by Paddy, 18/02/09, 11:20:

    In defense of a cautious interpretation of this, the overall result was as follows:

    HR 0.70 (95% ci 0.48-1.08), p=0.10

    This means, as you say, that there appeared to be a 30% reduction; however, the confidence interval stretches from a 52% reduction to an 8% increase, so we really don’t know what the actual effect is, although we should know a lot more when the main trial’s results are released (due later in 2009). Also, a p of 0.1 can be expected one time in ten by pure chance, and I’d guess we’ve had about that many looks at different microbicide candidates so far. The suggestion of an effect is very encouraging, but we can’t quite start celebrating yet. (I’m hopeful though, partly because PRO-2000 has very decent lab work also in support of a beneficial effect, and partly because no effect was seen on other STIs which suggests they didn’t randomly get people who were behaving differently sexually). And obviously, all congratulations are due to the people who ran this trial (especially for achieving 94% follow-up), and the good safety results are also very welcome.

    Where do we go from here? If PRO-2000 is shown to work about this much in the main trial, it still won’t be completely clear how much it would be worth as an intervention. Small effects of this kind under trial conditions don’t always translate well to general roll-out. Also, we only have evidence for it working in the prevention of transmission to women heterosexually so far; I don’t know what proportion of the 2.7m estimated new infections per year this accounts for off the top of my head, but whatever that is, this would undoubtedly be even better should it also protect against female-to-male transmission in penile-vaginal sex and should it also be effective for penile-rectal sex. However, this seems to be a very good proof of concept, and what I’d personally like to see next would be a trial of PRO-2000 vs PRO-2000 with added ARVs (there are good early indicators for the use of Tenofovir in a microbicide iirc), alongside trials of its use rectally, and possibly for PMTCT where a caesarian section cannot be carried out (it could in theory be provided at ANC together with the ARVs for PMTCT, &/or via local midwives &/or traditional birth attendants). Does this make sense, do you think?

  7. Comment by Paddy, 20/02/09, 12:46:

    Further on this story – it’s somewhat dispiriting how wrong avert.org have got it. Their news article (at http://www.avert.org/aidsnews.htm#news2) currently reads as follows:

    “New trial results show vaginal gel not effective in HIV prevention
    February 11, 2009

    Results from a study released at the 16th Conference on Retroviruses and Opportunistic Infections in Montreal, have shown that the effectiveness of a vaginal gel called PRO 2000 is not yet known.

    Despite reports that the trial shows promising results in the effectiveness of the potential microbicide, the results are not statistically significant. The phase 2b trial, which was conducted on 3099 women across four countries in Africa and one site in America, shows that there was a one-in-ten probability that the 30% reduction in HIV infection seen in those who used the gel was due to chance.

    While the study does not prove the vaginal gel to be effective in preventing HIV transmission, it does suggest that more trials need to be conducted in order to conclusively determine if PRO 2000 could be a potential microbicide. Salim Abdool Karim, who presented the results, said that there were no plans at the moment to move forward with further trials of the gel.”

    I’ve just sent them a snotty email about how absence of evidence is not evidence of absence, as well as pointing out that Slim said no such thing and in fact the phase 3 study results are due out later this year. Hope they take the point and pull/fix this.

Comments are closed at this time.