28/10/09

HIV vaccines: good news or bad?

tatoo_test

A month ago, the media got very excited about an HIV vaccine. Study results, released in Thailand with a maximum of fuss and a minimum of detail, showed that the two-step vaccine might protect about a third of the people who get the shots against HIV. Then the doom-mongers weighed in: without more information, we might be overestimating the effects of the jabs. So, is the syringe half full or half empty?

Now more details have been released, we’re still looking at half a glass. Hurrah! said some reports. The vaccine really is protective. Boo hoo, said others. Unless you torture the statistics, they don’t confess to much of an impact. I finally got around to combing through the full report of the trial in the New England Journal of Medicine. Both the optimists and the pessimists are right. It really depends on what your hopes and expectations were. If you are a basic scientist (as most of the people involved in the study were) you’d be pretty thrilled by the results, because they show that vaccines might one day work. If you are a public health boffin such as myself, you’d be pretty disappointed, because the study suggests that that this vaccine doesn’t work for the people who really need it — a point much underplayed in the official reports.

In their full paper the research team reported three sets of results for this study among young men and women in Northern Thailand; only the most optimistic of these was reported to the press in the initial release of results a month ago. Keith Alcorn of Aidsmap has produced a typically sound and balanced summary of the paper if you want more details. But here’s my more opinionated take.

Analysis 1: Real World.
This is technically known as an “intention to treat analysis”. The final analysis includes everyone who was enrolled in the study, regardless of whether or not they followed all the procedures correctly. This is the analysis which is most interesting to public health boffins, because it comes closest to showing how things might happen in the messy reality of life, where people forget to show up for appointments, get given the wrong dose by mistake, etc. In this analysis, people in the vaccinated group were 26.4% less likely than those in the unvaccinated group to get infected with HIV. There was a 92% probability that this difference was not due simply to chance; that means that the difference would not be considered “significant” by those who cleave to the mystical figure of 95% to dictate what is or is not worth considering. Taking into account random differences between the people assigned to the vaccine and the placebo groups, the researchers were 95% sure that the real effect of the vaccine did something between making you 48 percent less likely to contract HIV, and making you four percent more likely to get infected.

Analysis 2: Ideal World.
Known as the “per protocol analysis”, this looks only at the people who got all their shots on time, in the right doses. This is more or less the human equivalent of doing things in lab conditions, and is the sort of analysis that is most useful for basic scientists. Only three quarters of all the study subjects qualified. That in itself is worrying to people like me; if we can’t deliver four doses of vaccine to a quarter of the participants in an incredibly well-organised, well-funded study with hugely well-motivated study staff, how the hell are we going to do it in the real world? More worrying to the basic scientists, I would have thought, is the fact that in this sub-population of people who did everything exactly comme il faut, the vaccine did not have a more pronounced effect (26.2%, with an 16% chance that the effect was due to chance). Because numbers were smaller there was an even wider range that might have reflected the “true” outcome, from increasing infections by 13.3 percent to cutting them by 51.9 percent.

Analysis 3: Tidied-up World
Not a common convention, the “modified intention to treat analysis” essentially reflected the real world with the messiest bits knocked off. In this analysis, the researchers included everyone in the study, except the seven people who it turned out were already infected before their first jab. These people were missed in the initial screening test because they were still in the “window period” during which a person has the virus, but not yet the antibodies which cause a test to show up positive. They were discovered because they had turned positive by their last jab; the team then went back and used a (much more expensive) test for the virus itself on the original screening sample and found that they had already been infected. From a basic science point of view, it makes perfect sense to excluse these people from the analysis; obviously, a vaccine can’t protect people who are already infected. From a public health point of view, it’s debatable whether we should tidy up the data like this. If we put huge national vaccine programmes in place, we’re going to be vaccinating people who are in the window period, especially in the early years, and in groups at highest risk. I’d say we want to take that into account when estimating the potential effect of a vaccine. it was this “tidy” analysis that hit the headlines a month ago, and gave the study its only “significant” result — a 31.2% reduction in HIV infection, with a 96% probability that the effect was not a statistical fluke. This time, we could be 95% sure that the vaccine didn’t make things worse, that it reduced infection by at least 1.1%, and perhaps by as much as 51.2%.

As a public health nerd, I’m most interested in the Real World Analysis. But I’m even more interested in something that’s buried down at the bottom of Table 2.

Click to enlarge

Click to enlarge

Here, the research team looks at the effect of the vaccine on people with different levels of risk behaviour. In a shockingly poor piece of paper writing/ editing, it is not actually possible to tell from the Methods section of this paper how the different levels of risk are defined. But the definition for high risk does seem to include at least some of the usual suspects: needle sharing during drug injection, same sex partners for men, commercial sex etc. And what Table 2 shows is that the vaccine makes no difference at all for those at highest risk. It might cut infection rates by nearly half in that group, or it might increase the chance of getting HIV by nearly three quarters. The best-guess estimate is that it cuts infection rates by under 4% among the people who are most likely to be exposed to the virus. Four percent is as good (or bad) as nothing.

The researchers point out that the study was not designed to look at these differences, but call the results “intriguing”. To an immunologist, they must be. Perhaps the immunity conferred by the vaccine is not strong enough to withstand the repeated assaults suffered by someone who shares needles daily or turns tricks three times a week — I have no idea. But to public health workers, it is not intriguing, it is devastating. If a vaccine doesn’t work for the people who need it most, what’s the point? It depends on costs, of course. But would we really develop something that we could give to people who have a very low probability of exposure, while leaving those who are likely to be at risk for HIV unprotected?

It’s a false dichotomy, of course. This trial is a triumph for basic science, because it gives us something positive to work with. It is very far from being a triumph for public health, and it is not helpful that in the early rush of euphoria it was presented as such. I’d even be wary of the language used by the authors of the NEJM paper: in their headline result, they reported quite wrongly that the study showed that “there was a trend toward the prevention of HIV-1 infection among the vaccine recipients”. A trend is something that develops over time. If anything, the data suggest that the effect of the vaccine was weakened over time, so the trend was away from protection, not towards it. But I’m splitting hairs. With vaccines, the basic science has to be right before we even think about the public health questions. This study will send the immunologists back to the drawing board. They need to figure out how we have a possibly succesful vaccine that makes no difference to viral loads in those who do get infected. They need to understand why people who are most exposed to HIV are least likely to be protected. They need to parse out the mechanism by which these two sets of shots, each of which has failed on its own, might be working together. If (and it is still only an if) they can do all of that, then develop something that really does work, the public health nerds can start worrying about how to deliver it, and to whom.

This post was published on 28/10/09 in Science.

Send this post to a friend Send this post to a friend

6 comments

You can follow the comments on this post via this RSS feed.

Tags: , , , , .

  1. Comment by peripheries, 28/10/09, 03:24:

    The risk behaviour was measured using a list of 8 criteria, two of which were Same-sex gender partner (2.3% vs. 2.2% V:P) and needle-sharing (0.8% in each group) other criteria were:
    – STI symptoms
    – Sex with HIV positive partner
    – No condom use during High-risk encounters
    – Occupation entertainment
    – Occupation CSW
    – Multiple sex partners

    The obvious reason explaining “why people who are most exposed to HIV are least likely to be protected” and why the vaccine “makes no difference to viral loads in those who do get infected” is simply that this vaccine does not work and it was all random and chance.

  2. Comment by Muscleguy, 28/10/09, 05:36:

    I wonder also if the people in the high risk category had lower immune systems because of their lifestyles and hence had a lower titre of antibodies or a lesser cell mediated immune response.

    If this is so then it would make the idea of a vaccine that could protect the high risk groups even harder to do.

    It might still mean it would be worth vaccinating the rest of the population (there is a stark difference in the risk group analysis between medium and high) in the same way that it is more effective to vaccinate children than their elderly grandparents if you want to reduce the incidence of influenza infection in the elderly. If the high risk group was entirely self contained then vaccinating everyone else makes no sense but we know this is not the case and since everyone else cannot tell by sight who is high risk this would seem sensible. Once we have a working vaccine that is.

  3. Comment by elizabeth, 30/10/09, 12:59:

    Recieved from Mark VC, via e-mail:

    C’mon it’s good news: not that this is the vaccine, which it isn’t, but that
    there will, one day, be a vaccine. Both basic scientists and public health
    boffins should be overjoyed about that! For so long it has been an open
    question as to whether such efforts will ever succeed, with a lot of scientists
    saying ‘no’. The Alvac/AIDSVAX combo will now be followed by every
    conceivable permutation of it not already tried, and we’ll get a vaccine that
    protects 50%, or something. Lighten up!

  4. Comment by Daniel Reeders, 01/11/09, 12:40:

    Brilliant post. Does a really nice job of explaining the analytical terms to an everyday reader.

  5. Comment by Steve Smith, 03/11/09, 07:04:

    When you say:
    “We’re endlessly told that HIV is about poverty and underdevelopment and human rights. But in the pink neon of Asia’s red light districts, HIV is mostly about people doing dumb things in the pursuit of pleasure or money. We’re just not supposed to say so.”

    Who’s not supposed to say so? Why? I’ve never heard anybody not talking about people doing dumb things in the pursuit of pleasure or money. Is it an ‘Asia thing’?

  6. Comment by elizabeth, 03/11/09, 06:13:

    Good question. By “we” I mean the General Public. But the implicit question is who is telling us this? And the answer is emphatically the AIDS industry. It is very difficult indeed to stand up at an AIDS conference, surrounded by peers and colleagues who have spent years working to reduce the stigma that has since the inception of the epidemic been attached to HIV because of its association with “unacceptable” behaviours, and say “people get HIV by ignoring what they know and having uprotected sex or sharing needles in any case”. One is usually accused of being stigmatising, and always accused of ignoring the fact that some people have limited or no choice about the behaviours that expose them to the virus.

Comments are closed at this time.