Two statisticians involved in the PROUD and iPrEx trials of pre-exposure prophylaxis (PrEP) warn that future trials to test new PrEP drugs and formulations may be extremely difficult to design.
David Dunn of the UK Medical Research Council and David Glidden of the University of California, San Francisco say that statisticians will need to choose and analyse their trial population very carefully if they wish to be able to demonstrate meaningful results.
While pre-exposure prophylaxis (PrEP) has been remarkably effective in preventing HIV, both in scientific trials such as PROUD and Ipergay, and in public availability programmes in the US, there is a general consensus that alternatives to tenofovir + emtricitabine (Truvada) need to be found. This is the only drug combination licensed for PrEP in the US and some other countries, and under consideration for licensing in Europe.
Truvada can cause short-term initial side-effects such as nausea that may put off people wanting to take it, and while long-term serious side effects are uncommon, studies do document slight, reversible declines in kidney function and also in bone mineral density in people taking PrEP. There is already a new formulation of tenofovir, tenofovir alafenamide or TAF, that produces lower rates of these side effects and is already licensed as part of an HIV-treatment combination pill. It is known that manufacturers Gilead are keen to test TAF + emtricitabine as PrEP.
In addition, HIV resistant to both the drugs in Truvada does occur, though rarely, and arises when people start Truvada soon after becoming unwittingly infected with HIV, though this appears to fade over time.
However the most urgent need for formulations of PrEP other than Truvada is that while knowledge of the effectiveness of PrEP has spread, certain groups – young women in some African trials, and young gay men and black men in the US, for instance – still struggle to adhere to Truvada as PrEP, taken either daily or intermittently. For these groups of people and others in situations that may make PrEP adherence difficult (homelessness, poor mental health, incarceration) other drugs and, especially, long-acting drugs including injectable formulations already studied for treatment may be needed to protect them from HIV. We also have no direct data for the efficacy of Truvada as PrEP for people who inject drugs, only for solo tenofovir, and that comes from a trial with hard-to-interpret results.
It may not even be a new drug or formulation we need to test. We also need to test Truvada against itself in the form of different regimens – such as intermittent versus daily dosing. So far, we only have the result from the placebo-controlled Ipergay trial – which found that intermittent PrEP was effective in a high-incidence population, but didn’t compare daily with intermittent PrEP. We await final effectiveness figures from the three branches of the ADAPT study, but it will be difficult to get a final answer from a study with only 180 participants per site.
The problem with devising trials of new PrEP is the very effectiveness of Truvada. Firstly, this now makes it not only unethical, but also scientifically meaningless, to do a placebo-controlled trial in which one half of the group receive dummy pills instead of the new drug – meaningless because the new PrEP drug should be tested against the best alternative, which is now Truvada rather than other forms of prevention.
However directly comparing the new pill with Truvada might require an unfeasibly large trial. Dunn and Glidden cite a design pitting Truvada against a drug which in reality has exactly the same effectiveness. This, of course, would not be known beforehand – that’s the point of doing the trial. The trial design used would be a ‘non-inferiority’ one, in which the object is to find out whether the drugs are equivalent to each other in terms of effectiveness, or whether one is clearly better. The researchers define that the new product would be defined as ‘superior’ or ‘inferior’ to Truvada if it was over 25% more, or less, effective.
If this trial was conducted in a population with a fairly typical annual HIV incidence (infection rate) of 2.25%, which is roughly equivalent to the annual incidence seen in UK gay men attending STI clinics who’ve had an HIV test within the last year, then it would require a two-year trial with 19,000 participants to be able to rule out for sure that the new product was 25% more or less effective than Truvada – clearly impractical. If you wanted to rule out that it was 10% more or less effective, you would need an even larger trial. A smaller trial would be required if there were advance indications that the product was very much superior to Truvada but, given the already high biological efficacy of Truvada, this is quite unlikely and would depend on the product being much easier to adhere to for the trial population.
There are ways of getting round this. The first is to select your trial population from groups that have clearly found it difficult to adhere to Truvada. This is the hope of researchers trialling injectable formulations, where the adherence is to quarterly clinic appointments rather than daily pill-taking.
However there are statistical ways round it too. Non-inferiority trials report something called a hazard ratio – which is the proportional or percentage difference between the effectiveness of one drug and another.
For a public health prevention tool like PrEP, an absolute rather than proportional measure called the rate difference is more important. This is simply the absolute difference between the number of HIV infections in people in different arms of the trial, rather than the proportion. Because even in a high-incidence group, the majority of the trial population would not have caught HIV anyway, the rate difference is often a numerically larger and more relevant figure, especially as it translates directly into the most important figure of all – the NNT or Number Needed to Treat, which says how many people would need to be given PrEP in order to prevent one HIV infection. This is the figure that tells you how cost-effective PrEP might be.
However, Dunn and Glidden add, in order to put this rate difference into its true perspective, one needs a third piece of information – the ‘background incidence’ or what the HIV infection rate in your trial population would have been if they had not been offered PrEP of any kind. In the placebo-controlled trials one could assume that the background incidence was that seen in the placebo group – though in practice, that was not always the case.
In a trial without a placebo arm, however, establishing background incidence is going to be crucial but difficult. We may need several years’ data on the population in whom we intend to do the trial and even then we cannot be certain that the people who eventually consent to take part in the trial reflect the population from whom they are selected.
For instance, let’s look at a situation in which a new-experimental drug or regimen in fact turns out to be less effective than Truvada – which could easily be the case. If in a two-year trial with 5000 participants, 15 HIV infections were seen in participants on the new regimen and 8 on Truvada, then it looks like Truvada is nearly twice as effective.
Compare it, however, with two different background incidences. In one, the background rate is 0.4% a year, meaning that 20 infections would be expected in a year in people not taking any PrEP at all. In this case, the 15 infections on the new drug and the 8 on Truvada look like a huge difference – 25% versus 60% effectiveness – a hazard ratio of 2.4.
In the other study, the background incidence rate is 4% a year, meaning that in 5000 people one would expect 200 HIV cases without PrEP. That then means that the new drug is 93% effective and Truvada is 98% effective – a hazard ratio of 1.03, which is statistically indistinguishable from 1.0, i.e. no difference.
But the rate difference in both trials between the experimental drug/regimen and Truvada remains the same in both trials – 0.14%, or 1.4 cases of HIV per 1000 patients per year. That is the number of extra infections per year you would see if you used your new drug/regimen/formulation instead of Truvada. If the new formulation had other advantages such as cost, availability or a better side effect profile, then you might consider it worth using.
As we said, it is going to be difficult to establish a reliable figure for background incidence. One could use a historical control, such as incidence in the placebo arm of a previous study, as in the Partners PrEP demonstration study, but incidence changes over time and a suitable historical control group may not be available.
In some situations one could simply use the proportion of patients among people applying for the trial who are diagnosed on screening, but this would not work in situations where people test frequently.
The only other alternative is to try and work out the background incidence from HIV diagnosis figures in the population one hopes will be interested in PrEP – see this report for a quite detailed analysis of this sort conducted in Barcelona.
However, as Dunn and Glidden show, background incidence in the population targeted for a trial has been notoriously bad at predicting actual incidence in the trial. In almost all the early placebo-controlled trials, incidence in the trial was 2 to 4 times lower than that predicted by pre-trial surveys. In PROUD and Ipergay, the opposite was the case and incidence in the control arms was 3 to 4 times higher than predicted.
In the former case, as well as being cautious because they knew they might be on a placebo, participants may have joined at a period of particularly high risk and then ‘reverted to the mean’: their risk declined to what it had been more typically. In the latter case, standard HIV tests may fail to detect recent infections and thus underestimate recent incidence.
This introduces another idea: in the PrEP trials so far, the effectiveness figure given has excluded cases where people caught HIV immediately before starting PrEP, during their ‘window period’ when no test can detect HIV. This is because there is no way PrEP could have stopped these people getting HIV. There were three cases like this in PROUD.
But there is no perfect way to eliminate every person who may just have caught HIV when they come forward for PrEP and testing requirements in rollout programmes are unlikely to be as elaborate as they are in trials. For this reason, Dunn and Glidden say, cases like this should be henceforward included in realistic estimates of PrEP’s effectiveness within widespread provision. In the PROUD study, for instance, this would have changed the effectiveness seen from 86% to 78%.
“Future trials of PrEP are highly challenging to design,” conclude Dunn and Glidden. “New statistical paradigms for non-inferiority trials are required, with statisticians and expert clinicians working closely to develop these.”
Dunn DT and Glidden DV. Statistical issues in trials of preexposure prophylaxis. Current Opinion in HIV & AIDS 11(1), 116-121. 2016.
