First RCT of Lopinavir–Ritonavir on SARS-CoV2

journal club
, ,
1

https://www.nejm.org/doi/full/10.1056/NEJMoa2001282

The abstract:

Methods

We conducted a randomized, controlled, open-label trial involving hospitalized adult patients with confirmed SARS-CoV-2 infection, which causes the respiratory illness Covid-19, and an oxygen saturation (Sao2) of 94% or less while they were breathing ambient air or a ratio of the partial pressure of oxygen (Pao2) to the fraction of inspired oxygen (Fio2) of less than 300 mm Hg. Patients were randomly assigned in a 1:1 ratio to receive either lopinavir–ritonavir (400 mg and 100 mg, respectively) twice a day for 14 days, in addition to standard care, or standard care alone. The primary end point was the time to clinical improvement, defined as the time from randomization to either an improvement of two points on a seven-category ordinal scale or discharge from the hospital, whichever came first.

Results

A total of 199 patients with laboratory-confirmed SARS-CoV-2 infection underwent randomization; 99 were assigned to the lopinavir–ritonavir group, and 100 to the standard-care group. Treatment with lopinavir–ritonavir was not associated with a difference from standard care in the time to clinical improvement (hazard ratio for clinical improvement, 1.24; 95% confidence interval [CI], 0.90 to 1.72). Mortality at 28 days was similar in the lopinavir–ritonavir group and the standard-care group (19.2% vs. 25.0%; difference, −5.8 percentage points; 95% CI, −17.3 to 5.7). The percentages of patients with detectable viral RNA at various time points were similar. In a modified intention-to-treat analysis, lopinavir–ritonavir led to a median time to clinical improvement that was shorter by 1 day than that observed with standard care (hazard ratio, 1.39; 95% CI, 1.00 to 1.91). Gastrointestinal adverse events were more common in the lopinavir–ritonavir group, but serious adverse events were more common in the standard-care group. Lopinavir–ritonavir treatment was stopped early in 13 patients (13.8%) because of adverse events.

Conclusions

In hospitalized adult patients with severe Covid-19, no benefit was observed with lopinavir–ritonavir treatment beyond standard care. Future trials in patients with severe illness may help to confirm or exclude the possibility of a treatment benefit. (Funded by Major Projects of National Science and Technology on New Drug Creation and Development and others; Chinese Clinical Trial Register number, ChiCTR2000029308. opens in new tab.)

First, I greatly admire everybody doing trials in this difficult situation and am very grateful to the study authors, I imagine running the study was not easy. My goal is however to help clinicians understand what to take from the study and I believe that the wording of the conclusions could have been better.

For almost all of the reported outcomes (Table 3), the value observed in treatment group was better than in control and the confidence intervals span from some deterioration (e.g., +6 percentage points mortality rate, 0.95 hazard ratio for clinical improvement ) to quite big improvements (e.g. -17 percentage points mortality rate, 1.85 hazard ratio for clinical improvement). I would therefore summarise the results as:

“For severe Covid-19 patients, the study observed a small benefit in using Lopinavir–Ritonavir, but neither some deterioration nor relatively large improvement can be ruled out. We can with some confidence rule out that the drug results in large harm and that the drug is highly effective against SARS-CoV2”.

Further considerations: the study was open label, there were some side-effects of the medication.

The strongest argument against the clinical usefulness of Lopinavir–Ritonavir is IMHO that throughout the study, the treatment group has only very slightly smaller viral loads than the control group (Figure 3), despite already starting with lower viral load at baseline. Not sure how seriously to take this though.

How would you phrase your summary of the study to practitioners? Any other important considerations about the study?

Thanks for any input!

6 Likes
2

The sample size was calculated based on a difference of median 8 days until clinical improvement between treatment and control. So their sample size was not large enough to detect smaller differences and other end points such as mortality.

4

Their primary outcome was a bit weird (time to “clinical improvement” = discharge or improvement of 2 categories or more on a 7 category ordinal scale).
So the “event” was a bit contrived - not clear that improvement of 2 categories is the same for all starting points. In fact it certainly isn’t. Also I don’t think they ever analysed the 7 category ordinal outcome properly with ordinal regression models - it was dichotomised into “percentage with clinical improvement at day 14” (which favoured intervention 45.5% versus 30%).

4 Likes
5

Well stated. As always, such ordinal endpoints are crying out for ordinal models which avoid highly problematic change scores!

2 Likes
6

Please share the raw patient level data please!! Especially including the day lopinavir-ritonavir was initiated.

2 Likes
7

There have been over 200,000 COVID19 infections and a study on 199 is seen as adequate enough to publish in NEJM. We need large simple trials. Cluster randomize hospitals on a crossover basis. Get hospitals to agree to treat patients with a given Rx each week, randomize centrally, call each week with allocation. data from routine care (e.g. mortality)

3 Likes
8

Thanks for all the pointers to potential limitations. I would however prefer if you could instead focus on what do you think can be learned from the study as is, despite the flaws (I am open to the possibility that the answer is “very little”). Clinicians unfortunately don’t have the option to pause treatment choices until a better study is conducted. Thanks a lot!

3 Likes
9

I think that the conclusion should indicate that there it is still plausible that L-R is beneficial - look at the CI for the Hazard Ratio for the primary outcome. (by the way, the Primary outcome HR in the Abstract and in the Text are slightly different:
*ABSTRACT: hazard ratio for clinical improvement, 1.24; 95% confidence interval [CI], 0.90 to 1.72
TEXT: hazard ratio for clinical improvement, 1.31; 95% confidence interval [CI], 0.95 to 1.85; P=0.09
*
I’ve asked NEJM to clarify.

3 Likes
10

Sure; but we could learn more from it with better analysis that used the information more efficiently.

I’d say we don’t learn a huge amount from it. The data are consistent with the intervention having some benefit, so I wouldn’t write it off based on these data alone. I’ve seen it said elsewhere that these drugs are unlikely to be harmful so if that’s right, maybe we should put more credence on the potential benefits. The very black and white interpretations in the NEJM (and in lots of commentary - a “negative trial”) seem unhelpful. A trial of this size was never going to definitively tell us whether there was any benefit, so it’s important to look at the results with that in mind.

11

Unfortunately, I’ve heard some messages from doctors that have stopped treating with these drugs based on this specific trial as of yesterday. So regrettably, some clinicians are changing their behaviour over this already.

4 Likes
12

Another aspect that I find problematic is that they do not adjust their analysis on any patient covariate. And given the results they have it is likely that such adjustments would make the results significant. Just from listening to the media, age and comorbidities seem like a safe bet as important prognostic covariates. And I’m sure that there are more appropriate covariates to consider. They do not even adjust for their stratification factors:

randomization was stratified on the basis of respiratory support methods at the time of enrollment: no oxygen support or oxygen support with nasal duct or mask, or high-flow oxygen, noninvasive ventilation, or invasive ventilation including ECMO.

and it is known that stratifying on a covariate and then not taking it into account in the analysis leads to deflated type 1 error and an underpowered analysis.

4 Likes
13

I abhor this sentiment. We need small, theoretically substantive trials capable of advancing medical science rapidly in a mode of strong inference. We do not need large, theoretically shallow trials that grope blindly* for small-effect-size metaphorical ‘oomphs’ that lack realistic theoretical connections to physiology.

  • Cf. #amoEBMism:
3 Likes
14

Hell, we need large observational datasets NOW for hypothesis generation.

4 Likes
15

In Israel , two tertiary hospitals have stopped using L-R due to interpreting the results as " no added benefit". I hope some official reply or letter to the editor maybe published so that clinicians would not stop a potentially helpful treatment based on wrong interpretation.

2 Likes
16

We have digitized the curve, and reanalyzed it with a Bayesian Cox model, under a weakly informative prior. The posterior probability of effect (clinical improvement with HR>1.15) is 73%, while the probability that the therapeutic effect is in the region of practical equivalence (ROPE) is 17%. I would still use lopinavir-ritonavir in my patients, until more evidence is available.

11 Likes
17

We have digitized the curve, and reanalyzed it with a Bayesian Cox model, under a weakly informative prior. The posterior probability of effect (clinical improvement with HR>1.15) is 73%, while the probability that the therapeutic effect is in the region of practice equivalence (ROPE) is 17%. I would still use lopinavir-ritonavir in my patients, until more evidence is available.

Would love to see this in medRxiv.

4 Likes
18

I have submitted a short letter to the editor, as an urgent plea to the researchers to use Bayesian methods in these critical times. I don’t think they’ll accept it but I had to try.

6 Likes
19

Any update on this? I feel like this would be a good opportunity to learn how to do this.

2 Likes
20

The original curve from the Lotus-China trial may be scanned using free software such as:

https://apps.automeris.io/wpd/

The database is then reconstructed according to the method by Guyot et al.


This method is available in the R survHE package.
It’s simple but it takes a little practice.
Then, you have an approximate database very similar to that of the original study. Below I show my approximate plot.

curva edu
curva edu901×769 28.3 KB

Quite similar to the original plot here:

original
original1143×860 113 KB

In fact, my frequentist analysis yields the same result as the original.

Once you’ve done this, you can do a Bayesian reanalysis. In this case I fitted a Bayesian Cox model with the R brms package. Below I show the resulting half-eye plot. You can see that most of the probability mass in favor of the therapeutic effect tilts to the right.

halfeye
halfeye1421×685 17.4 KB

Therefore, the posterior probability that the effect is in the region of practical equivalence (+/- 10%) is 17%. There is a clear directionality, and according to the frequentist statistical plan used by the authors, the study was underpowered, as it was stopped prematurely.

7 Likes
21

That’s for their time to event primary outcome, right? It’s a bit of a strange outcome. It would be interesting to analyse the 7-category ordinal outcome (at 7 and 14 days) using an ordinal regression (preferably Bayesian) - I think there is enough information in the paper to do that…

1 Like