Proportional Odds Analysis for 4/29 Lancet Remdesivir Study

Question about statistical analysis for ordinal outcomes, baseline differences, and the proportional odds assumption.

Lancet published a much awaited study on use of Remdesivir for Covid19. This was a randomized (2:1 allocation ratio), double-blind, placebo-controlled trial (n=237):
https://www.thelancet.com/journals/lancet/article/PIIS0140-6736(20)31022-9/fulltext

The primary outcome was “time to clinical improvement within 28 days after randomization.” Clinical improvement was defined using a 6-point ordinal scale:

  • 6 = death
  • 5 = hospital admission for ECMO or mechanical ventilation
  • 4 = hospital admission for non-invasive ventilation or high-flow oxygen
  • 3 = hospital admission for regular oxygen therapy
  • 2 = hospital admission but not requiring oxygen
  • 1 = discharged or having reached discharge criteria

The statistical analysis, briefly, was as follows:

  • “The primary efficacy analysis was done on an intention-to-treat (ITT) basis with all randomly assigned patients. Time to clinical improvement was assessed after all patients had reached day 28; no clinical improvement at day 28 or death before day 28 were considered as right censored at day 28. Time to clinical improvement was portrayed by Kaplan-Meier plot and compared with a log-rank test. The HR and 95% CI for clinical improvement and HR with 95% CI for clinical deterioration were calculated by Cox proportional hazards model.”

Results:

Question: (Edited)
I am trying to understand why the outcomes appear to shift for the worse at Day 7, and then for the better at Day 14.

  • Is this just normal “noise” that we see in data early on in a trial?
  • Is it possible chance handed Remdesivir a slightly sicker group of patients at baseline?
  • Does Cox proportional hazards model take into account baseline differences?
  • Does this violate the proportional odds assumption?
  • Does this effect the optimal choice for a statistical model?
  • Is it possible the choice of outcome scale flawed?

My initial assumption is that this is just expected noise/variation in data at the early point of the study. But I thought I would ask others so they could offer expertise on how to best interpret & learn form this example.

Thanks!
Raj

3 Likes

worth noting clinical improvement defined as: “a decline of two levels on a six-point ordinal scale of clinical status (from 1=discharged to 6=death) or discharged alive from hospital, whichever came first”, also phrased as: "a two-point reduction in patients’ admission status on a six-point ordinal scale, or live discharge from the hospital, whichever came first. "

i guess they mean 2 levels or more, although im not sure what decline of 2 levels means on the scale you give above

1 Like

Angst. There are severe problems with this endpoint. See here.

3 Likes

One imbalance in risk factors from table 1 suggesting to me that sicker patients were in the placebo arm:
Hospitalized, needing invasive ventilation or ECMO.
-Remdesivir 125 (23%) | Placebo 147 (28%)

My understanding is that this risk factor is more substantially associated with death than any other.

I don’t see how it’s valid to compare baseline distributions in a randomized study. Plus you’ll easily find counterbalancing factors.

Highlighting the source that prompts my concern for persons requiring mechanical ventilation: “Of those receiving mechanical ventilation, 17% (276/1658) were discharged alive, 37% (618/1658) died, and 46% (764/1658) remained in hospital.

Features of 20 133 UK patients in hospital with covid-19 using the ISARIC WHO Clinical Characterisation Protocol: prospective observational cohort study https://www.bmj.com/content/369/bmj.m1985

Results The median age of patients admitted to hospital with covid-19, or with a diagnosis of covid-19 made in hospital, was 73 years (interquartile range 58-82, range 0-104). More men were admitted than women (men 60%, n=12 068; women 40%, n=8065). The median duration of symptoms before admission was 4 days (interquartile range 1-8). The commonest comorbidities were chronic cardiac disease (31%, 5469/17 702), uncomplicated diabetes (21%, 3650/17 599), non-asthmatic chronic pulmonary disease (18%, 3128/17 634), and chronic kidney disease (16%, 2830/17 506); 23% (4161/18 525) had no reported major comorbidity.

Overall, 41% (8199/20 133) of patients were discharged alive, 26% (5165/20 133) died, and 34% (6769/20 133) continued to receive care at the reporting date.

17% (3001/18 183) required admission to high dependency or intensive care units; of these, 28% (826/3001) were discharged alive, 32% (958/3001) died, and 41% (1217/3001) continued to receive care at the reporting date.

Of those receiving mechanical ventilation, 17% (276/1658) were discharged alive, 37% (618/1658) died, and 46% (764/1658) remained in hospital.

I had a question regarding the clinical interpretation of the common odds ratio. This is a segment of Table 3, with the results of the ordinal scale at day 28:

I realize that the CI for the common odds ratio is very wide, but let’s just assume the point estimate is valid for argument purposes. I think I would interpret it as follows:

  • If a patient started treatment requiring supplemental oxygen (3 on the scale), he has a 1.15x higher odds of being at a 2 on the scale 28 days later (being hospitalized, but not requiring supplemental oxygen).

How would the common OR change if I wanted to estimate the odds of a patient improving by > 1 point on the scale? Taking the example above, what are the odds the patient goes from a 3 to a 1 (discharged from the hospital)?

Also, @f2harrell mentioned how there is a major problem with the primary endpoint of clinical improvement.

This seems to be the main problem with ACTT-1 study as well right? The difference being that the common odds ratio in that study showed a significant benefit in favor of remdesivir.

As an aside note that only using the 28d result results in a power loss.

The OR doesn’t have to do with within-patient changes. If the OR is not covariate adjusted for initial ordinal state, then the interpretation is the ratio of odds for treatment B : A of being in outcome category y or worse, for any y other than the first. If it is covariate adjusted, then the OR responses the same ratio of odds but for a subject on treatment B with initial state x being compared to a subject on treatment A who is in the same initial state x.

1 Like