Question about statistical analysis for ordinal outcomes, baseline differences, and the proportional odds assumption.
Lancet published a much awaited study on use of Remdesivir for Covid19. This was a randomized (2:1 allocation ratio), double-blind, placebo-controlled trial (n=237):
The primary outcome was “time to clinical improvement within 28 days after randomization.” Clinical improvement was defined using a 6-point ordinal scale:
- 6 = death
- 5 = hospital admission for ECMO or mechanical ventilation
- 4 = hospital admission for non-invasive ventilation or high-flow oxygen
- 3 = hospital admission for regular oxygen therapy
- 2 = hospital admission but not requiring oxygen
- 1 = discharged or having reached discharge criteria
The statistical analysis, briefly, was as follows:
- “The primary efficacy analysis was done on an intention-to-treat (ITT) basis with all randomly assigned patients. Time to clinical improvement was assessed after all patients had reached day 28; no clinical improvement at day 28 or death before day 28 were considered as right censored at day 28. Time to clinical improvement was portrayed by Kaplan-Meier plot and compared with a log-rank test. The HR and 95% CI for clinical improvement and HR with 95% CI for clinical deterioration were calculated by Cox proportional hazards model.”
- Time to clinical improvement:
- Proportion distribution at Day 1, 7, 14, & 28:
I am trying to understand why the outcomes appear to shift for the worse at Day 7, and then for the better at Day 14.
- Is this just normal “noise” that we see in data early on in a trial?
- Is it possible chance handed Remdesivir a slightly sicker group of patients at baseline?
- Does Cox proportional hazards model take into account baseline differences?
- Does this violate the proportional odds assumption?
- Does this effect the optimal choice for a statistical model?
- Is it possible the choice of outcome scale flawed?
My initial assumption is that this is just expected noise/variation in data at the early point of the study. But I thought I would ask others so they could offer expertise on how to best interpret & learn form this example.
worth noting clinical improvement defined as: “a decline of two levels on a six-point ordinal scale of clinical status (from 1=discharged to 6=death) or discharged alive from hospital, whichever came first”, also phrased as: "a two-point reduction in patients’ admission status on a six-point ordinal scale, or live discharge from the hospital, whichever came first. "
i guess they mean 2 levels or more, although im not sure what decline of 2 levels means on the scale you give above
Angst. There are severe problems with this endpoint. See here.
One imbalance in risk factors from table 1 suggesting to me that sicker patients were in the placebo arm:
Hospitalized, needing invasive ventilation or ECMO.
-Remdesivir 125 (23%) | Placebo 147 (28%)
My understanding is that this risk factor is more substantially associated with death than any other.
I don’t see how it’s valid to compare baseline distributions in a randomized study. Plus you’ll easily find counterbalancing factors.
Highlighting the source that prompts my concern for persons requiring mechanical ventilation: “Of those receiving mechanical ventilation, 17% (276/1658) were discharged alive, 37% (618/1658) died, and 46% (764/1658) remained in hospital.”
Features of 20 133 UK patients in hospital with covid-19 using the ISARIC WHO Clinical Characterisation Protocol: prospective observational cohort study https://www.bmj.com/content/369/bmj.m1985
Results The median age of patients admitted to hospital with covid-19, or with a diagnosis of covid-19 made in hospital, was 73 years (interquartile range 58-82, range 0-104). More men were admitted than women (men 60%, n=12 068; women 40%, n=8065). The median duration of symptoms before admission was 4 days (interquartile range 1-8). The commonest comorbidities were chronic cardiac disease (31%, 5469/17 702), uncomplicated diabetes (21%, 3650/17 599), non-asthmatic chronic pulmonary disease (18%, 3128/17 634), and chronic kidney disease (16%, 2830/17 506); 23% (4161/18 525) had no reported major comorbidity.
Overall, 41% (8199/20 133) of patients were discharged alive, 26% (5165/20 133) died, and 34% (6769/20 133) continued to receive care at the reporting date.
17% (3001/18 183) required admission to high dependency or intensive care units; of these, 28% (826/3001) were discharged alive, 32% (958/3001) died, and 41% (1217/3001) continued to receive care at the reporting date.
Of those receiving mechanical ventilation, 17% (276/1658) were discharged alive, 37% (618/1658) died, and 46% (764/1658) remained in hospital.
I had a question regarding the clinical interpretation of the common odds ratio. This is a segment of Table 3, with the results of the ordinal scale at day 28:
I realize that the CI for the common odds ratio is very wide, but let’s just assume the point estimate is valid for argument purposes. I think I would interpret it as follows:
- If a patient started treatment requiring supplemental oxygen (3 on the scale), he has a 1.15x higher odds of being at a 2 on the scale 28 days later (being hospitalized, but not requiring supplemental oxygen).
How would the common OR change if I wanted to estimate the odds of a patient improving by > 1 point on the scale? Taking the example above, what are the odds the patient goes from a 3 to a 1 (discharged from the hospital)?
Also, @f2harrell mentioned how there is a major problem with the primary endpoint of clinical improvement.
This seems to be the main problem with ACTT-1 study as well right? The difference being that the common odds ratio in that study showed a significant benefit in favor of remdesivir.
As an aside note that only using the 28d result results in a power loss.
The OR doesn’t have to do with within-patient changes. If the OR is not covariate adjusted for initial ordinal state, then the interpretation is the ratio of odds for treatment B : A of being in outcome category y or worse, for any y other than the first. If it is covariate adjusted, then the OR responses the same ratio of odds but for a subject on treatment B with initial state x being compared to a subject on treatment A who is in the same initial state x.