Effectiveness of Paxlovid a retrospective study

data analysis modeling strategy
1

I have some questions concerning a retrospective study on the effectiveness of Paxlovid in high risk patients:

Briefly:
Included patients: 180,351
Patients who received Paxlovid ( a five days treatment course): 4747
Outcome: composite of severe COVID-19 or COVID-19 specific mortality
Number of events: 942
Number of variables in the multivariable cox PH model:14 + Paxlovid that was modelled was a time-dependent variable.
Patients were followed for 28 days
Paxlovid group: 39 events in the 4737 patients
Non-paxlovid group: 903 events in the 175614 patients
HR for treatment with Paxlovid from the multivariable cox PH model 0.54 ( 95% CI , 0.39-0.75)

  1. Can sparse data bias be an issue in this study? ( It can still be an issue even when sample size is large but with adjustment for many variables)

  2. Is it justified to model Paxlovid treatment as time-dependent variable? the authors state that " Paxlovid was modeled as time-dependent variable, allowing subjects to transfer from one exposure group to another during follow-up" , but patients who received first dose of Paxlovid more than 5 days since positive sars-cov-2 test were excluded, so basically a patient is “allowed to transfer” between exposure groups only in the first five days since positive test . ( I suppose that a product term time*Paxlovid was added to the model but I am not sure of this and the authors did not specify how exactly this was done )

2

General comment: time-dependent treatment can be analyzed as a time-dependent covariate if patient status is re-assessed at times of treatment change and these updated statuses are also included as time-dependent covariates.

1 Like
3

  1. In my opinion, for the main effect of Paxlovid, sparse data bias is probably not an issue for this sample size and event rate. However, since the authors do check quite a lot of interactions that might be a problem (both because interactions requires larger sample for the same statistical power as main effects, and the multiplicity of statistical tests).
    [In addition, I’m not a fan of them deciding what terms to interact with based on data-driven variable elimination, especially when they not mention what criterion was used to eliminate variables (p-value? :scream_cat:)].
    So, overall, I trust their main effect, but I trust their secondary analyses less. I should also note their data source is of very high quality.

  2. Treating Paxlovid as time-dependent variable actually makes some sense, in my opinion.
    They care about initiation within 5 days - they want early treatment (as indicated for use) but there are logistic constraints so it usually takes a few days to get the drug to people. However, coarsening the time steps for 5 days for the entire analysis is probably not ideal for measuring the relatively short-term effects usually associated with COVID. I also tend to believe why that’s the reason you shouldn’t necessarily adjust for time-varying confounding - the decision is made at a point, but the delay is until people fill their prescription.
    Treating Paxlovid as time-varying variable should prevent immortal time bias - where people that would’ve been treated late can still contribute their event to the control group. Otherwise, you would have a period of up to 5 days in which treated person cannot, by definition, have an event because we know they initiate treatment later in the future.
    It is indeed unclear how they adjust for it (they are lacking in statistical details, same as with the variable selection above), and I agree a time-treatment interaction might have been appropriate.

1 Like
4

Using p-values to select interactions is simply awful.

5

in the paper,12 interactions were tested, one of them was treatment * vaccination status, and since it was with a p-value > 0.05 , the authors concluded that Paxlovid was effective irrespectively of vaccination status. ( which goes against the evidence from the clinical trial EPIC HR were the risk difference for seropositive at baseline was greatly smaller compared with serongeative at baseline , and against the fact that in EPIC SR ( vaccinated included) the effect of Paxlovid was smaller compared to EPIC HR ( vaccinated patients were not included)

6

Classic “absence of evidence is not evidence of absence” error.

1 Like