I generally avoid reading papers involving propensity scores, but recently made an exception for [1] on account of its senior author. While the paper employs some cautious-sounding language, if one takes it at all seriously it advances the claim that synthetic ‘external controls’ (EC) constructed via propensity score matching from curated EHR data (here, Flatiron) might in general enable single-arm registration trials.
If you advance such a claim generically, have you not effectively claimed that your EC construction procedure yields a calibrated clinical prediction model, with the propensity score serving as a sufficient statistic (in the Bayesian sense) for all the measured covariates? (Thought experiment: the generic claim would apply to a hypothetical trial that only enrolled patients with the same PS as the one in front of you, whom you would like to counsel using a calibrated clinical prediction model.)
If I’m right about that, then the enterprise of EC construction is a subterfuge that seeks to acquire all the results that would follow from a calibrated clinical prediction model, yet without the cost and expertise needed to build such a model, and without having to endure the rigorous criticisms to which such models are subject.
Related to the point I’m exploring, is that 1 of 9 ECs constructed in [1] demonstrated an HR quite different from that in the RCT against which it was validated (see Figure 3). The reason for this is investigated retrospectively by the authors, with a partial explanation offered based on the RCT having enriched for expression of an oncogene (MET) which is not routinely collected in practice and was therefore absent from the EHR database. Had the authors attempted to build a clinical prediction model, they would have drawn upon expertise that would have highlighted this issue prospectively.
- Carrigan G, Whipple S, Capra WB, et al. Using Electronic Health Records to Derive Control Arms for Early Phase Single‐Arm Lung Cancer Trials: Proof‐of‐Concept in Randomized Controlled Trials. Clin Pharmacol Ther. 2020;107(2):369-377. doi:10.1002/cpt.1586