Thanks. Sorry if my question seems muddled- probably a reflection of my muddled thinking. I’m mainly just stuck on the following statement, quoted in the original post:
“there are always unmeasured strong outcome-predictive covariates and these will vary across studies and settings.“
Since this statement seems to be used as a premise for other arguments about the pros/cons of various effect measures, it seems important to establish whether there’s universal agreement with the premise.
If the term “outcome predictive covariates” is referring to effect modifiers, then the statement is suggesting that, when we test and approve a therapy, we don’t thoroughly understand the link between disease development and a therapy’s mechanism of action. My discomfort with the statement stems from the fact that I think there are many clinical situations where we do understand therapy/disease mechanisms of action well enough to reasonably assume absence of effect modification across settings. Primary PCI for STEMI is one example. In contrast, we might have grounds to question the transportability of the RCT effect of, say, a new antidepressant in patients with “treatment-refractory depression.” Whereas our understanding of the causal mechanism for STEMI is advanced/complete, we’re not nearly as close to understanding the causal mechanisms underlying development of depression. And a not-insignificant portion of patients in a primary care setting with treatment-resistant depression might actually have undiagnosed bipolar illness, which might respond unfavourably to the new antidepressant.
I guess what I’m trying to say is that, clinically (except, maybe, in fields like oncology), we very often assume transportability of an RCT effect, if we know that an approved therapy targets a well-understood causal mechanism for a disease. For diseases whose development we understand well, this assumption is usually the best bet we can make for the average patient.