This long thread about odds ratios Should one derive risk difference from the odds ratio?contained a brief exchange about the idea of “random confounding.” After seeing experts debate this term online, I wondered whether the controversy might be fuelling confusion among students of epidemiology, statistics, and medicine.
Reading around this topic led me to the following article:
This publication seems important. Is the idea of confounding “in measure” widely recognized or accepted by statisticians, who seem to speak exclusively (?) about confounding “in expectation”? Does the epidemiologic notion of confounding “in measure” underlie historically common MD critical appraisal practices (e.g., scrutinizing Table 1 in RCTs for between-arm covariate imbalances)? At least some MDs were taught to do this by MD instructors who had an epidemiology background, only later to hear statisticians tell them, in the immortal words of Bob Newhart, to “Just Stop It !!”
Could some of the heated exchanges in the Odds Ratio thread stem from disagreement about the “in-distribution” versus “in-measure” conceptualization of confounding and effect modification?
Some notable excerpts from the article (with bolded and underlined text inserted by me for emphasis):
“This general notion of confounding or exchangeability can be defined both with respect to the distribution of potential outcomes and with respect to a specific measure. The distinction has been drawn before (Greenland et al., 1999)…
…A further distinction can be drawn between confounding “in expectation” and “realized” confounding (Fisher, 1935; Rothman, 1977; Greenland, 1990; Greenland et al., 1999). In a randomized trial the groups receiving the placebo and the treatment will be comparable in their potential outcomes on average over repeated experiments. However, for any given experiment, the particular randomization may result in chance imbalances due to the particular allocation. Such a scenario would be one in which there is no confounding “in expectation” but there is realized confounding for the particular experiment (conditional on the allocation). Some authors (Greenland et al., 1999; Greenland and Robins, 2009) prefer to restrict the use of “no confounding” to that that is realized; a number of authors (e.g. Rubin, 1991; Robins, 1992; Stone, 1993) use terms like “no confounding” to refer to that in expectation; here we will adopt the latter practice…
…We have seen that a distinction can be drawn between confounding in distribution and confounding in measure. A similar distinction can in fact also be drawn with regard to effect modification…
…More recently, the expression "effect- measure modification" (Rothman, 2002; Brumback and Berg, 2008) has been used in place of the expression, “effect modification.” This has arguably occurred for two reasons. First, as has often been pointed out (Miettinen, 1974; Rothman, 2002; Brumback and Berg, 2008; Rothman et al., 2008), there may be effect modification for one measure (e.g. the risk difference) but not for another (e.g. the risk ratio). Effect modification in measure is thus scale-dependent and the expression "effect- measure modification" makes this more explicit. Second, with observational data, control for confounding is often inadequate; the quantities we estimate from data may not reflect true causal effects. The expression "effect- measure modification" suggests only that our measures (which may not reflect causal effects) vary across strata of Q, rather than the effects themselves (which we may not be able to consistently estimate).
…Confounding and effect modification, as conceived in this paper, and in much of modern epidemiology are causal concepts: they relate to the distribution of counterfactual variables. In practice, however, statistical models are often used to reason about the presence or absence of confounding and effect modification.
…Third, the distinction between confounding in distribution versus measure becomes important when considering “collapsibility” approaches to confounding assessment i.e. in settings in which an investigator evaluates confounding by comparing an adjusted and unadjusted estimate. Greenland et al. (1999) showed that for the risk difference and the risk ratio scales, collapsibility follows from no-confounding and vice versa. However, this implication holds for confounding in measure, not confounding in distribution. One may have collapsibility on the risk difference scale and therefore conclude that a particular variable is not a confounder of the risk difference (conditional on the other covariates); however, this does not imply that the variable is not a confounder for the risk ratio; it might be necessary to make control for that variable in evaluating the risk ratio. Collapsibility of the risk difference implies no confounding in measure for the risk difference; collapsibility of the risk ratio implies no confounding in measure for the risk ratio; however, neither implies no confounding in distribution. One must be careful when changing scales - not only in assessing effect modification - but also when thinking about confounding.
Question: Do statisticians subscribe to the idea of confounding “in measure”? If not, why not? Do these differences of opinion (if they exist) relate to statisticians’ focus on RCTs (as compared with the observational study focus of epidemiologists)? And where do these disagreements leave students, who will struggle to reconcile the views of their epidemiology and statistics instructors?