I’m writing here a plea for literature recommendations or theoretical guidance. My research group work in RCTs with continuous outcomes (e.g. symptoms of depression) where the typical presentation of results is a mean difference (average treatment effect) with a 95% CI. But something I haven’t seen reported (at least not in my field) is a measure of treatment effect heterogeneity.
When you identify substantial heterogeneity in a random-effects meta-analysis, it’s typically more valuable to explore the sources of that heterogeneity rather than calculating a pooled mean difference. This is because the latter ‘averages over’ important information and is of limited clinical utility. I cannot see how the same logic would not apply in an RCT with a highly variable treatment effect.
I can imagine the value of presenting heterogeneity in an RCT would be that, like meta-analysis, it would shift the goals of your analysis. If your trial found low heterogeneity this would support the validity of reporting an ‘average treatment effect’. The average treatment effect is roughly what clinicians can expect to see in future patients. While if there was high heterogeneity, the study could shift focus and give greater emphasis to exploring potential effect modification. A natural way to measure this heterogeneity might be with something like a 95% prediction interval, as in meta-analysis.
But i’m not sure if this makes sense? As I do not see it used routinely. If anyone is able to provide guidance on this issue and any potentially helpful literature I would be very grateful. If it is sensible, it’s something I would like to adopt in my own work. Thanks!