Thanks for the question.
Failure Mode Analysis (FMA) is a systematic method for asking:
- *How can, or did, this system fail? (For example, an RCT or RCT derived guideline.)
- How can that failure be detected or prevented?
Highly advanced in aviation, FMA does not merely establish that its mathematics are correct. You map a process (eg structural casual modeling) before the trial and examine ways the design could fail.
A lengthy discussion of the “limitations of the trial” and even sensitivity analysis are not a substitute for FMA.
Failing to distinguish true FMA creates risk. Extensive analysis of downstream technical details may produce false confidence that a system has been adequately tested while leaving the underlying design failure hidden for decades.This generates a repeating pattern of “mathematical arrogance despite unmitigated implementation failure”. Repeated failures are attributed to what I call “amorphous academic excuses from the ether” such as chance, heterogeneity, implementation, or unavoidable uncertainty rather than to a reproducible structural defect.
The “RCT failure mode trilogy” is the first set of papers to combine historical and structural analysis to search for a primary failure mode underlying decades of reversals for harm of critical care trials and RCT-derived guidelines. I hope you will read them and offer critique, if interested in the causal stucture if RCTs.
Getting back to your earlier question, this helps explain why some critical care physicians hesitate to grant statisticians trust relavant greater mathematical discretion BEFORE these statisticians first formally investigate why earlier RCT derived guidelines repeatedly failed for decades. In other words, we know they don’t know why those earlier RCTs failed, and we can’t trust them to fix with math that which they have not even deeply examined.
Critical care physicians are the guardians of the critically ill and we know that flexible mathematics cannot correct an unidentified structural failure mode. We also know that, if applied to the same pathological trial structure, more complex mathematics can embellish the trial providing an “elegant pseudo-fix” thereby provided an alternative path bypassing the required FMA. We know this could bring to the field another decadal wave of failures and patient harm which can be each be eventually swept under the rug without deep structural FMA like the December 2023 Bayesian RE-MAP CAP cause agnostic RCT already was.