In an interesting case recounted with humility and good humor in this open-access Editorial , the specification of clustered variances and fixed effects at the same level resulted in wrong inferences in a highly touted article. (On the technical point,  cites this presentation by Austin Nichols & Mark Schaffer; see slide 8.)
I would guess many of you will find valuable lessons to draw for yourselves and your students from this case. I’ll just offer a few of my own quick observations, plus one question.
- This is a major reason why I am a ‘methodological Bayesian’. Using Bayesian methods has the effect of rendering all of the most substantive aspects of the modeling transparently, while submerging all of the least important stuff into the ‘magic’ of MCMC.
- Even bigger lesson is, when you have a result you plan to tout highly, you should try several different modeling approaches. Ironically, the original piece employed ‘cluster-robust’ variance estimators, yet did not seek genuine robustness to variation in model specifications.
- Fallibilism is true.
Question: It seems to me inconceivable that a mishap like this could befall a Bayesian analyst doing hierarchical modeling; am I wrong? Does the statistical modeling/interpretation error leading to this retraction have a Bayesian analogue?
- Shafer SL. Broken Hearts. Anesthesia & Analgesia. 2016;122(5):1231-1233. doi:10.1213/ANE.0000000000001253