This editorial was recently published in JAMA: When Should Clinicians Act on Non–Statistically Significant Results From Clinical Trials?
The editorial is clearly well intentioned and written by a team of well respected clinical trialists in intensive care medicine. However, it makes a number of basic errors (in my opinion), including: absence of evidence is not evidence of absence and a misrepresentation of point estimates. They also raise some important points, for example, taking into consideration financial considerations of treatment options, and viewing trial results in the context of what came before.
Overall, I suspect the editorial will encourage a number of studies to come out in the next year proclaiming: “X was numerically higher than Y” or “the point effect for X was higher than Y” and so on. While p values certainly have their problems, this appears to be a piece that encourages us to interpret trials as we see fit, will little regard for objectivity. Almost as if we should throw the baby out with the bathwater.
In my view, understanding uncertainly is going to be essential to the next 10 years of evidence based medicine. It’s going to require a mammoth educational and cultural shift in medicine. Given the popularity and traction of this article, it feels like it will set us back, and will be used as a reference for ongoing misrepresentation.
I would be really interested to hear if my concerns here are valid and shared by the community? Or perhaps I’m totally way off base. Either way, I thought I could nail my colours to the mast and invite course correction as necessary!