Here is another example of apparent adverse guideline effect induced by misinterpretation or incorrect application of statistics.
The initial practice guidelines for COVID-19 were based on the estabished guidelines for treatment of Adult Respiratory Distress Syndrome (ARDS) (as due to influenza).
This statistically misinterpreted study shows why there was confidence in this approach.
Note the raw mortality is not significantly different but encouragingly the “SOFA score-adjusted mortality of H1N1 patients was significantly higher than that of COVID-19 patients, with a rate ratio of 2.009 (95% CI, 1.563-2.583; P < .001).”
In march 2020, not to worry!
We now know this was misleading and that modification of the guidelines for COVID-19 were required but this was delayed. Many were probably compelled by the SOFA adjusted P value here.
The problem is that SOFA is a “fake” (guessed) one-size-fits-all measurement tool for RCT for reasons I have noted in this forum.
Looking deeper, why would adjusting for SOFA be so misleading?
SOFA is a summation score derived from ordinal thresholds of 6 signals.
https://www.google.com/url?sa=t&source=web&rct=j&url=https://www.mdcalc.com/sequential-organ-failure-assessment-sofa-score&ved=2ahUKEwjskMueh7nxAhWDFlkFHaooDjkQFjACegQIGhAC&usg=AOvVaw2H1jBD1fkPUCqZHwK6xXqJ&cshid=1624839636768
Of these 6 signals the one most likely to be perturbed in COVID19 is the PaO2/FIO2. But the PaO2/FIO2 is a volatile signal and affected by reversible things like low PEEP or mucous pluging. In many cases PaO2/FIO2 is readily correctable. Therefore the timing of the PaO2/FIO2 (early or late in the care) greatly influences the mortality prediction. Here we see influenza (H1N1) cases presented with lower PaO2/FIO2.
The “take home point” of the editor published in March 2020 in the prestigious journal Chest was misleading.
"Interpretation:
Compared with H1N1, patients with COVID-19-induced ARDS had lower severity of illness scores at presentation and lower SOFA score adjusted mortality."
The article also suggests (without data presented) that corticosteroids were not beneficial and suggested they may be harmful. (Later corticosteroids became the standard of care.)
This is how adjustment with a guessed (1996) traditional score (which has reached standard use by PI & statiscians) can be misleading.
Yet SOFA was originally guessed for use with sepsis and we have discussed its limitations elsewhere in this forum. Clearly adjustments like this can produce misleading results and adversly effect public policy and medical guidelines.
This is a form of “Threshold Science”. Threshold science is not the use of thresholds in science as this is a compromise often made. Rather “Threshold Science” is a specific subset of science. It is a strange science which emerged in the late 1970s and 80s and uses of guessed threshold sets like SOFA as gold standards (criteria), independent variables, adjustments, or outputs in RCT.
Here is an example of the harm threshold science can render.
If anyone would like to learn more about this unique pathologic science please message me.
I have been researching threshold science for over a decade and have a robust archive to share. This would be a powerful overarching and nascent topic for publication.