…
I’m also troubled by the approach taken here, but the ‘primary insult’ arises not from statistical technique, but from the philosophy of science that undergirds the whole project. This entire technocratic enterprise of ‘risk adjustment’ proceeds from something akin to a Machian idealism that supposes the phenomenalistic deliverance of medical billing codes sufficient to the task of centrally administering a health care system:
First, what do biostatisticians take as their phenomenalistic deliverance, analogous to the dials on Mach’s lab instruments? Here, it seems you have the data you tabulate, and quantities readily calculable from those data. This is, after all, your ‘direct experience’. 5/
— David C. Norris, MD (@davidcnorrismd) March 18, 2019
Beneath all of this seems to be the faith that we can measure “metaphorical oomphs” …
The primary focus in interpreting therapeutic clinical research data should be on the treatment (“oomph”) effect, a metaphorical force that moves patients given an effective treatment to a different clinical state relative to their control counterparts.
Mark DB, Lee KL, Harrell FE. Understanding the Role of P Values and Hypothesis Tests in Clinical Research. JAMA Cardiol. 2016;1(9):1048. doi:10.1001/jamacardio.2016.3312.
… the deeper (mechanistic/theoretical) probing of which is thought to be neither necessary nor even desirable.
By contrast, a genuinely scientific approach to risk adjustment would begin with deep theorizing about processes of disease progression and management at the individual-patient level, and would always recur to this level as providing a principal criterion of truth.
I put it to you that a risk-adjustment model for HF care is plainly invalid if it eschews theoretical knowledge sufficiently detailed for retrospective analysis to identify care process errors in particular cases. (For example: this patient’s self-recorded weights at home were clearly increasing for 4 days and should have triggered a nurse home visit.) Without such a concrete grounding in the care of the individual person, this discussion can only float off into the abstract realm of statistical method. There will be no realistic basis for correcting errors, or refining understanding.
Since a dispute with an economist prompted the OP, I’d like to advance a variation on the old economist-deflating jibe, “If you’re so smart, how come you ain’t rich?” In this case it’s “If your risk-adjustment models are so great, how come you asked a nurse to recommend a cardiologist for your parent with heart failure?”
As a rule, I generally regard survey research as belonging in the same do-not-read class as nutritional epidemiology. But I strongly suspect a survey of nurses would perform better than any of these risk-adjustment models, at least for identifying high-quality doctors within and across institutions.
We all know that, when it comes to things we sincerely care about (like care for ourselves and family), we use our personal connections and not statistical models to assess quality-of-care questions. These risk-adjustment models are by and large about the care that elites are willing to accord to ‘the other people’.