Bayes for Medical Quality Improvement Analysis

data analysis
1

I posted a query on twitter (https://twitter.com/ehlJAMA/status/1831516506699788366) regarding interpretation of clinical quality improvement data. The tweet was a reaction to a statement made in grand rounds talk. The presenter showed a figure like the one attached here and said that hospitals with observed/expected (OE) ratios significantly different from what is expected are outliers in need of improvement.

It occurred to me that graphs like these (which are commonly used today in medical QA) are just a sampling distribution and that being at the tails of the distribution could reflect sampling phenomena or a true quality problem. Referring to a facility with an OE ratio above 2 as an outlier and in need of improvement seems wrong.

In response to my query, one commentator (@AdanBecerraPhD) referred to a 1997 publication demonstrating a Bayesian analytic approach to this type of data (https://t.co/in7g8Um8Cu). It used Bayes techniques to show posterior distributions of outcomes and use these distributions to determine if a hospital has problematic quality or not.

Like so many other aspects of medical research, QA investigators are using out of date statistical methods and using them incorrectly. There is a need to improve the quality improvement movement.

Is anyone aware of recent publications and/or software that can be used to compare facility/practitioner outcomes using Bayes?

OR Ratios

2 Likes
2

Ed this has been extensively studied, and one of the leaders in the field is Sharon Lise-Normand. Bayesian hierarchical models must be used in this situation to get stable per-provider estimates. I have a lot of papers marked under the quality keyword in https://hbiostat.org/papers - you can direct mail me to get the password for protected pdfs.

3 Likes
3

Hello Edward,

I recommend reading the rstanarm vignette on partial pooling as well as the Stan tutorial on partial pooling that it references. They look at binary outcomes but changing to count data is straightforward in rstanarm.
The important part is that the hierarchical model results in a smaller slope compared to no pooling. That obviously affects which hospitals are identified as needing improvement.