I recently was debating a health economist on a risk adjustment model used for a heart failure (HF) readmission model. The metric is meant to measure the number of HF hospitalization over the total number of HF patients cared for a given 12 month period. The model is risk adjusted to account for the severity of the outpatient population. The risk adjustment model uses administrative coding data to primarily estimate the expected number of HF hospitalizations by facility.
The approach to risk adjustment used is not causal. The model has evaluated all available variables that coded for most patients. This included age, sex, minimal insurance/SES variables, and marital status. It then includes every CCS code. CCS codes (about 260) are developed by the AHRQ to collapse all ICD10 codes into workable categories. Some of the CCS codes I noticed had large weights in the final logistic model included factors like ectopic pregnancy. If a patient had an ectopic pregnancy they were really unlikely to ever be admitted with HF (kind of obvious). Therefore, if a facility treats many patients with ectopic pregnancies, there HF population is seen as “healthier” and they should not have as many expected admission. The c-statistic for this administrative model is ~0.8. Most HF readmission models of hospitalized HF patients have c-statistics around 0.6 - no good predictors for who is likely to be rehospitalized at discharge.
I am troubled by this approach to risk adjustment. It is akin to dredging meaningless associations that are false and do not account for the concept of risk. The purpose of risk adjustment is to adjust for known factors that make disease management more difficult. When you dump a bunch of garbage codes you induce false associations that do not actually adjust for risk. If there are only 150 faciltiies, and they very in how they code diagnoses or if they have a busy “women’s clinic” with more ectopic pregnancies, then the model may just be selecting for differences in facilities that have nothing to do with the severity of HF seen in the clinic.
Anyway, I feel risk adjustment requires more theory to be done appropriately and there also needs to be sufficient predictive ability or else the risk adjustment is arbitrary. Appreciate other’s thoughts both on the theories behind risk adjustment and how to evaluate adequacy of risk adjustment models. I think like anything we would look at model fit, log-likelihoods, c-statistics etc.