How would you relate to late death vs early heart attack in the context of ordinal longitudinal modeling?
I don’t think that prediction models are making any kind of utility interpretation directly, but they do so by using performance metrics and inclusion-exclusion criteria of the target population.
For example:
- c-index for time-to-event implies that the earliest events should be prioritized.
- lift implies that patients at the highest risk will gain the highest benefit.
- Brier Score implies that the absolute differences between the predictions and the outcomes are equally important for nonevents (p = 0.3, y =0) and events (p = 0.7, y = 1).
And so on… Sometimes these assumptions are reasonable and sometimes they are not, but I think that it will be much easier to interpret and communicate utility under a counterfactual setting. For lift we can use the uplift setting which is very natural to marketing profiling but I do believe that is relevant for healthcare as well.
My main takeaway is that we should strive for alignment between narrative thinking that comes from domain-experts / decision-makers of all sorts and the underlying assumptions behind performance metrics. I used to think that we should train clinicians to play poker in order to improve their probabilistic thinking, but the following thread changed my point of view: