Informative priors to reduce prediction uncertainty

[Sreenath_Madathil]

Hello everyone,

We have had a hand full of excellent discussion on quantification of uncertainty in prediction models/ risk calculators: here, here.
The latter discussion touched briefly on Bayesian approach to prediction modeling.

My question are:

1. Why the uncertainty prediction/risk score is rarely reported ?
   I have seen two main comments on this i) it is hard to interpret the confidence intervals - here,
   ii) may not really matter if the decision to be made do not change much.

I am still not convinced.
So posing the question here:
Should we calculate and report a measure of uncertainty around predicted risk score?

2. Posterior predictive distribution from a Bayesian prediction model can accommodate both epistemic and aleatoric uncertainty. If then, can’t we use informative prior to reduce the uncertainty in the posterior predictive distribution?

[f2harrell]

@Sreenath_Madathil

Posterior predictive distribution from a Bayesian prediction model can accommodate both epistemic and aleatoric uncertainty. If then, can’t we use informative prior to reduce the uncertainty in the posterior predictive distribution?

We don’t just use an informative prior to reduce uncertainty in the posterior distribution (not sure why you are bringing the posterior predictive distribution into play here). We use it to reflect what we know in advance of acquiring the data, to improve parameter estimates. So prior knowledge needs to drive the process, not the other way round.

@Sreenath_Madathil

Why the uncertainty prediction/risk score is rarely reported ?

I think that uncertainties in risk estimates should be reported much more often, and Bayesians say that we should not even report the risk point estimate but rather the posterior distribution. But one reason we don’t see uncertainties reported is that people have enough problems dealing with probabilities without telling them that we are uncertain about the uncertainty.