The overarching goal of statistics is to make decisions in the face of uncertainty. Debates continue to rage about how to actually do this, with some of the choices being
- frequentist null hypothesis testing
- frequentist confidence limits
- likelihood ratios (the likelihood school is similar to Bayes without priors)
- likelihood support intervals (using relative likelihood)
- Bayesian hypothesis tests
- Bayes factors
- Bayesian posterior probabilities (from subjective, skeptical, or objective Bayes)
- Full Bayes decisions, maximizing expected utility (an integral of the posterior distribution and the utility function)
What seems to be missing is head-to-head comparisons of approaches to see which ones optimize utility/loss/cost functions where such functions reflect real, concrete goals.
This is not a comparative study, but Don Berry had a wonderful paper showing how to design a vaccine clinical trial for a Native American reservation in which the objective function was to maximize health of the entire reservation, not just those persons enrolled in the trial.
Does anyone know of comparative studies that inform us of the value of two or more statistical approaches when the goal is making the best decisions?
See this NY Times article for a nice non-statistical description of decision theory.