I pulled off Savage’s Foundations of Statistics from my shelf. I can’t find a direct quote, but both the chapter on elicitation of personal probabilities and personal utilities involved gambling games, so distinguishing one from another is a challenge.
He mentions in passing (p. 95):
Blockquote
It seems mystical to talk about moral worth [ie. utility] apart from probability…
A direct reference to the problem of separating personal probabilities from utilities can be found in:
Joseph B. Kadane & Robert L. Winkler (1988) Separating Probability Elicitation from Utilities, Journal of the American Statistical Association, 83:402, 357-363, DOI: 10.1080/01621459.1988.10478605
A follow up to the prior paper reports extending the results more generally. The last section of the paper states:
Blockquote
In so far as the economic implications are concerned, the failure of probability elicitation procedures implies a corresponding impossibility of reconstructing utility functions from asset demands by the method of revealed preference
I must have confused Savage with (the late) Herman Rubin (Purdue University), who wrote a technical report on the (non)-separability of the prior from the utility function. I’ve seen references to a similar version being formally published in 1987.
Rubin, Herman (1983) A weak system of axioms for ‘rational’ behavior and the non-separability of utility from prior. (pdf)
This discussion with Herman Rubin describes his attitude towards the relationship between loss and prior in the context of prior Bayesian Robustness:
Bock, Mary Ellen. Conversations with Herman Rubin. A Festschrift for Herman Rubin, 408–417, Institute of Mathematical Statistics, Beachwood, Ohio, USA, 2004. doi:10.1214/lnms/1196285408. (link)
Blockquote
Q. One of your strong, ongoing interests is prior Bayesian robustness. How do you describe it?
Rubin: One of the difficulties of Bayesian analysis is coming up with a good prior and loss function. (I have been saying for years that the prior and loss cannot be separated. The Carnegie Mellon school is doing some work on this now). When I talk about prior Bayesian robustness, I assume that one does not yet see the random observation X whose distribution depends upon the unknown state of nature. One considers a choice of priors for which one averages over the possible states of nature and over the possible observations. This is different from posterior Bayesian robustness in which one considers the the choice of priors given the random observation X … When I am faced with a choice of priors, all of which seem about the same to me then I am very concerned about the possible alternative consequences of applying either one if it is drastically wrong.
In the context of the question by the OP, I interpret this question of skeptical priors as inextricably related to the question: “How optimistic can I afford to be in assuming that the model is (approximately) correct?”
If I was pressed to defend a point of view, I’d say that there exist circumstances when subjective utility and probability can be usefully separated, but there also exist important cases where probability and utility are inseparable.
As these cases did not seem to bother Herman Rubin, I don’t think they should bother anyone concerned with applications. But it is good to be aware of them.
Addendum: additional citations to some work by Herman Rubin, as well as a paper that extends the results of Kadane and Winkler.