R-cubed: Regarding “questions of ‘what the data say’ should not, in principle, depend on the prior of the experimenter”…
Contrast that to my oft-stated position: Data say nothing at all; they are just bits in a medium or numbers in tables that sit there. If you hear them say something, seek professional help. In other words, for me the claim that data say something without a prior is akin to saying a stack of lumber is a house.
More precisely, I regard the quoted statement as based on a failure to take into account Box’s point (echoed many since including Gelman) that a data-probability model (which all approaches must use in some form) is itself a construct from prior distributions, albeit distributions for data given parameters. As such it can and often does vary drastically across analysts. Just consider the controversy over the covid surveys and IFR estimates of Bhattacharya, Ioannidis and colleagues. There were very different priors in play about how selective the analyzed data were, and many other conflicts over assumptions.
Anything the data and its derived statistics purportedly say about actual inference targets are interpretations of observers filtered through their assumptions about the data generator (crudely caricatured or hidden by an estimating or likelihood function), plus perhaps some information external to that function (crudely caricatured or hidden by a prior distribution). Sometimes, as in very simple tightly controlled experiments, everyone is confident enough in the data-generator assumptions to switch all attention to the parameter prior. Unfortunately that special case seems to have led to neglect of uncertainties about the data generator, turning most of statistical theory into a misleading model of rational inference in other settings (including all those I see in health and medical research).
-That said, I agree with
“I’d prefer reports that permit me to substitute my own prior or α”
which I would express by saying I want to use my own assumptions, external information and utilities in drawing any inference or decision from the data. Often though I also would wish I could redo the entire analysis using a different data model (even some of my own); it’s the lack of the data and time that makes that impractical.