Research into the performance of tests for diagnosis and treatment selection

Thank you for your comments @med_stat and @f2harrell. I accept the point that I did not explain myself. However, I did describe my reasoning in another post: The Higgs Boson and the relationship between P values and the probability of replication.

The Bayesian posterior probability that a true result will lie within a credible interval relies on a prior probability distribution of possible true results that is estimated informally. The data arising from a real study is used to estimate a likelihood distribution of the observed study mean or proportion conditional on a range of possible true results. However, the patients studied to estimate the latter likelihood distribution from data are not a subset of the set those patients imagined in order to arrive at the former prior distribution, so this violates a condition of Bayes rule. The latter only holds true when U is a ‘universal set’ so that X⊆U and Y⊆U so that p(Y|X) = p(Y|U)*p(X|Y)/p(X|U). In other words, the informal Bayesian prior has to be equal to the result of a previous posterior probability obtained from a prior probability distribution conditional on some universal set and an informally estimated likelihood distribution.

If the prior probability distribution conditional on some universal set is uniform, the informally estimated Bayesian prior distribution can also be regarded as an estimated likelihood distribution of an imagined mean or proportion conditional on a range of possible true results. However, this raises the question of what is the universal set. The universal set could be the scale of possible values to be used on the horizontal axis. As each of these values is unique then their prior probabilities would be uniform. This information about the scale would be known prior to the knowledge used to estimate the informal Bayesian distribution and the result of the real study. This is the only logical model that I can think of as being consistent with a universal set in this situation. If the imagined and real study generated distributions are regarded as the result of random selections from a single population with a true mean or proportion, then their joint distributions can be modelled by an assumption of conditional independence.

In the situation that I describe in my original post, there is no informally estimated prior distribution, so I did not think I could call it a ‘Bayesian’ posterior probability of a result falling within a credibility interval. The 95% confidence interval summarises the likelihood distribution of the observed mean or proportion conditional on a range of possible true means or proportions. By adopting a universal set of unique values with uniform priors, the posterior distribution is identical to the likelihood distribution, so that the 95% posterior probability of replication interval is identical to the 95% confidence interval. We use the same reasoning that includes uniform prior probabilities in clinical settings to calculate 95% posterior probability prediction intervals from standard deviations (as opposed to SEMs). I follow this approach to maintain consistency between my clinical reasoning and statistical reasoning. It does not affect the results of calculating Bayesian posterior probability distributions by incorporating a distribution that has been estimated informally.

1 Like