Sander this is very clear, well set up, and convincing. All I can add is a bit more context.
In my view we need to be pre-specifying primary study analyses that are Bayesian, and to settle the choice of prior (whether skeptical or previous data- or knowledge-based) before the choice could possibly be influenced by the results, or be manipulated by unscrupulous investigators. That being said, there is great value in Bayesian interpretation of already-completed results. Since for this setting it is not possible to find an already pre-specified prior, a skeptical prior is often fitting. I can think of two modes for choosing such priors:
- Finding the ultimate judges of the research and elicit skeptical priors from them, or
- Do as Sander wrote and select a reasonable skeptical prior that is likely to either convince most skeptics or to just make large effects unlikely. The latter assumption is very plausible in most areas of research. Even if the observed effect was equivalent to a “cure” (effect ratio of 0.0) the Bayesian posterior median effect would remain very impressive under such a skeptical prior.
The second option is more feasible. As with Sander, I take ‘skeptical’ to mean equal probability of harm as benefit. More specifically I take the prior probability that the effect ratio < r to be equal to the prior probability that it is > 1/r. I often simplify this to solving for the variance of a normal prior distribution for the effect log ratio, but Sander advocates a more flexible F-distribution-based approach, which I also like.