# Is Repeated Bayesian Interim Analysis Consequence-Free?

**URL:** <https://discourse.datamethods.org/t/is-repeated-bayesian-interim-analysis-consequence-free/28342>\
**Category:** journal club\
**Created:** [August 20, 2025, 10:56am UTC](https://discourse.datamethods.org/t/is-repeated-bayesian-interim-analysis-consequence-free/28342 "2025-08-20T10:56:16Z")\
**Posts on this page:** 1\
**Showing post:** 4

<div class="post-metadata">

**Author:** ![f2harrell](https://discourse.datamethods.org/user_avatar/discourse.datamethods.org/f2harrell/32/165_2.png) [@f2harrell](https://discourse.datamethods.org/u/f2harrell)\
**Post date:** [August 20, 2025, 3:19pm UTC](https://discourse.datamethods.org/t/is-repeated-bayesian-interim-analysis-consequence-free/28342/4 "2025-08-20T15:19:22Z")

</div>

The paper mixes two ideas, which creates a great deal of confusion: choice of priors, and sequential assessment. These ideas need to be separated. Differences of opinion about priors will have a similar effect whether talking about interim analyses or final analyses. The paper would be more more clear had it concentrated on sequential analysis only. When doing that it becomes clear that in the Bayesian paradigm, analyzing data at n=200 in a study that planned to enroll 200 patients gives exactly the same evidence as an early look at n=200 for a planned study of n=800. Posterior probabilities do not change interpretation because of stopping rules.

The authors wrote the manuscript from a standpoint that somehow “blesses” the planned final sample size, giving it special meaning that it never has. One researcher’s interim analysis is another person’s final analysis. All statisticians know that fixed target sample sizes are always derived using voodoo and are arbitrary is almost every way except for budgeting purposes.

Any study of the effect of multiple testing in the Bayesian world needs to respect Bayesian operating characteristics, discussed in detail [here](https://hbiostat.org/bayes/bet/design). For example, when considering the Bayesian operating characteristic for an assessment of treatment superiority, compute the posterior probability of superiority and stop the moment it exceeds a threshold such as 0.95. Then reveal the actual efficacy used to generate the data and see if the efficacy was positive. The number of times the true data generating efficacy was positive divided by the number of simulated studies in which there was sufficient evidence for efficacy is THE Bayesian operating characteristic for this situation. You will find as I’ve simulated in the link given above that this operating characteristic is perfectly preserved with infinitely many data looks. For example if the decision threshold is P(efficacy \> 0 | data, prior) \> 0.95 you may find that the average probability of efficacy at the moment of study stopping is 0.96. If the sampling prior equals the analysis prior, you’ll find that the proportion of trials with efficacy \> 0 is exactly 0.96. The only relevant and interesting study of operating characteristics is what happens to decision accuracy when there is a mismatch between sampling (simulation) prior and analysis prior. You may find that the accuracy goes from 0.96 to 0.93, for example, validating the excellent performance of the Bayesian decision rule even under prior mismatch.

The authors of the manuscript made a critical decision that Bayes is interested in hypothesis testing. It is not. Bayes deals with the probability that an assertion is true. Assertions are almost always about intervals of unknown treatment effects and having nothing to do with single points. This has everything to do with how you simulate the performance of a decision rule.

Bayes has nothing to do with \alpha. Any attempt to make it deal with \alpha will result in

- a hard-to-interpret mess
- a frequentist procedure and not a Bayesian one
- violation of the likelihood principle
- \alpha will increase due to having the opportunity to make a decision that was not actually made
- much more complex simulations needed
- stopping much later for futility, resulting in wasted time and money
- most importantly, a reduction in the probability that made decisions are correct

It is **extremely important** to note that P(made decision is correct) is an **unconditional** probability in the sense that it does not condition on any unobservable quantity such as the true treatment effect. It is the accuracy of the decision **in the field** , not an assessment of **what might happen** if the treatment were to be ignorable. \alpha is a pre-study concept that makes no use of real data in any way.

---

_[View the full topic](https://discourse.datamethods.org/t/is-repeated-bayesian-interim-analysis-consequence-free/28342)._