Reporting frequentist and Bayesian results side by side in physiotherapy evidence: helpful or confusing?

[Ertan]

Hi all,

I would value the community’s views on a reporting question that came up in a recent project.

I reanalysed the meta-analyses in a recent systematic review on exercise for rotator cuff-related shoulder pain (Lafrance et al., JOSPT 2024). The review concluded that motor control exercise is “probably slightly superior” to nonspecific exercise and recommended it to clinicians, interpreting effects with Cohen’s effect-size categories rather than a clinical threshold.

Using group-level data extracted from the review’s forest plots (19 RCTs), I fitted Bayesian random-effects meta-analyses (bayesmeta in R) with neutral, sceptical and enthusiastic priors calibrated to a clinical threshold, and half-normal priors on τ. For pain (0–10 scale), I used an MCID of 1.1 points, deliberately choosing the lowest published value.

Two results illustrate my question:

1. Motor control vs nonspecific exercise, pain at ~3 months (5 trials): The original review reported SMD 0.38, p = 0.03. In the Bayesian analysis (neutral prior), the posterior mean difference was 0.69 points (95% CrI −0.10 to 1.50); P(benefit) = 96%, but P(effect > MCID) = 12%. The two analyses agree on direction but tell a different clinical story.
2. Same comparison at ~12 months (only 2 trials, 96 patients): A classical random-effects analysis gives p = 0.002, the “strongest” result in the dataset. The Bayesian analysis gives a 95% CrI that includes zero, and P(effect > MCID) falls from 73% (neutral prior) to 23% (sceptical prior), largely reflecting the uncertainty in τ with two studies.

My questions: In physiotherapy, most readers and reviewers still think in terms of p-values. I am considering recommending that both frequentist and Bayesian results be presented together in future trials and reanalyses (e.g., frequentist primary analysis plus a pre-specified Bayesian analysis reporting probabilities of exceeding clinically meaningful thresholds).

• Is side-by-side reporting genuinely useful, or does it risk confusing readers or inviting “pick the more favourable result”?
• If both are reported, which should be primary, and how should discordant results (as in example 2) be communicated?
• Would you instead recommend dropping p-values and reporting only compatibility intervals alongside the Bayesian results?
• Any concerns about using within-patient MCIDs as thresholds for between-group effects in this setting?

I would be grateful for any thoughts, references, or examples of good practice.

Thanks in advance.

[R_cubed]

Reporting likelihood functions along with the compatible confidence distributions would satisfy a very broad range of statistical philosophies, and make meta-analysis somewhat easier.

How to discuss the results in text is up for debate. In a number of threads @f2harrell has persuasively argued mixing up frequentist concepts with Bayesian ones especially after the data have been collected is a confusing mess, and I’ve (finally) come to understand why he holds that POV.

Is Repeated Bayesian Interim Analysis Consequence-Free?

The paper mixes two ideas, which creates a great deal of confusion: choice of priors, and sequential assessment. These ideas need to be separated. Differences of opinion about priors will have a similar effect whether talking about interim analyses or final analyses. The paper would be more more clear had it concentrated on sequential analysis only. When doing that it becomes clear that in the Bayesian paradigm, analyzing data at n=200 in a study that planned to enroll 200 patients gives exa…

So, I think it is defensible for a Bayesian to put frequentist summaries like confidence distributions in an appendix, and use the main body of the paper to discuss the specification of the prior, and the design that produces the data summarized by the likelihood function.

A Bayesian can make direct use of a likelihood function, but might need to work out how to extract it if only a confidence distribution is reported. So to make things convenient for everyone, just report both.

Study of the monograph Confidence, Likelihood, and Probability will provide a lot of ideas on how to use sets of confidence intervals as frequentist summaries, and how they relate to Bayesian inference.

FWIW, I’m less enthusiastic about their claims regarding confidence distributions as a replacement for Bayesian inference, and I still find the frequentist notion of “epistemic confidence” opaque. Regardless, the relationship between confidence distributions and likelihood functions is well worth exploring.

Confidence, Likelihood, Probability

Further Reading

1. Pawitan, Y., & Lee, Y. (2021). Confidence as likelihood. Statistical Science, 36(4), 509-517.

2. Lee, Y. (2025). Epistemic Confidence Statement via Extended Likelihood. arXiv preprint arXiv:2512.24701. link

3. Efron, B. (1993). Bayes and likelihood calculations from confidence intervals. Biometrika, 80(1), 3-26.

4. Infanger, D., & Schmidt‐Trucksäss, A. (2019). P value functions: An underused method to present research results and to promote quantitative reasoning. Statistics in medicine, 38(21), 4189-4197.

5. Rosenthal, R., & Rubin, D. B. (1994). The counternull value of an effect size: A new statistic. Psychological Science, 5(6), 329-334.

The second paper by Lee (2025) partially addresses Frank’s concerns about extended likelihood in this thread:

Change the range not the language on confidence intervals

What do you think of broadening the discussion from a single interval to the set of “confidence” intervals, leading to confidence distributions? I agree that the Bayesian perspective is more natural and relevant, but frequentist reports are common, and I think it is important to understand how to interpret them on their own terms. Confidence distributions, accoring to Pawitan, have an interpretation as an “extended likelihood” ie. a likelihood that partially accounts for its uncertainty. A mo…

Language for communicating frequentist results about treatment effects

Correct interpretation of p > 0.05: The current sample size did not yield sufficient evidence to reject, at the incredibly arbitrary 0.05 level, the supposition that the treatment does nothing, assuming data model M (describe M …) which forgot to include covariates. Editorial Notes As Sander Greenland has so well stated below, this initial draft was still too “dichotomous” in that it was written assuming we would have different language for “positive” vs. “negative” studies. This implies a…