Prior design in a bayesian re-analysis of an RCT

data analysis bayes
1

I am conducting a Bayesian re-analysis of a non-inferiority RCT. Since it is not possible to use a meta-analytical prior, our work focuses on constructing a prior based on the synthesis of experts opinion.

Following the literature, particularly the work of Spiegelhalter, Parmar, and others, the idea was to conduct a questionnaire asking two things:

1.Their opinion on the non-inferiority interval, of which we would take the arithmetic mean, as indicated by Spiegelhalter in his work.
2. To construct the prior with the arithmetic means of the intervals weighted by each clinician’s confidence that it is the true value (on a scale of 0 to 100, each clinician is asked to assign a score to each interval such that the scores sum to 100, and the arithmetic means are calculated in each interval between clinicians).

However, there is a problem: The outcome we are asking clinicians about is a cumulative incidence of recurrences. We are asking for the percentage difference in cumulative recurrence at 5 years between the two groups. Therefore, it seems very important to estimate the cumulative incidence of the reference group because the same percentage difference between the two groups has a different impact depending on the frequency of the event in the reference group.

I expect the variance of the distribution of the cumulative incidence in the control group to depend on its mean. How can I solve this problem? Is it better to fix the cumulative incidence of recurrences as specified in the control group, or is it better to use hyperparameterization like in hierarchical models? Or maybe to simulate scenarios based on the possible distribution of the control group cumulative incidence distribution? Are there any other options?

EDIT: I’m reading some material about and I think this could be considered a “nuisance” parameter, right ?

1 Like
2

Afraid I can’t give any specific recommendations. This paper is a good recent review of prior elicitation techniques: https://arxiv.org/abs/2112.01380.

1 Like
3

The worst possible way to design a prior is to ask experts. That will just reinforce existing opinion, and that is one of the main reasons
why people distrust Bayesian methods. It is very rare to have enough solid, hard,data to construct an objective prior. The only sensible approach is to adopt a skeptical prior, eg in cases where it’s sensible to test a point null, put a lump of prior probability on the null. P(H0)=0.5 would probably be overoptimistic for drug trials, but it would be a lot more cautious than a p value, eg if you’ve observed p=0.049, and claim that you’ve discovered an effect, the probability that you’re wrong is not 4.9%, but it’s roughly 25%.

2 Likes
4

David Spiegelhalter has said essentially the same thing, in the context of oncology. He suggests trusting the experts to provide the variance of the prior, since experts disagree with each other so much. But don’t trust them to provide the central tendency of the prior.

4 Likes
5

For re-analysis of oncology RCTs we have developed this empirical prior and webpage to use it. A general version for re-analysis of any medical RCT that would be eligible for inclusion in the Cochrane Database of Systematic Reviews is available here with explainer here.

1 Like