I will fix this post about medical decision making with an example that relates to a manuscript my colleagues and I posted recently on MedRxiv ( https://medrxiv.org/cgi/content/short/19011924v1). Women at high risk of tubo-ovarian carcinoma, such as women who carry a disease associated variant in BRCA1 or BRCA2 are usually offered prophylactic surgery (bilateral salpingo-oophorectomy) as a preventive measure. The lifetime risk of tubo-ovarian carcinoma in a BRCA1/2 carrier is over 15 per cent. Recently other genes with uncommon disease associated variants have been identified and there has been some debate about the level of risk that would be appropriate to offer prophylactic surgery. Some have suggested a lifetime risk of 3 to 5 per cent. This would be equivalent to a odds ratio of about 3. The purpose of this topic is not to discuss the rights and wrong of whether an odds ratio threshold of 3 is appropriate but about how we deal with estimates of risk at or around this value.
The example here is provided by the manuscript above. We sequenced the gene PALB2 in 13,000 samples from multiple case-control studies. Protein truncating variants were associated with an increased risk with an odds ratio of 3.0 (95% CI 1.6 – 5.7). The question now arises as to whether or not this is clinically useful.
Given that we cannot be certain what the true odds ratio is, how certain, as a percentage do we need to be that the odds ratio is above 3 to offer prophylactic surgery to carriers of a deleterious variant. I would say at least 50% and perhaps as high as 80%, but definitely not as high as 97.5% (corresponding to the lower 95% confidence limit). At this point it is worth remembering that a frequentist confidence interval does not give us any sort of certainty that the true value is within a given range (despite its name). Ideally, we would have done a formal Bayesian analysis of the original data in order to obtain a distribution of the likely odds rato estimate, but my team and I have not yet managed to convert to this. Instead I will resort to using mixture distributions to combine a prior distribution for the odds ratio and the observed distribution from the frequentist point estimate and standard error to generate a posterior. I have assume the prior distribution of the log(odds ratio) to be normal with mean log(3) and standard deviation 0.65, but truncated at an odds ratio of 1 because it is extremely unlikely that these variants would be associated with a reduced risk. These values were taken from a previously published risk estimate and confidence interval.
The prior, observed and posterior distributions are shown in the Figure. From the posterior distribution there is a 51% chance that the true odds ratio is 3 or greater. However, if we wanted to be 80% certain that the true odds ratio is 3 or greater, these data would be insufficient to warrant offering prophylactic surgery to carriers. If the true odds ratio is 3, even with a huge sample size, we will not have that degree of certainty.
My question for others is how should we think about precision when the true effect is at or around and effect size that might be considered a threshold for decision making? I suspect that some sort of probabilistic thinking around the decision threshold is required, but I would be interested to hear the views of others.