Bayesian Re-Analysis of Clinical Trial Data

interpretation
bayes

#1

@DanLane911 and I have been working on a broader application of the Bayesian re-analysis application that was originally built for the ANDROMEDA-SHOCK trial. I was hoping to get some feedback on usability and ideas for improvement given that this is the audience most likely to use this type of tool. Right now the app is very much still under construction, but most of the basic functionality is implemented.

Shiny link: https://benjamin-andrew.shinyapps.io/bayesian_trials/

If the above link is not working, you can run the most recent version on your local machine (assuming you have Shiny installed) using: runGitHub(repo = “bayesian_trials”, username = “andrew10043”)


#2

So glad you continue to generalize this! It’s neat that you are asking for the number of events separately by treatment group, which I assume you are using to get a more accurate estimate of the standard error of the log hazard ratio by summing the two reciprocals of number of events. It would be good to include in the output the comparison of the standard error using this approach with that obtained by backsolving from the confidence interval for the log HR.

One cosmetic suggestion that is a very minor thing is to see if shading tail areas or equivalence zones in the posterior densities would add something.


#3

I had actually included event rates to allows for interpretation of studies with dichotomous outcomes (there is a drop down box that allows you to select HR or OR which then toggles the entry fields). In the current version, the standard deviation of the log HR likelihood is estimated as:

s = \frac{log(UCI) - log(HR)}{qnorm(0.975)}

I see, though, that the standard error of log HR can (as you point out) be estimated using:

SE = \sqrt{\frac{1}{E_1} + \frac{1}{E_2}}

where

E_1 = events\ in\ treatment\ group\\ E_2 = events\ in\ control\ group

Is this a more appropriate estimate for the standard deviation of the log HR likelihood?


#4

It’s as close as we have to a gold standard, so I would use it but list beside it the back-computed s from your first formula to show the reader a check. Maybe better would be to use log(UCL) - log(LCL) and to adjust the denominator accordingly. This would be very slightly less affected by roundoff perhaps.

If the user finds that the SE of the log HR was published, they should be able to enter that and to have it compared with the other 2 SE estimates.


#5

Great, I’ll start working on that update. For dichotomous outcome studies I am using the following for the likelihood:

\theta = log\frac{(a + \frac{1}{2})(d + \frac{1}{2})}{(b + \frac{1}{2})(c + \frac{1}{2})}
s = \sqrt{\frac{1}{a + \frac{1}{2}} + \frac{1}{b + \frac{1}{2}} + \frac{1}{c + \frac{1}{2}} + \frac{1}{d + \frac{1}{2}}}

where a = event in intervention group, b = event in control group, c = no event in intervention group and d = no event in control group.

Are there alternative estimates that would be more appropriate here, or are these the best to move forward with? Thank you for your help!


#6

That’s right, for the log OR. I don’t routinely use the 1/2 but I think that using it is better than not using it. You might ask for the published SE and recalculate it for the user with and without the 1/2, then say you’re going to use the 1/2 for what follows. Of course what we really want is adjusted OR and HR but then we don’t have simple formulas. Anyway, RCTs shy away from these better adjusted estimates, preferring to lose power instead of having to get a statistician to explain the result :smile:.


#7

A few of these updates have been implemented (screenshots attached). There is now a separate “study data” page that uses user input to dynamically guide data entry. As @f2harrell recommended, the various approaches to estimating the SE of the point estimate have been included, and the best option is chosen to build the likelihood. The other values are displayed for the user’s reference. Still working on some of the aesthetic changes as these are less in my wheelhouse.


#8

Thinking back to dichotomous outcome studies. Assuming all event rates are reported, could this be similarly broadened to RR as well as OR?

For RR:

L \sim N(\theta, s)\\ \theta = log\frac{\frac{a}{a + c}}{\frac{b}{b + d}}\\ s = \sqrt{\frac{c}{a(a + c)} + \frac{d}{b(b + d)}}

where a = event in intervention group, b = event in control group, c = no event in intervention group and d = no event in control group.

Is there a role for the addition of the 1/2 term in these estimates (as there was for OR)?


#9

Hiya guys.

I’m a clinician, not a statistician, so be gentle!

I’ve tried to use this to re-analyse SPRINT-MIND - https://jamanetwork.com/journals/jama/fullarticle/2723256

They use a HR, not RR or OR, so not sure which to choose.

Also, and this is possibly a moot point, but I guess might affect my interpretation.
The study showed that lowering BP reduced mild cognitive impairment (
I think, from my prior knowledge, MCI is strongly linked to dementia, and is on the causal pathway.

Am i allowed to use that information to adjust my prior? Knowing the study reduced a variable on the causal pathway to the primary outcome?

Thanks
Gus

Infectious Disease, Bristol, UK.


#10

Hi Gus —

I’m neither a clinician nor a statistician, but hopefully I can help!

Using the “probable dementia” outcome, I would set up the app as shown in the attached images (you can play with the prior here). You can use the HR + event rates to get a good estimate of the standard error for the log(HR) which helps to reconstruct the likelihood function.

I would agree that causally MCI is on the path to dementia (although some etiologies of dementia may occur independently of MCI, I suppose, such as a more abrupt onset vascular dementia for example). I think when setting priors for this study it would be incorrect to use information gained from the study itself (e.g., using the MCI outcome to change prior for the dementia outcome analysis). Moving forward it would be reasonable to use the signal for MCI when constructing a prior for dementia studies, I would think, given that we theorize that MCI is an intermediate step on the causal path towards dementia.

I’ll let the more experienced group chime in, though, as I suspect there is more to add!


#11

This is fantastic, Ben. Thanks so much.

Just a really great example of this sort of work. My fiance is an old-age psychiatrist, so hence my (vague) interest in dementia as an ID clinician.

I showed her last night the influence of her skeptical prior on the results and it really became clear how to combine her prior information with this new work.

As you say about the MCI adjusting your dementia prior - I agree - I guess you can’t use it!

Thanks
Gus


#12

Happy to help! Let me know if any other questions should arise, or you have any difficulty with the app in the future.