As discussed recently on twitter there is a great deal of uncertainty in the clinical trial community about how to assess treatment safety when there are numerous possible adverse events to monitor. One of the most promising approach is the use of Bayesian hierarchical models to connect various safety outcomes through biologic pathways, as developed by Berry and Berry. Without such an approach it is uncertain about how and whether to adjust for multiplicities in the frequentist world.
A general minimal-assumption multivariate approach was advocated in this presentation. This is based on an idea by Peter O’Brien in which he inverts the binary logistic regression model to predict treatment from outcome. Outcomes have some information for predicting randomization assignment if and only if the randomized treatment is associated with one or more outcomes. By using multiple outcomes to predict treatment, one can use the likelihood ratio \chi^2 statistic as a global test, and it has a perfect multiplicity adjustment because the degrees of freedom of the test is the number of adverse event types tested. One can easily also mix binary adverse events with continuous clinical lab safety variables in running this test.
One drawback from the approach is that it loses power when the number of degrees of freedom is large without many of the adverse events being different between treatments. Also, some events co-occur, creating a co-linearity that wastes degrees of freedom. So a new strategy is proposed.
- Use unsupervised learning to learn about the apparent dimensionality of the problem. For example, use variable clustering or sparse principal components to separate the adverse events into clusters that are non-overlapping, with co-expressed events grouped within clusters.
- Score the separate event/lab data clusters into single numbers, e.g., first principal component or count of number of events
- Use these cluster summaries in the binary logistic model predicting treatment assigned. Now the degrees of freedom is manageable and power will be concentrated.
The presentation linked above has several examples using some of these methods. Comments and questions about this proposed exploratory analysis method are welcomed.