How to conduct and analyze utility-focused trials

In a prior thread, I had mentioned that a trial should provide the joint distribution of outcomes p(mortality, side effect|treatment, patient covariates), and I appreciated the suggestion that one could also use an ordinal model with outcomes ranked by utility. I also see a post on adaptive trials, a post on meaningful clinical trials, and a post on ordinal modeling.

I hope this topic can serve as a resource for how to conduct and analyze utility-focused trials.

Suppose we are assessing a vaccine. We could randomize n people to receive it and then some number of years later assess protection against the disease of interest and also side effects. How to then collect utilities (or not) and how to analyze?

1 Like

I write a bit below on different approaches (understanding is still in progress for some).

Joint outcome distribution

For the joint distribution, we would have outcomes (disease present, vaccinated, side effects), (disease present, vaccinated, no side effects),…, (disease present, unvaccinated, side effects), etc. We would estimate the joint probability of the outcome, p(disease present, side effects| vaccine, patient info).

Benefits:

Formal application at point of care: if a patient assigns utilities to outcomes, one can perform a full expected utility calculation.

Drawbacks:

Difficult from a statistical standpoint to estimate a joint distribution

Formal application at point of care involves challenging optimization.

Formal application at point of care involves elicitation of utilities.

Conditional outcome distributions

We would estimate p(disease present|vaccine, patient info) and p(side effects|vaccine, patient info).

Benefits:

Sometimes, heuristic medical decisions making involves these conditional probabilities; ie, a patient asks about (or provider thinks about) the chance of side effects and chance the vaccine protects them, and they decide from there. So it could be helpful to have these quantities.

We have tools such as logistic regression to estimate conditional probabilities like this.

Drawbacks:

These conditional probabilities are insufficient to perform a full expected utility calculation, so a patient is just given the risks and benefits and expected to somehow perform this calculation themselves

Although easier than the joint, it would still be difficult to estimate these conditional probabilities.

Ordinal model (not sure if I have this correct, still thinking about the thread mentioned above and the resources here and here)

With an ordinal model, we might have the outcomes (disease present, side effects), (disease present, no side effects), (disease absent, side effects), (disease absent, no side effects), where these would be ranked by participants, and we would fit an ordinal model with rank as the outcome and vaccination status as a covariate. Then, the coefficient corresponding to the vaccination status would be a treatment effect. We would also have patient info as covariates.

Benefits:

Estimation and inference feasible

We don’t need to elicit or quantify utility (I actually don’t know if we would even need patients to agree on the outcomes; each patient would just supply a ranking, and then we could model their ranks without thinking further about the outcomes that they correspond to)

Drawbacks:

Some information reduction when converting outcomes to ranks, especially if the outcomes are continuous

In the thread, had also mentioned multistate models; I think of this as a Markov chain, and perhaps then we are estimating a matrix of transition probabilities. To me though that seems similar to estimating a joint distribution.

1 Like