This is a discussion of how best to analyze and to present clinical outcomes when an outcome of major interest can be interrupted by another outcome making the first outcome unobservable. The most common example is when the main outcome of interest is a non-fatal clinical event and the interrupting event is death. For the moment let’s assume that event timing can be ignored, i.e., we deal with short-term follow-up.
Those engaged in clinical trials or are a consumer of clinical trial results were asked via Twitter to fill out this brief 4-question survey. The last question in the survey is for any comments they cared to make, but it better to put your comments here as a reply to this post. Here is the survey’s preface:
Suppose that in a high risk patient population a new treatment is intended to reduce the risk of stroke within 30 days of a certain intervention. Treatments are labeled A and B and 100 patients are randomized to each treatment. Stroke may be interrupted by all-cause death, and death may also occur at or after a stroke. So the count of the number of strokes is the number of patients having a stroke before or at the moment of death.
Suppose the trial results in the following event frequencies, and for the sake of discussion assume that the strokes that occur, while serious, did not result in disability severe enough to be deemed worse than death.
A | B | |||
---|---|---|---|---|
a | stroke, alive | 11 | 15 | |
b | stroke, fatal | 1 | 1 | |
c | stroke, later death | 1 | 1 | |
d | death, no stroke | 5 | 1 | |
e | stroke | 13 | 17 | a + b + c |
f | death | 7 | 3 | b + c + d |
g | stroke or death | 18 | 18 | a + b + c + d |
The survey asks you to select one of the above outcome measures for emphasis if you were forced to emphasize only one.
Survey Results
Survey responses were tabulated on 2021-06-09. Survey respondents provided a number of valuable comments, found here. The results on the multiple choice questions are as follows.
193 survey responders
Choice | Frequency |
---|---|
a stroke, alive | 10 |
b stroke, fatal | 2 |
c stroke, later death | 0 |
d death, no stroke | 2 |
e stroke | 44 |
f death | 19 |
g stroke or death | 116 |
Choice | Frequency |
---|---|
clinical trialist | 40 |
clinician not engaged in clinical trial design | 37 |
statistician | 79 |
epidemiologist | 17 |
other | 20 |
Choice | Frequency |
---|---|
A traditional comparison of two proportions where death and stroke are considered equally bad | 52 |
An ordinal analysis where death is counted as worse than a nonfatal stroke, without assuming how much worse | 141 |
clinical trialist | clinician not engaged in clinical trial design | statistician | epidemiologist | other | |
---|---|---|---|---|---|
a stroke, alive | 2 | 4 | 2 | 0 | 2 |
b stroke, fatal | 0 | 2 | 0 | 0 | 0 |
c stroke, later death | 0 | 0 | 0 | 0 | 0 |
d death, no stroke | 0 | 1 | 1 | 0 | 0 |
e stroke | 6 | 6 | 22 | 5 | 5 |
f death | 3 | 4 | 5 | 4 | 3 |
g stroke or death | 29 | 20 | 49 | 8 | 10 |
clinical trialist | clinician not engaged in clinical trial design | statistician | epidemiologist | other | |
---|---|---|---|---|---|
A traditional comparison of two proportions where death and stroke are considered equally bad | 14 | 6 | 22 | 4 | 6 |
An ordinal analysis where death is counted as worse than a nonfatal stroke, without assuming how much worse | 26 | 31 | 57 | 13 | 14 |
Assessment of Survey Responses
Even though this informal survey undoubtedly had a biased sample, the results are still informative and interesting. I am glad that (1) the most frequent choice in every category of respondent was g: stroke or death
, and (2) the most frequent choice of analysis strategy in every category of respondent was an ordinal analysis where death is counted as worse than a nonfatal stroke, without assuming how much worse
. My take is that death and stroke are not possible to separate, so choices a-e lead to very hard-to-interpret statistical summaries. More generally, my opinion is that any outcome that is interrupted by another outcome, especially one that is worse (such as death). makes the original outcome target hard if not impossible to interpret. And any analysis that uses outcome-conditional probabilities (e.g., P(stroke if alive)) will be a challenge and not respect intent to treat (ITT) since it conditions on a post-randomization outcome. That being the case it is advantageous to count the worse outcome as worse, here by treating the outcomes as ordinal levels A (no stroke), B (stroke), C (death) where death overrides stroke. Had the study been longitudinal with daily outcome assessments, death would not override an earlier stroke but would be counted as the worst outcome on the day of the death. A stream of outcomes for one patient might be for example A A A B C for 5 days.
Even though death and stroke are impossible to separate, an ordinal or a polytomous (aka multinomial or unordered categorical) analysis would allow one to estimate unconditional probabilities respectful of ITT. For example one can estimate P(death), P(death or stroke), P(stroke and alive). Not that the last probability is smaller than P(stroke given alive).
A Better Way
This all leads to a proposed better way to analyze the data and interpret the result. Make a daily assessment of an ordinal scale with, say, 10 levels of stroke severity, and death. The ordinal outcome on a given day would be 1-9 for a non-debilitating stroke, 10 for death, and 11 for a stroke requiring total medical care. An ordinal Markov state transition model can be used for the longitudinal analysis. The results can be stated for example in terms of the probability of being in outcome level y or worse on a given day, or the mean number of days in a range of outcomes. One can estimate the mean number of days with Y > 6 (which might mean moderately severe or severe stroke or death) or the mean number of days with Y > 6 and Y < 10 (moderately severe stroke but alive). Ordinal longitudinal models are detailed here.
As argued later in this post, state transition models yield quantities that are easier to interpret than competing risk analysis. They also elegantly handle recurrent events such as hospitalization.