I have recently had an interesting experience. As I commented in a previous post in Datamethods (https://discourse.datamethods.org/t/is-the-interaction-between-clinical-researchers-and-statisticians-working-my-insight-after-reading-four-failed-clinical-trials-on-immunotherapy/1949) we have recently seen some immunotherapy trials that are declared negative despite the likely existence of benefit. In some cases this was due to the presence of crossing survival curves, in trials analyzed with Cox models or log-rank tests, which were not able to capture the existence of possibly real differences.
What to do next in these circumstances? One possibility would be to repeat the trial with a more appropriate statistical approach, e.g. comparison of milestone survival rates at a late point. However, when we have proposed this approach, there has been some discussions, as it appears that this type of endpoint (% survival at fixed time points) might not be acceptable to regulatory agencies.
As I have understood, the point is that it would be controversial to assume that late benefits of some individuals can be a kind of trade-off, at the expense of the early harm of others. For example, sometimes the median survival of a whole group could be shortened, so that a few could become long-term survivors.
Therefore, the solution for the crossing curves could not go through the repetition of trials just changing the analytical approach, but the stratification by covariates, thus avoiding or trying to avoid that the curves cross. The problem is that in many tumours the predictive and prognostic factors are not really so defined. What it seems to me is that in study protocols it is forbidden to assume that this can happen, even if you can never rule it out…
I also can imagine that if the final benefit (difference is OS rates) is very big, it might not be so far-fetched to design the study even assuming the crossing curves. The problem is when the difference is not so big.
In the end all decision making is a trade-off to the extent that uncertainty is inherent in our work.
Since it is a subject that combines ethics, philosophy and statistics, I would like to ask you for a philosophical reflection. Is it adequate to try to replicate a study with crossing hazards? What do you think?