Change of study design of an ongoing clinical trial

treatment comparison design
,
1

I have a general question about a statistical analysis plan (SAP) and the change of study design of an ongoing trial.
Suppose you joined a study team while a trial is ongoing, which was designed with a traditional non-sequential design by a previous statistician. The SAP was already written by this statistician and the protocol for this trial as well as the key components of the SAP has been already published. You are currently reviewing this SAP and need to sign off the SAP before the enrollment will be done, which has not been completed yet (e.g., 1/2 or 2/3 of the planned sample size was enrolled). However, a situation arises (e.g., some logistical issues), which delayed the trial and could justify a sequential design (instead of the planned traditional design). If this logistical issue is nothing to do with the study outcomes (i.e., the change of design would not lead to bias), can we change the study design even though the originally planned design has been published as a protocol paper?
I think it is O.K. to change the design if it is scientifically justifiable and would benefit the trial, but I’d like to hear other people’s thoughts about this.

2

Hi,

I think that we might need more information here. The questions that come to mind for me:

  1. Is this RCT being used for a regulatory submission for a drug or medical device?

  2. Was the sample size for the study determined based upon formal a priori power/sample size calculations/simulation?

  3. Are you keeping the original primary endpoint intact, or will that also be modified in some fashion?

  4. Is the desire to introduce sequential analyses due to very prolonged study enrollment (perhaps due to COVID), and the desire to make an early decision now to stop the study for efficacy or futility (in the absence of safety issues)?

  5. If you conduct an interim analysis, and you do not stop the study at that point, have you considered who will have access to that interim information (e.g. a DSMB/DMC) and what communication plan may be needed to ensure that the ongoing conduct of the trial is not affected by the knowledge of the interim results?

1 Like
3

Thank you so much for your reply and thoughtful questions. Please see my answers to your questions below:

  1. Is this RCT being used for a regulatory submission for a drug or medical device?
    – It is a repurposing drug that is already approved, and there is no regulatory submission.
  2. Was the sample size for the study determined based upon formal a priori power/sample size calculations/simulation?
    – The sample size was a priori calculated analytically within the frequent paradigm.
  3. Are you keeping the original primary endpoint intact, or will that also be modified in some fashion?
    – All endpoints will be intact.
  4. Is the desire to introduce sequential analyses due to very prolonged study enrollment (perhaps due to COVID), and the desire to make an early decision now to stop the study for efficacy or futility (in the absence of safety issues)?
    – The main goal of resigning the trial is to use sequential analyses due to very prolonged study, but the study could also benefit from early stopping for efficacy or futility (and no safety issues) if warranted.
  5. If you conduct an interim analysis, and you do not stop the study at that point, have you considered who will have access to that interim information (e.g. a DSMB/DMC) and what communication plan may be needed to ensure that the ongoing conduct of the trial is not affected by the knowledge of the interim results?
    – I am thinking a two-stage Bayesian design with sample size re-estimation. If we do not stop the trial, it is likely due to lack of power (i.e., need larger sample size). So I think it would minimally affect the ongoing conduct of the trial (also the study team is blinded). But this should be further discussed with the study team.
4

Hi,

Thanks for the reply.

The question as to whether or not this was for a regulatory submission, is that, in the end, it would be up to the regulators (e.g. FDA), and their statisticians, to approve the change in the study design before proceeding with it, as an outcome of discussions with them and any concerns that they might raise.

Also, just to clarify, you used the word “repurposing” above, which I take to perhaps mean a potential change in the indication for the drug. If so, then that would typically involve regulatory issues at some point, since it would require a label change to have it approved for that indication, if that is desired.

If this is not for a regulatory submission now, then the potential downstream hurdles would be any peer review challenges in the course of a journal submission, while being fully transparent in any publications/presentations about the change made to the study, the rationale for the change, and any potential impact on the study conduct and the results.

If the study was registered at ClinicalTrials.gov or similar, the sponsor will want to make the appropriate changes there as well, and of course, new IRB notifications and approvals for the new protocol. If this is a multi-site study, you will want to consider the potential impact, if any, of the possibly staggered implementation of the protocol change across the sites.

A key goal, in the end, is to try to find an appropriate approach that maximizes the value of the time invested and the work that has already been done, the financial investment by the sponsor and the risks faced by the already enrolled patients, in trying to answer a scientific question. The potential alternative, in the face of the challenges being confronted, is perhaps not being able to complete the study at all, or not in a time frame where the results would still be clinically relevant, under the original parameters.

In terms of the change itself, the introduction of previously unplanned interim analyses could adversely impact your sample size requirements. So, if you are already having enrollment challenges, if that is the key motivation for the change, the potential gotcha is that the interim analyses may only make that situation worse by increasing the required sample size over the original estimate, which you touch on in your reply.

Ideally, the interim analysis will provide some level of evidence that continuing the study is justified, because the direction and magnitude of the observed treatment effect are clinically relevant, in the absence of any safety issues. So the potential benefit of that information would hopefully outweigh the risks of any sample size increase and the impact on the study timeline and costs.

A best case scenario of sorts, is that the interim analysis would be to enable you to make an informed early stopping decision for efficacy given pre-defined thresholds for that decision, saving time, money and reducing the number of patients exposed to the risks of the study. However, keep in mind that one is likely to get optimistic estimates of treatment effect sizes in that setting.

You note the potential issue of diminished conditional power at the interim analysis, which would also become one of the parameters in making a futility decision, and you will need to define a priori conditional power thresholds for that decision as well. In confirmatory trials, I have seen conditional power thresholds for futility decisions as high as 20% and as low as 5%, and it all depends upon the risk that the sponsor is willing to take to continue the trial in the face of that information, since futility decisions are typically non-binding.

I do think that you will want to be thoughtful about a communication plan, including “what if” scenarios, regarding the interim analysis, since this new design was not anticipated when the study began. Who gets to review the results, who is involved in any decision making regarding the need to make a possible stopping decision, who do they communicate that decision/recommendation to so as not to affect trial conduct, what happens if the recommendation is ignored and the study continues, and what happens if we get “close to” but do not cross a futility decision boundary? Speaking from experience from serving on DSMBs, that communication plan is important to think about.

Importantly, if there is not a DSMB in place currently for this trial, I would argue that one should be convened to provide the venue for that independent review and recommendation process, if you proceed with the change in the plan.

3 Likes
5

For clarification of “repurposing”, it was not meant for the change in the indication, so no regulatory submission for a label change will be planned. I also recognize potential challenges in a peer review process while trying to publish the results, which concerns me as well.
All your points are great, and I really appreciate you providing the detailed guidance on things to consider before redesigning the trial. Based on all these points, it seems redesigning the trial with the interim analysis would not gain much considering potential adversary it might bring in.
Thank you so much again for your very thoughtful comments and guidance.

2 Likes