# Run-in phase in RCTs

**URL:** <https://discourse.datamethods.org/t/run-in-phase-in-rcts/6025>\
**Category:** research methods\
**Tags:** rct\
**Created:** [September 21, 2022, 6:56pm UTC](https://discourse.datamethods.org/t/run-in-phase-in-rcts/6025 "2022-09-21T18:56:17Z")\
**Posts on this page:** 15\
**Page:** 1

<div class="post-metadata">

**Author:** ![arthur\_albuquerque](https://discourse.datamethods.org/user_avatar/discourse.datamethods.org/arthur_albuquerque/32/3269_2.png) [@arthur\_albuquerque](https://discourse.datamethods.org/u/arthur_albuquerque)\
**Post date:** [September 21, 2022, 6:56pm UTC](https://discourse.datamethods.org/t/run-in-phase-in-rcts/6025/1 "2022-09-21T18:56:17Z")

</div>

The well-known [PARADIGM-HF trial](https://www.nejm.org/doi/full/10.1056/NEJMoa1409077) compared the LCZ696 drug (Entresto) to the standard of care, enalapril.

I am aware that there are multiple discussions about this trial, but I would like to focus in only one: the **run-in phase**.

Prior to randomization to either Entresto or enalapril, all patients used Entresto for ~29 days. Then, those who didn’t discontinued (see examples below), were randomized.

 ![image](https://discourse.datamethods.org/uploads/default/original/2X/c/cbe03e49160020b9dcb7510e8e2c65b95f3ef6dc.png)

Thus, only patients that tolerated Entresto were randomized.

### Does a run-in phase induce any kind of bias in the results of RCTs?

---

<div class="post-metadata">

**Author:** ![pmbrown](https://discourse.datamethods.org/user_avatar/discourse.datamethods.org/pmbrown/32/1382_2.png) [@pmbrown](https://discourse.datamethods.org/u/pmbrown)\
**Post date:** [September 21, 2022, 11:05pm UTC](https://discourse.datamethods.org/t/run-in-phase-in-rcts/6025/2 "2022-09-21T23:05:41Z")

</div>

no, but the claim of efficacy is affected by the exclusion. I’ve seen trial results presented at a medical conference where they displayed 3 groups: the 2 randomised groups and the non-randomised group i.e. they followed up those who were not randomised - in this case you have a problem.

---

<div class="post-metadata">

**Author:** ![arthur\_albuquerque](https://discourse.datamethods.org/user_avatar/discourse.datamethods.org/arthur_albuquerque/32/3269_2.png) [@arthur\_albuquerque](https://discourse.datamethods.org/u/arthur_albuquerque)\
**Post date:** [September 21, 2022, 11:13pm UTC](https://discourse.datamethods.org/t/run-in-phase-in-rcts/6025/3 "2022-09-21T23:13:55Z")

</div>

What do you mean by “affected”?

---

<div class="post-metadata">

**Author:** ![R\_cubed](https://discourse.datamethods.org/user_avatar/discourse.datamethods.org/r_cubed/32/1518_2.png) [@R\_cubed](https://discourse.datamethods.org/u/R_cubed)\
**Post date:** [September 21, 2022, 11:50pm UTC](https://discourse.datamethods.org/t/run-in-phase-in-rcts/6025/4 "2022-09-21T23:50:39Z")

</div>

I recall reading something similar in anti-depressant RCTs years ago in an effort to remove those who responded to placebo.

> **[Placebo washout is not a meaningful part of antidepressant drug trials - PubMed](https://pubmed.ncbi.nlm.nih.gov/8570378/)**
>
> Most trials testing the effectiveness of psychotropic drugs begin with a placebo washout phase. Hypothetically this technique rids studies of placebo responders before randomization of subjects to drug and placebo groups. In theory, this lowers the...

Seems like the RCT done the way you describe will under-estimate of negative effects in the clinic.

[https://www.bmj.com/rapid-response/2011/10/30/placebo-washouts-inflate-antidepressants-general-practice-selection-bias](https://www.bmj.com/rapid-response/2011/10/30/placebo-washouts-inflate-antidepressants-general-practice-selection-bias)

---

<div class="post-metadata">

**Author:** ![pmbrown](https://discourse.datamethods.org/user_avatar/discourse.datamethods.org/pmbrown/32/1382_2.png) [@pmbrown](https://discourse.datamethods.org/u/pmbrown)\
**Post date:** [September 22, 2022, 7:32am UTC](https://discourse.datamethods.org/t/run-in-phase-in-rcts/6025/5 "2022-09-22T07:32:25Z")

</div>

sorry, i just mean the phrasing, they ought to say the efficacy estimate is relevant for those who tolerate Entresto.

---

<div class="post-metadata">

**Author:** ![arthur\_albuquerque](https://discourse.datamethods.org/user_avatar/discourse.datamethods.org/arthur_albuquerque/32/3269_2.png) [@arthur\_albuquerque](https://discourse.datamethods.org/u/arthur_albuquerque)\
**Post date:** [September 22, 2022, 3:24pm UTC](https://discourse.datamethods.org/t/run-in-phase-in-rcts/6025/6 "2022-09-22T15:24:07Z")

</div>

Could one say that a run-in phase diminishes the external validity of RCTs?

---

<div class="post-metadata">

**Author:** ![s\_doi](https://discourse.datamethods.org/letter_avatar_proxy/v4/letter/s/ea666f/32.png) [@s\_doi](https://discourse.datamethods.org/u/s_doi)\
**Post date:** [September 23, 2022, 4:43am UTC](https://discourse.datamethods.org/t/run-in-phase-in-rcts/6025/7 "2022-09-23T04:43:25Z")

</div>

RCTs are not designed to have external validity - they just need to show that Entresto works as intended in the population defined by the trial. Its up to the physician to generalize beyond eligibility (based on their content expertise or other criteria).  
see [How to assess the external validity of therapeutic trials: a conceptual approach](https://academic.oup.com/ije/article/39/1/89/710528)

---

<div class="post-metadata">

**Author:** ![pmbrown](https://discourse.datamethods.org/user_avatar/discourse.datamethods.org/pmbrown/32/1382_2.png) [@pmbrown](https://discourse.datamethods.org/u/pmbrown)\
**Post date:** [September 23, 2022, 11:13am UTC](https://discourse.datamethods.org/t/run-in-phase-in-rcts/6025/8 "2022-09-23T11:13:05Z")

</div>

yes, but the rct has a particular purpose: [Statistical Thinking - Randomized Clinical Trials Do Not Mimic Clinical Practice, Thank Goodness](https://hbiostat.org/blog/post/rct-mimic/) Discontinuations would reduce power because any imputation method should draw the treatment groups closer together and muddy interpretation, so i can see the desire to anticipate and remove them

incidentally, it’s a difficult problem with the vaccine trials because the rct’s produce an estimate of efficacy and then subsequent to this there is an estimate on so called real world data which yields a lower estimate. This generates vaccine scepticism, people think the trial was rigged, but they have different purpose and answer different questions

---

<div class="post-metadata">

**Author:** ![arthur\_albuquerque](https://discourse.datamethods.org/user_avatar/discourse.datamethods.org/arthur_albuquerque/32/3269_2.png) [@arthur\_albuquerque](https://discourse.datamethods.org/u/arthur_albuquerque)\
**Post date:** [September 23, 2022, 3:07pm UTC](https://discourse.datamethods.org/t/run-in-phase-in-rcts/6025/9 "2022-09-23T15:07:50Z")

</div>

Very nice paper, thanks. Following their definitions, a run-in phase might decrease generalizability.

This discussion reminded of another issue: only including “high-risk” patients in RCTs, eg [https://www.nejm.org/doi/full/10.1056/NEJMoa1704559](https://www.nejm.org/doi/full/10.1056/NEJMoa1704559)

In clinical practice, these RCTs end up limiting the treatment to high-risk patients, because healthcare systems won’t generalize the results to lower risk patients.

I wonder if this is ideal or not…

---

<div class="post-metadata">

**Author:** ![s\_doi](https://discourse.datamethods.org/letter_avatar_proxy/v4/letter/s/ea666f/32.png) [@s\_doi](https://discourse.datamethods.org/u/s_doi)\
**Post date:** [September 24, 2022, 6:31am UTC](https://discourse.datamethods.org/t/run-in-phase-in-rcts/6025/10 "2022-09-24T06:31:18Z")

</div>

To paraphrase from [this study](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3369519/), “the NEJM study participants were likely different in many ways from groups to which we might want to “generalize” the principal results. But, perhaps, they were not fundamentally different in ways that would affect our conclusions about the effect of aspirin on pregnancy outcomes and ultimately pre-eclampsia”.

---

<div class="post-metadata">

**Author:** ![cwatson](https://discourse.datamethods.org/user_avatar/discourse.datamethods.org/cwatson/32/2970_2.png) [@cwatson](https://discourse.datamethods.org/u/cwatson)\
**Post date:** [September 24, 2022, 3:08pm UTC](https://discourse.datamethods.org/t/run-in-phase-in-rcts/6025/11 "2022-09-24T15:08:18Z")

</div>

I am neither a MD nor a statistician but it seems to me that their procedures introduce some bias. To give the authors benefit of the doubt, if there’s not a (serious) bias then I think that the paper’s title (as well as the _Background_ and _Conclusions_ sections of the Abstract) is too general as it implies that the study results are applicable to **all** “_patients who had heart failure with a reduced ejection fraction_”. (Of course, authors must contend with word/character limits)

For example, wouldn’t the initial run-in phases alter either the patients’ physiology or disease progression in ways that wouldn’t mimic a “new” patient? That is, would the outcome be different if the patient first presented with heart failure (HF), never having taken any ACE inhibitor or related drugs? Or whenever a doctor would first prescribe such a medication for treatment of HF. I only glanced at a few replies to the study but it looks like one letter to the Editor expressed a similar point to this.

They also had an initial run-in phase of 2-3 weeks in which participants received enalapril (10 mg twice daily) prior to the run-in phase with LCZ696. N = 1,102 patients discontinued the study at this phase (N = 591 for having an adverse event). However, in the Introduction the authors state that

> The effect of angiotensin-receptor blockers (ARBs) on mortality has been inconsistent,[3,4](https://www.nejm.org/doi/full/10.1056/NEJMoa1409077#) and thus, these drugs are recommended primarily for patients who have unacceptable side effects (primarily cough) while receiving ACE inhibitors.

So shouldn’t they _want_ to assess the effect of the new drug in these patients (i.e., those N = 591 who were excluded due to adverse side effects from enalapril), in addition to those who underwent randomization?

They “bury” the following in the second-to-last paragraph of the paper (the paragraph which is worded like a Limitations section without calling it one, which I personally dislike):

> During enrollment, we evaluated patients who were already taking various doses of ACE inhibitors or ARBs and required that they be able to take the equivalent of a relatively low dose of enalapril (10 mg daily) without unacceptable side effects.

But if LCZ696 is intended to replace enalapril (or other ACE inhibitors), then “does it matter” that taking enalapril would result in adverse side effects?

---

<div class="post-metadata">

**Author:** ![PaoloTolomeo2](https://discourse.datamethods.org/user_avatar/discourse.datamethods.org/paolotolomeo2/32/3579_2.png) [@PaoloTolomeo2](https://discourse.datamethods.org/u/PaoloTolomeo2)\
**Post date:** [March 30, 2024, 4:12pm UTC](https://discourse.datamethods.org/t/run-in-phase-in-rcts/6025/12 "2024-03-30T16:12:35Z")

</div>

Dear @f2harrell , I would be happy to know your opinion about this. Do you think run-in phase has to be considered a source of bias in clinical trials?

---

<div class="post-metadata">

**Author:** ![f2harrell](https://discourse.datamethods.org/user_avatar/discourse.datamethods.org/f2harrell/32/165_2.png) [@f2harrell](https://discourse.datamethods.org/u/f2harrell)\
**Post date:** [April 1, 2024, 1:34pm UTC](https://discourse.datamethods.org/t/run-in-phase-in-rcts/6025/13 "2024-04-01T13:34:24Z")

</div>

@Stephen is the definitive source on this, including his book Statistical Methods in Drug Development.

---

<div class="post-metadata">

**Author:** ![PaoloTolomeo2](https://discourse.datamethods.org/user_avatar/discourse.datamethods.org/paolotolomeo2/32/3579_2.png) [@PaoloTolomeo2](https://discourse.datamethods.org/u/PaoloTolomeo2)\
**Post date:** [April 1, 2024, 7:32pm UTC](https://discourse.datamethods.org/t/run-in-phase-in-rcts/6025/14 "2024-04-01T19:32:31Z")

</div>

Thanks. I hope @Stephen will share his thought. I’ll have a look at his book.

---

<div class="post-metadata">

**Author:** ![f2harrell](https://discourse.datamethods.org/user_avatar/discourse.datamethods.org/f2harrell/32/165_2.png) [@f2harrell](https://discourse.datamethods.org/u/f2harrell)\
**Post date:** [April 1, 2024, 8:26pm UTC](https://discourse.datamethods.org/t/run-in-phase-in-rcts/6025/15 "2024-04-01T20:26:52Z")

</div>

Sorry I misquoted the title. Should be _Statistical Issues in Drug Development_.