# Non inferiority Margin without Placebo controlled Trial

**URL:** <https://discourse.datamethods.org/t/non-inferiority-margin-without-placebo-controlled-trial/4250>\
**Category:** general\
**Created:** [March 31, 2021, 7:31am UTC](https://discourse.datamethods.org/t/non-inferiority-margin-without-placebo-controlled-trial/4250 "2021-03-31T07:31:27Z")\
**Posts on this page:** 14\
**Page:** 1

<div class="post-metadata">

**Author:** ![ashwinim13](https://discourse.datamethods.org/letter_avatar_proxy/v4/letter/a/c5a1d2/32.png) [@ashwinim13](https://discourse.datamethods.org/u/ashwinim13)\
**Post date:** [March 31, 2021, 7:31am UTC](https://discourse.datamethods.org/t/non-inferiority-margin-without-placebo-controlled-trial/4250/1 "2021-03-31T07:31:27Z")

</div>

How to estimate the Non inferiority Margin/Equivalence Margin if we do not have Placebo controlled Trial

---

<div class="post-metadata">

**Author:** ![f2harrell](https://discourse.datamethods.org/user_avatar/discourse.datamethods.org/f2harrell/32/165_2.png) [@f2harrell](https://discourse.datamethods.org/u/f2harrell)\
**Post date:** [March 31, 2021, 3:03pm UTC](https://discourse.datamethods.org/t/non-inferiority-margin-without-placebo-controlled-trial/4250/2 "2021-03-31T15:03:08Z")

</div>

This is not the answer you are looking for, but the tradition of determining noninferiority margins from previous studies has got to end IMHO. Non-inferiority should be based on what’s important to the patient no matter how well an active control has done in comparison to placebo previously.

---

<div class="post-metadata">

**Author:** ![bythl](https://discourse.datamethods.org/user_avatar/discourse.datamethods.org/bythl/32/2245_2.png) [@bythl](https://discourse.datamethods.org/u/bythl)\
**Post date:** [June 24, 2021, 7:42am UTC](https://discourse.datamethods.org/t/non-inferiority-margin-without-placebo-controlled-trial/4250/3 "2021-06-24T07:42:00Z")

</div>

Is this then equivalent to the [minimal clinically important difference](https://jamanetwork.com/journals/jama/article-abstract/1910090)? Many trials seem to still derive this from previous studies when it ought to be from patients or subject matter experts. And [others](https://www.ahajournals.org/doi/10.1161/CIRCULATIONAHA.118.031313) just pick a value…

There has been [some work](https://www.tandfonline.com/doi/abs/10.1080/10543406.2015.1074920?journalCode=lbps20) done on Bayesian noninferiority trial designs - but is a “noninferiority trial” not really a purely frequentist concept? A trial analysed with Bayesian modelling could produce an estimate of treatment effect anywhere on the spectrum from inferiority to equivalence to superiority. The outcome might be noninferiority, but there doesn’t seem to be a need to specify this as the goal at the outset under this paradigm.

---

<div class="post-metadata">

**Author:** ![pmbrown](https://discourse.datamethods.org/user_avatar/discourse.datamethods.org/pmbrown/32/1382_2.png) [@pmbrown](https://discourse.datamethods.org/u/pmbrown)\
**Post date:** [June 24, 2021, 8:57am UTC](https://discourse.datamethods.org/t/non-inferiority-margin-without-placebo-controlled-trial/4250/4 "2021-06-24T08:57:07Z")

</div>

if i remember correctly Senn argued against pre-specification for the non-inferiority margin, but that may have been many years ago, or even just an off hand remark made at a conference

edit: actually it’s in his book, section 15.2.9 It is essential that the noninferiority margin be prespecified: “What this suggests is that it is not prespecification of the interval of equivalence  
that is important but rather the interval that may be believed to obtain once the data  
have been analysed. A cynical interpretation of current accepted practice is that the  
regulators are merely abandoning their obligation to decide what is and what is not  
equivalent and attempting to hide behind prespecification. To be fair, these guidelines  
do require that the margin should also be medically justified in some way, but it is not  
clear why this can be left to the sponsor.”

---

<div class="post-metadata">

**Author:** ![f2harrell](https://discourse.datamethods.org/user_avatar/discourse.datamethods.org/f2harrell/32/165_2.png) [@f2harrell](https://discourse.datamethods.org/u/f2harrell)\
**Post date:** [June 24, 2021, 11:40am UTC](https://discourse.datamethods.org/t/non-inferiority-margin-without-placebo-controlled-trial/4250/5 "2021-06-24T11:40:05Z")

</div>

> [@bythl](#):
>
> has been [some work](https://www.tandfonline.com/doi/abs/10.1080/10543406.2015.1074920?journalCode=lbps20) done on Bayesian noninferiority trial designs - but is a “noninferiority trial” not really a purely frequentist concept?

It is so trivial to do with Bayesian posterior inference that I think you’re right there is little need for it in the Bayesian paradigm.

---

<div class="post-metadata">

**Author:** ![R\_cubed](https://discourse.datamethods.org/user_avatar/discourse.datamethods.org/r_cubed/32/1518_2.png) [@R\_cubed](https://discourse.datamethods.org/u/R_cubed)\
**Post date:** [June 24, 2021, 7:15pm UTC](https://discourse.datamethods.org/t/non-inferiority-margin-without-placebo-controlled-trial/4250/6 "2021-06-24T19:15:39Z")

</div>

> Blockquote  
> How to estimate the Non inferiority Margin/Equivalence Margin if we do not have Placebo controlled Trial.

There seems to be a distinction between a “non-inferiority” trial (where the control is a treatment presumed to be effective), and an efficacy trial, that is compared to placebo (presumed to be inert).

The following has a nice chart that elaborates on the distinction.

> **[Noninferiority and Equivalence Designs: Issues and Implications for Mental...](https://pmc.ncbi.nlm.nih.gov/articles/PMC2696315/)**
>
> “Noninferiority” and “equivalence” are often used interchangeably to refer to trials in which the primary objective is to show that a novel intervention is as effective as the standard intervention. The use of these designs is becoming increasingly...

The regions and the frequentist procedure to provide evidence against the default:

1. Noninferiority – .One sided test against lower bound of effect size of interest.
2. Equivalence – TOST (Two one-sided tests).
3. Efficacy – Two Sided conventional test against effect of exactly 0.

I do think the Bayesian intervals or Jeffrey Blume’s interval intersection procedure are interesting alternatives.

> **[Statistical Evidence | SGPV](https://www.statisticalevidence.com/second-generation-p-values)**
>
> Second-Generation p-values, Statistical Evidence, SGPV

---

<div class="post-metadata">

**Author:** ![Vitaly\_Druker](https://discourse.datamethods.org/user_avatar/discourse.datamethods.org/vitaly_druker/32/192_2.png) [@Vitaly\_Druker](https://discourse.datamethods.org/u/Vitaly_Druker)\
**Post date:** [June 25, 2021, 5:43pm UTC](https://discourse.datamethods.org/t/non-inferiority-margin-without-placebo-controlled-trial/4250/7 "2021-06-25T17:43:42Z")

</div>

I’ve been struggling with this recently - especially in the context of efficacy testing vs. non inferiority testing. Specifically, why do we talk about efficacy testing as a two sided test with alpha = 0.05? If the investigational treatment is ‘significantly worse’ and the NH test has a p-value of \< .05 then wouldn’t we still say that the trial fails? Why don’t we just do one sided tests with alpha = .025?

---

<div class="post-metadata">

**Author:** ![R\_cubed](https://discourse.datamethods.org/user_avatar/discourse.datamethods.org/r_cubed/32/1518_2.png) [@R\_cubed](https://discourse.datamethods.org/u/R_cubed)\
**Post date:** [June 26, 2021, 12:21am UTC](https://discourse.datamethods.org/t/non-inferiority-margin-without-placebo-controlled-trial/4250/8 "2021-06-26T00:21:13Z")

</div>

With inferiority testing, the default hypothesis is that the treatment is inferior, so there is no interest in any effect below the lower bound of the MCID,

When testing at a presumed effect of 0, a significant true negative result would be surprising, and likely reported. If you just test for one side, you miss significant effects in the opposite direction.

---

<div class="post-metadata">

**Author:** ![Stephen](https://discourse.datamethods.org/user_avatar/discourse.datamethods.org/stephen/32/201_2.png) [@Stephen](https://discourse.datamethods.org/u/Stephen)\
**Post date:** [July 27, 2021, 8:08am UTC](https://discourse.datamethods.org/t/non-inferiority-margin-without-placebo-controlled-trial/4250/9 "2021-07-27T08:08:28Z")

</div>

THe clinically irrelevant difference would usually be much lower than the clinically relevant difference. (IMO, certainly no more than 1/3 of the former.) There is some related discussion here [Minimally important differences v2 (slideshare.net)](https://www.slideshare.net/StephenSenn1/minimally-important-differences-v2)

---

<div class="post-metadata">

**Author:** ![arthur\_albuquerque](https://discourse.datamethods.org/user_avatar/discourse.datamethods.org/arthur_albuquerque/32/3269_2.png) [@arthur\_albuquerque](https://discourse.datamethods.org/u/arthur_albuquerque)\
**Post date:** [August 2, 2021, 3:32pm UTC](https://discourse.datamethods.org/t/non-inferiority-margin-without-placebo-controlled-trial/4250/10 "2021-08-02T15:32:08Z")

</div>

Dr. Harrell,

Could you elaborate on the relationship between “Bayesian posterior inference” and noninferiority analysis?

---

<div class="post-metadata">

**Author:** ![f2harrell](https://discourse.datamethods.org/user_avatar/discourse.datamethods.org/f2harrell/32/165_2.png) [@f2harrell](https://discourse.datamethods.org/u/f2harrell)\
**Post date:** [August 2, 2021, 3:38pm UTC](https://discourse.datamethods.org/t/non-inferiority-margin-without-placebo-controlled-trial/4250/11 "2021-08-02T15:38:42Z")

</div>

Briefly, with classic frequentist non-inferiority testing you set a non-inferiority margin, i.e., how much harm can the treatment get away witih before you wouldn’t use it, then do a hypothesis test to try to reject the new treatment being that bad, in favor of it being better than that. The margin is pretty arbitrary and the inference is very indirect. With Bayes you define a similarity margin, e.g., for a binary or ordinal outcome you might set an odds ratio (OR) between 4/5 and 5/4. Then try to show that the Bayesian posterior P(4/5 \< OR \< 5/4) is high, to bring evidence for similarity. The corresponding evidence for non-inferiority would be P(OR \< 5/4).

---

<div class="post-metadata">

**Author:** ![arthur\_albuquerque](https://discourse.datamethods.org/user_avatar/discourse.datamethods.org/arthur_albuquerque/32/3269_2.png) [@arthur\_albuquerque](https://discourse.datamethods.org/u/arthur_albuquerque)\
**Post date:** [August 2, 2021, 3:45pm UTC](https://discourse.datamethods.org/t/non-inferiority-margin-without-placebo-controlled-trial/4250/12 "2021-08-02T15:45:33Z")

</div>

Totally agree. Equivalence analysis using Bayesian methods is indeed very informative. The latest REMAP-CAP preprint is a great example:

 ![image](https://discourse.datamethods.org/uploads/default/original/2X/0/0cbf464b79aba47707414fb82b948b57b6ff550f.jpeg)

Ref: [https://www.medrxiv.org/content/10.1101/2021.06.18.21259133v2.supplementary-material](https://www.medrxiv.org/content/10.1101/2021.06.18.21259133v2.supplementary-material) “Supplementary Appendix”, page 45

---

<div class="post-metadata">

**Author:** ![f2harrell](https://discourse.datamethods.org/user_avatar/discourse.datamethods.org/f2harrell/32/165_2.png) [@f2harrell](https://discourse.datamethods.org/u/f2harrell)\
**Post date:** [August 2, 2021, 3:46pm UTC](https://discourse.datamethods.org/t/non-inferiority-margin-without-placebo-controlled-trial/4250/13 "2021-08-02T15:46:42Z")

</div>

Fantastic. P(non-inferior) must be \> P(similar) as they show.

---

<div class="post-metadata">

**Author:** ![arthur\_albuquerque](https://discourse.datamethods.org/user_avatar/discourse.datamethods.org/arthur_albuquerque/32/3269_2.png) [@arthur\_albuquerque](https://discourse.datamethods.org/u/arthur_albuquerque)\
**Post date:** [August 2, 2021, 3:50pm UTC](https://discourse.datamethods.org/t/non-inferiority-margin-without-placebo-controlled-trial/4250/14 "2021-08-02T15:50:40Z")

</div>

I would like to point out that one could even perform great sensitivity analyses with different margins while using Bayesian methods. John Kruschke has a post with informative figures on this topic: [Doing Bayesian Data Analysis: How much of a Bayesian posterior distribution falls inside a region of practical equivalence (ROPE)](https://doingbayesiandataanalysis.blogspot.com/2013/08/how-much-of-bayesian-posterior.html)