# The Ethics of Underpowered Trials

**URL:** <https://discourse.datamethods.org/t/the-ethics-of-underpowered-trials/28806>\
**Category:** general\
**Tags:** sample-size, design\
**Created:** [June 25, 2026, 1:58pm UTC](https://discourse.datamethods.org/t/the-ethics-of-underpowered-trials/28806 "2026-06-25T13:58:29Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![trumanfrancis](https://discourse.datamethods.org/user_avatar/discourse.datamethods.org/trumanfrancis/32/47_2.png) [@trumanfrancis](https://discourse.datamethods.org/u/trumanfrancis)\
**Post date:** [June 25, 2026, 1:58pm UTC](https://discourse.datamethods.org/t/the-ethics-of-underpowered-trials/28806/1 "2026-06-25T13:58:29Z")

</div>

This editorial appeared in a veterinary journal. It espouses producing experiments even when the number of subjects (dogs, cats, horses …) is low. The reasoning is that these trials could later be synthesized to produce a useful result. From an ethical viewpoint isnt it unethical to recruit subjects for a trial when there is a high probability that the result could be uninterpretable? Im thinking things like the winners curse and the absence of evidence fallacy. Often these trial will be considered hypothesis generating or pilot studies but taken by clinicians as confirmatory. [Underpowered Trials](https://onlinelibrary.wiley.com/share/GRSY6MRDGDN9NNIYRVFI?target=10.1111/jsap.70063)

---

<div class="post-metadata">

**Author:** ![ESMD](https://discourse.datamethods.org/letter_avatar_proxy/v4/letter/e/9e8a1a/32.png) [@ESMD](https://discourse.datamethods.org/u/ESMD)\
**Post date:** [June 25, 2026, 6:30pm UTC](https://discourse.datamethods.org/t/the-ethics-of-underpowered-trials/28806/2 "2026-06-25T18:30:26Z")

</div>

Interesting article- thanks for sharing it. These same issues arise in human medicine all the time.

Arguably, researchers who defend conduct of small RCTs by saying that their goal is to contribute to a future meta-analysis should provide reasons to believe that such a meta-analysis _will ever be conducted_. A nebulous statement that researchers in other parts of the world might share an interest in the topic wouldn’t suffice. Rather, what would be needed, to make conduct of a small RCT more defensible and credible, is for researchers to _already have made an effort at collaboration_, _prior_ to proposing their RCT. Ideally, anyone planing to conduct a small RCT on that topic would design their study in such a way that it would be “combinable” with the results of previous and future RCTs on the same topic.

Since _pre-planning_ of a meta-analysis of RCTs can significantly increase its credibility (as pre-planning helps to ensure that apples will be meta-analyzed with apples and not oranges), and, since pre-planning requires _collaboration_ with other research groups, why not simply collaborate _from day 1_ and design a _larger_, perhaps _multi_-centre RCT?

To be clear, I have never designed an RCT and this proposition might seem terribly naive to those who do design trials. But, arguably, inadequate foresight (i.e., no clear advance plan as to how a study’s results might realistically be used) has, historically, been the source of a lot of research waste. RCTs in both human and animal medicine should _serve a purpose_ (other than burnishing a CV or dissertation), _regardless_ of their results. If their results simply generate a shrug, something has gone very wrong.

---

<div class="post-metadata">

**Author:** ![f2harrell](https://discourse.datamethods.org/user_avatar/discourse.datamethods.org/f2harrell/32/165_2.png) [@f2harrell](https://discourse.datamethods.org/u/f2harrell)\
**Post date:** [June 25, 2026, 7:06pm UTC](https://discourse.datamethods.org/t/the-ethics-of-underpowered-trials/28806/3 "2026-06-25T19:06:18Z")

</div>

@ESMD’s observations are spot on.

On a purely technical note, in my past I’ve worked with many animal researchers, and their statistical practices make this situation much worse:

- They want to conclude that there is evidence for the absence of an effect when p \> 0.05
- They want to use p-values at all with very small sample sizes
- They avoid uncertainty intervals for key quantities of interest; the width of uncertainty intervals is a good way to show the yield of the experiment no matter what the sample size

---

<div class="post-metadata">

**Author:** ![trumanfrancis](https://discourse.datamethods.org/user_avatar/discourse.datamethods.org/trumanfrancis/32/47_2.png) [@trumanfrancis](https://discourse.datamethods.org/u/trumanfrancis)\
**Post date:** [June 25, 2026, 7:57pm UTC](https://discourse.datamethods.org/t/the-ethics-of-underpowered-trials/28806/4 "2026-06-25T19:57:54Z")

</div>

Yes I review for veterinary journals, and all of these points are accurate. They also frequently run multiple null hypothesis tests and report the “significant” results as the headline finding. I actually think these studies might benefit from a Bayesian approach although I am not an expert in that domain.

---

<div class="post-metadata">

**Author:** ![f2harrell](https://discourse.datamethods.org/user_avatar/discourse.datamethods.org/f2harrell/32/165_2.png) [@f2harrell](https://discourse.datamethods.org/u/f2harrell)\
**Post date:** [June 26, 2026, 11:07am UTC](https://discourse.datamethods.org/t/the-ethics-of-underpowered-trials/28806/5 "2026-06-26T11:07:01Z")

</div>

A Bayesian approach would not use approximations that fail in small samples, would allow experimenters to be skeptical about large effects, and would improve ethics by allowing for efficient sequential learning that avoids fixed and arbitrary sample sizes.

---

<div class="post-metadata">

**Author:** ![davidcnorrismd](https://discourse.datamethods.org/user_avatar/discourse.datamethods.org/davidcnorrismd/32/3502_2.png) [@davidcnorrismd](https://discourse.datamethods.org/u/davidcnorrismd)\
**Post date:** [June 26, 2026, 12:02pm UTC](https://discourse.datamethods.org/t/the-ethics-of-underpowered-trials/28806/6 "2026-06-26T12:02:03Z")

</div>

A reminder: the whole [post-hoc power fiasco](https://pubpeer.com/publications/4399282A80691D9421B497E8316CF6) arose out of attempts to justify small surgical trials.

---

<div class="post-metadata">

**Author:** ![trumanfrancis](https://discourse.datamethods.org/user_avatar/discourse.datamethods.org/trumanfrancis/32/47_2.png) [@trumanfrancis](https://discourse.datamethods.org/u/trumanfrancis)\
**Post date:** [June 26, 2026, 12:28pm UTC](https://discourse.datamethods.org/t/the-ethics-of-underpowered-trials/28806/7 "2026-06-26T12:28:13Z")

</div>

I was told when I was at Oxford that whenever there was a null result for a surgical procedure the American surgeons would tell the Europeans it was a difference in skill not power.  
Similarly in my field with stem cell research. When it doesnt work its because the investigator is not preparing the stem cell product correctly.

---

<div class="post-metadata">

**Author:** ![MSchwartz](https://discourse.datamethods.org/user_avatar/discourse.datamethods.org/mschwartz/32/2579_2.png) [@MSchwartz](https://discourse.datamethods.org/u/MSchwartz)\
**Post date:** [June 26, 2026, 3:36pm UTC](https://discourse.datamethods.org/t/the-ethics-of-underpowered-trials/28806/8 "2026-06-26T15:36:14Z")

</div>

There was a lengthy thread last year on the underpowered RCT topic, albeit, it also segued into some discussions about interim monitoring and related topics:

> [@Decades of underpowered RCT](https://discourse.datamethods.org/t/decades-of-underpowered-rct/28236):
>
> One of the most frustrating problems with medical science is the construction of the incomplete bridge. No other industry could build bridges which are incomplete and then simply say “one problem with our bridge is that we did not have enough steel to complete it”. This is ubiquitous and always ends with a call for another trial leaving the clinician with no answer. The important question which could have been answered by this RCT remains unanswered. Why can’t this be fixed? [https://jamanetw…](https://jamanetwork.com/journals/jamanetworkopen/fullarticle/2835888)

In the course of the discussion, in one of my replies, I referenced the following 2013 paper by Button et al:

[Power failure: why small sample size undermines the reliability of neuroscience](https://www.nature.com/articles/nrn3475)

On the skill related issues that @trumanfrancis raises above, that is a key reason why in many studies, especially for implantable devices in IDE studies, you will see that for each of the one or more study sites/PIs, there will be an initial series of “run-in” subjects of some modest number, to get over the learning curve issue that may be present and bias outcomes for early subjects.

---

<div class="post-metadata">

**Author:** ![trumanfrancis](https://discourse.datamethods.org/user_avatar/discourse.datamethods.org/trumanfrancis/32/47_2.png) [@trumanfrancis](https://discourse.datamethods.org/u/trumanfrancis)\
**Post date:** [June 26, 2026, 3:51pm UTC](https://discourse.datamethods.org/t/the-ethics-of-underpowered-trials/28806/9 "2026-06-26T15:51:44Z")

</div>

I was wondering if it would be useful to include in a supplement a dashboard sequencing the evolution of evidence on a topic with estimates and uncertainty from studies across the timeline as well as perhaps indicators of potential bias. Then allow the reader to use the current study to see the impact of different priors on the posterior distribution.

---

<div class="post-metadata">

**Author:** ![simongates](https://discourse.datamethods.org/user_avatar/discourse.datamethods.org/simongates/32/367_2.png) [@simongates](https://discourse.datamethods.org/u/simongates)\
**Post date:** [June 27, 2026, 7:57am UTC](https://discourse.datamethods.org/t/the-ethics-of-underpowered-trials/28806/10 "2026-06-27T07:57:28Z")

</div>

When I started working in clinical trials there was quite a lot of the “underpowered = unethical” view. And yes, people shouldn’t do small trials that are bound to be inconclusive when they could and should do something better. But it’s not black and white. Very often we know it’s going to be hard to recruit the number that we would ideally like - as others have said, Bayesian methods are really useful here.

What should we do in rare diseases where we know we can only recruit a small number? Often not randomising and using some sort of single arm trial design is advocated. I’m not sure about this, as it seems to me that it’s often proposing a solution for lack of precision that sacrifices accuracy - non-randomised trials are likely to much more prone to biases, which are potentially unknown and large. Especially if they don’t do the complicated statistical work of trying to understand and correct the biases (and lots of studies don’t).

---

<div class="post-metadata">

**Author:** ![f2harrell](https://discourse.datamethods.org/user_avatar/discourse.datamethods.org/f2harrell/32/165_2.png) [@f2harrell](https://discourse.datamethods.org/u/f2harrell)\
**Post date:** [June 27, 2026, 11:49am UTC](https://discourse.datamethods.org/t/the-ethics-of-underpowered-trials/28806/11 "2026-06-27T11:49:33Z")

</div>

That’s a great question. My current opinion is that failing to use 1:1 randomization is resulting in the need for **increasing** the total sample size in the long run, once we’re honest about (1) inefficiencies of unbalanced randomization and (2) realize that standard errors are not what is important (in the frequentist world) but rather variance + bias ^2 = mean squared error of effectiveness estimates. We bias term will dominate in many situations, coming from the use of non-concurrent or non-randomized data.

Backstory: I’m seeing sponsors want to use historical controls or to do 2:1 randomization in rare diseases, the latter as an enticement to desperate participants who believe (usually falsely) that the new treatment will improve their condition in a big way; they want higher odds of getting the treatment.

There are better ways to accomplish the goals:

- Use 1:1 (which usually lowers the total sample size) randomization and have a more discriminating, more continuous, more frequently measured outcome variable
- Follow patients for an adequate length of time
- After that time convert the study into an open-label extension where everyone gets the new treatment, if the treatment is demonstrated in the main study phase to have _some_ benefit
- Use a Bayesian sequential design so that we can tell patients that if they were assigned to the less effective treatment they will remain on it a minimum of time until there is evidence for it being ineffective.

---

<div class="post-metadata">

**Author:** ![davidcnorrismd](https://discourse.datamethods.org/user_avatar/discourse.datamethods.org/davidcnorrismd/32/3502_2.png) [@davidcnorrismd](https://discourse.datamethods.org/u/davidcnorrismd)\
**Post date:** [July 2, 2026, 6:50am UTC](https://discourse.datamethods.org/t/the-ethics-of-underpowered-trials/28806/12 "2026-07-02T06:50:16Z")

</div>

The question of _marginality_ explored formally in my working paper [On Marginal Therapy in Degenerative Disease](https://osf.io/59sya/files/q6trp) (newly updated July 1) goes to the heart of your problem, Simon. If there is genuine reason to suppose that your candidate therapy could produce clinically detectable responses in individual patients, then randomization would be scientifically and therapeutically counterproductive. But if your 1-arm trial should fail to demonstrate any clinically evident response, the therapy looks increasingly _marginal_ as n grows — a fact which must be conveyed as part of true informed consent. Provided that no more attractive candidate therapy has emerged in the meantime, your trial might reasonably add a placebo arm and randomize. Patients will be disappointed to learn during informed consent that you now believe the therapy is unlikely to produce any definite effect they will notice, but this may at least allay concerns about assignment to placebo.

Certain formal elements in the paper correlate nicely with Frank’s advice above:

- The question of “more discriminating … outcome variable” is addressed formally in the \theta\_c parameter of the analysis
- Follow-up duration is addressed in footnote 3
- The entire analysis is Bayesian-sequential in character.

Related: [Where are the exceptional responders?](https://discourse.datamethods.org/t/where-are-the-exceptional-responders/17961)