# Reanalyzing MERIT design using Julia + recent numerical algorithms

**URL:** <https://discourse.datamethods.org/t/reanalyzing-merit-design-using-julia-recent-numerical-algorithms/27822>\
**Category:** languages\
**Tags:** sample-size, design\
**Created:** [April 18, 2025, 11:21am UTC](https://discourse.datamethods.org/t/reanalyzing-merit-design-using-julia-recent-numerical-algorithms/27822 "2025-04-18T11:21:44Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![davidcnorrismd](https://discourse.datamethods.org/user_avatar/discourse.datamethods.org/davidcnorrismd/32/3502_2.png) [@davidcnorrismd](https://discourse.datamethods.org/u/davidcnorrismd)\
**Post date:** [April 18, 2025, 11:21am UTC](https://discourse.datamethods.org/t/reanalyzing-merit-design-using-julia-recent-numerical-algorithms/27822/1 "2025-04-18T11:21:45Z")

</div>

Here’s an interesting case of a paper that made it through peer review at _Statistics in Medicine_ despite using Monte Carlo estimation _without regard to its inherent error_. An extensive tabulation and simulation study were presented, with absolutely no mention of MCSE’s! (As one would expect, this vitiates the reported results.)

> **[PubPeer - Design and sample size determination for multiple‐dose rando...](https://pubpeer.com/publications/4363C25F7D88E21EA31D516F997F90)**
>
> There are comments on PubPeer for publication: Design and sample size determination for multiple‐dose randomized phase II trials for dose optimization (2024)

Two interesting aspects of the above critique are:

- Relatively recent numerical work [1] played a crucial role
- Julia proved remarkably effective in rendering the reanalysis clearly

* * *

1. Frey J. An algorithm for computing rectangular multinomial probabilities. _Journal of Statistical Computation and Simulation_. 2009;79(12):1483-1489. doi:[10.1080/00949650802286753](https://doi.org/10.1080/00949650802286753)

---

<div class="post-metadata">

**Author:** ![Eslam\_Maher](https://discourse.datamethods.org/user_avatar/discourse.datamethods.org/eslam_maher/32/4899_2.png) [@Eslam\_Maher](https://discourse.datamethods.org/u/Eslam_Maher)\
**Post date:** [April 27, 2025, 1:11pm UTC](https://discourse.datamethods.org/t/reanalyzing-merit-design-using-julia-recent-numerical-algorithms/27822/2 "2025-04-27T13:11:57Z")

</div>

Thanks for sharing this feedback. I had read the paper, but these details go over my head.

However, generally speaking, there seems to be a need (following Project Optimus as well) to randomise in dose-optimization oncology studies. Yet, bec this typically requires large N, I don’t think this has happened in oncology (no? except when requested following later confusion in the development pipeline), whereas these dose-ranging studies are pretty typical outside of this domain.

There seems to be a plethora of designs, but most prominent of which are MERIT and DROID (from Ying Yuan’s team, I presume). The former, which you are discussing here, seems to be comparing each arm to predefined thresholds of toxicity & efficacy, rather than actually comparing arms to each other, resulting in an OBD admissable set (Stage 1). Do you find this as another valid point against MERIT?

---

<div class="post-metadata">

**Author:** ![davidcnorrismd](https://discourse.datamethods.org/user_avatar/discourse.datamethods.org/davidcnorrismd/32/3502_2.png) [@davidcnorrismd](https://discourse.datamethods.org/u/davidcnorrismd)\
**Post date:** [April 28, 2025, 10:07am UTC](https://discourse.datamethods.org/t/reanalyzing-merit-design-using-julia-recent-numerical-algorithms/27822/3 "2025-04-28T10:07:24Z")

</div>

This is a keen observation, Eslam! It had not occurred to me to criticize MERIT on such grounds, since one [valid?] perspective on early-phase trials is that they are to inform the grossest kind of considerations, taking existing therapies as the comparator. (“Is this new drug in a reasonable ballpark to trial further, considering what we already have available? If so, which of several doses should we study?”) But I now think you have indeed identified yet another vulnerability here.

---

<div class="post-metadata">

**Author:** ![davidcnorrismd](https://discourse.datamethods.org/user_avatar/discourse.datamethods.org/davidcnorrismd/32/3502_2.png) [@davidcnorrismd](https://discourse.datamethods.org/u/davidcnorrismd)\
**Post date:** [June 13, 2026, 11:10am UTC](https://discourse.datamethods.org/t/reanalyzing-merit-design-using-julia-recent-numerical-algorithms/27822/4 "2026-06-13T11:10:47Z")

</div>

After over a year of silence from the authors 🤔 I’ve revived the critique this morning, posting the _Statistics in Medicine_ DATA AND CODE POLICY,

> **[PubPeer - Design and sample size determination for multiple‐dose rando...](https://pubpeer.com/publications/4363C25F7D88E21EA31D516F997F90#5)**
>
> There are comments on PubPeer for publication: Design and sample size determination for multiple‐dose randomized phase II trials for dose optimization (2024)

and following up with an email to authors (cc’ing the Editorial Office) requesting the code for reproducing the Table 2 in question.

---

<div class="post-metadata">

**Author:** ![davidcnorrismd](https://discourse.datamethods.org/user_avatar/discourse.datamethods.org/davidcnorrismd/32/3502_2.png) [@davidcnorrismd](https://discourse.datamethods.org/u/davidcnorrismd)\
**Post date:** [July 8, 2026, 6:59pm UTC](https://discourse.datamethods.org/t/reanalyzing-merit-design-using-julia-recent-numerical-algorithms/27822/5 "2026-07-08T18:59:40Z")

</div>

How it’s going …

 ![image](https://discourse.datamethods.org/uploads/default/original/2X/c/cf582d9bcc0a321c967c016aecc0425f9d5265cd.jpeg)

---

<div class="post-metadata">

**Author:** ![davidcnorrismd](https://discourse.datamethods.org/user_avatar/discourse.datamethods.org/davidcnorrismd/32/3502_2.png) [@davidcnorrismd](https://discourse.datamethods.org/u/davidcnorrismd)\
**Post date:** [August 11, 2026, 5:47pm UTC](https://discourse.datamethods.org/t/reanalyzing-merit-design-using-julia-recent-numerical-algorithms/27822/6 "2026-08-11T17:47:44Z")

</div>

After several attempts, I managed to get through to an actual person in the SIM Editorial Office with the following email, sent July 7:

> My previous emails to the Editorial Office seem to have been handled by chatbots or similar automation, or else to have been dealt with by inexperienced personnel.
> 
> 1. This email is about a _published paper_, not a manuscript under review or in production. Please do not ask automatically for a “manuscript ID”.
> 
> 2. The DOI for the paper in question is 10.1002/sim.10093. This should enable you to identify the paper uniquely.
> 
> 3. I am **not** asking an Editor to read and understand the PubPeer comments I have posted on this article, nor to form a judgment about the integrity of the work. (But FYI these may be looked up by DOI at [pubpeer.com](http://pubpeer.com).)
> 
> 4. My request to the Editor is simply **to uphold your DATA AND CODE POLICY** in relation to this paper.
> 
> 5. Your policy reads [the bold emphasis is mine]:
> 
> > _Statistics in Medicine_ expects that data supporting the results reported in the paper will be archived in an appropriate public repository. Whenever possible the scripts and artefacts used to generate the analyses presented in the paper should be publicly archived. Exceptions may be granted for sensitive information such as data which allows the identification of individuals, at the discretion of the Editors. Authors will be able to complete a data accessibility statement which will be published with their paper. Submitting authors should upload their data and code as “Data Files” within the online submission system.
> > 
> > **The journal also requires authors to supply any supporting computer code or simulations that allow readers to institute any new methodology proposed in the published article.** This may be in any form(s) the authors feel is most accessible for use, including programming code, SAS commands, R functions, packages etc. Submitting authors should upload their code/simulations as “Data Files” within the online submission system.
> 
> 1. I request specifically the code to reproduce the paper’s Table 2. To complete my critique of the paper, I will need to inspect this code, and modify it to apply a control variate technique. The web app which the authors reference in the paper is not sufficient for this.
> 
> With kind regards to any human who reads this email,
> 
> David C. Norris, MD

A July 9 reply informed me,

> I have contacted the Production team and Editor regarding your query and am currently awaiting their decision. I kindly ask for your patience, and I will update you as soon as I receive a response.

On Aug 10, I requested an update:

> This matter has now been with your Editor for at least 1 month.
> 
> Could you inform me whether the paper’s authors have provided code demonstrating the reproducibility of Table 2? If so, when will this code become available to readers?

Today’s (Aug 11) reply is:

> Currently, there is no separate Code Availability Statement requirement or question within the submission system. As such, we are unable to confirm whether the authors have provided code demonstrating the reproducibility of Table 2 without reviewing the materials submitted with the manuscript.
> 
> If code has been provided, its availability to readers will depend on the authors’ Data Availability Statement and the journal’s data and code sharing policies. Where necessary, the Editors may request additional information or a code availability statement during the revision process.
> 
> We hope this clarifies the current process.

So in sum, it looks to me as if the Editorial Office are earnestly pondering whether a paper that slips through the cracks during review and production — managing to get published without adhering to the journal’s CODE POLICY — should get a ‘pass’ against future requests for code.