# Critique of paper on generalizability of oncology trials

**URL:** <https://discourse.datamethods.org/t/critique-of-paper-on-generalizability-of-oncology-trials/28523>\
**Category:** journal club\
**Tags:** oncology, simulation, ml, generalizability, rct\
**Created:** [November 9, 2025, 2:32pm UTC](https://discourse.datamethods.org/t/critique-of-paper-on-generalizability-of-oncology-trials/28523 "2025-11-09T14:32:10Z")\
**Posts on this page:** 1\
**Showing post:** 9

<div class="post-metadata">

**Author:** ![R\_cubed](https://discourse.datamethods.org/user_avatar/discourse.datamethods.org/r_cubed/32/1518_2.png) [@R\_cubed](https://discourse.datamethods.org/u/R_cubed)\
**Post date:** [November 15, 2025, 12:28pm UTC](https://discourse.datamethods.org/t/critique-of-paper-on-generalizability-of-oncology-trials/28523/9 "2025-11-15T12:28:09Z")

</div>

I agree with most of your post, except for the following:

> [@Alessandro\_Rovetta](#):
>
> I also think all this should instead encourage adoption of frameworks such as [GRADE](https://doi.org/10.7326/annals-25-03828),

I’ve discussed GRADE in a number of threads, and it falls far short of what reasonable people would do in a community of [cooperative skeptics](https://datascience.uchicago.edu/insights/whats-game-theoretic-about-statistics-a-game-theoretic-notion-of-evidence-and-cooperative-skepticism/) interested in truth.

> [@Article: Going from evidence to recommendations: Can GRADE get us there?](https://discourse.datamethods.org/t/article-going-from-evidence-to-recommendations-can-grade-get-us-there/5445):
>
> I thought the following was worth reading: Mercuri, M, Baigrie, B, Upshur, REG. Going from evidence to recommendations: Can GRADE get us there? J Eval Clin Pract. 2018; 24: 1232– 1239. [https://doi.org/10.1111/jep.12857](https://doi.org/10.1111/jep.12857) From the abstract: Blockquote In this paper, we reveal several issues with the underlying logic of GRADE that warrant further discussion. First, the definitions of the “grades of evidence” provided by GRADE, while explicit, are functionally vague. Second, the “criteria for a…

Empirical assessment (linked to in that thread) indicate that applying GRADE overvalues RCTs (granting them more credibility than they contain) and undervalues observational evidence, when the outcome measure is predicting what a future study would show.

The right way was discussed in @Sander in his paper on modelling bias in observational studies, which is inherently Bayesian in perspective.

Greenland, S. (2005). Multiple-bias modelling for analysis of observational data. Journal of the Royal Statistical Society Series A: Statistics in Society, 168(2), 267-306 [https://academic.oup.com/jrsssa/article/168/2/267/7084313](https://academic.oup.com/jrsssa/article/168/2/267/7084313)

> Conventional analytic results do not reflect any source of uncertainty other than random error, and as a result readers **must rely on informal judgments regarding the effect of possible biases**. (my emphasis)

I pointed out in another thread that this “informal judgement” was actually formalized in a field of AI and computer science, known as [Subjective Logic](https://www.mn.uio.no/ifi/english/people/aca/josang/sl/), which is an application of Bayesian hierarchical models.

> [@Using Bayesian statistics in observational studies](https://discourse.datamethods.org/t/using-bayesian-statistics-in-observational-studies/10842/7):
>
> Since we are discussing papers by @Sander, I’d like to add a commentary he wrote for Statistical Science that discusses the application of both “Frequentist” and “Bayesian” concepts in a reasonable way. A number of sections touch directly upon the question asked in the original post. Greenland, S. (2010). Comment: The need for syncretism in applied statistics. Statistical Science, 25(2), 158-161. [link](https://www.jstor.org/stable/41058935) He discusses how hierarchical models provide a framework for integrating the considerations …

---

_[View the full topic](https://discourse.datamethods.org/t/critique-of-paper-on-generalizability-of-oncology-trials/28523)._