# Clinicians blaming evidence syntheses for research waste

**URL:** <https://discourse.datamethods.org/t/clinicians-blaming-evidence-syntheses-for-research-waste/6537>\
**Category:** general\
**Created:** [March 23, 2023, 5:28pm UTC](https://discourse.datamethods.org/t/clinicians-blaming-evidence-syntheses-for-research-waste/6537 "2023-03-23T17:28:52Z")\
**Posts on this page:** 1\
**Showing post:** 8

<div class="post-metadata">

**Author:** ![R\_cubed](https://discourse.datamethods.org/user_avatar/discourse.datamethods.org/r_cubed/32/1518_2.png) [@R\_cubed](https://discourse.datamethods.org/u/R_cubed)\
**Post date:** [March 26, 2023, 1:27pm UTC](https://discourse.datamethods.org/t/clinicians-blaming-evidence-syntheses-for-research-waste/6537/8 "2023-03-26T13:27:30Z")

</div>

> [@s\_doi](#):
>
> There is not a single paper to date that dismisses syntheses as you have suggested in this thread

My complaint was about the dogmatic assertion regarding “hierarchies of evidence” that is **clearly** false, not about the relative merits of meta-analysis or primary research per se.

Finding work critical of EBM depends on which literature you look at. The BMJ article made it **quite** clear that “meta-analysis” isn’t an independent method, but a perspective from which to view research reports. Glass himself desired a **replacement** of “meta-analysis” with something that is now called “fusion learning” or “confidence distributions” in the frequentist stats literature. Nelder’s quote is compatible with the opinions of the BMJ authors and Gene Glass.

Complaints about research “waste” are a direct consequence of a flawed theory of evidence. A flawed theory of evidence will lead to:

1. ignoring information that should be conditioned upon, leading to studies that should not be done, or
2. conditioning on false information, causing surprise and controversy in practice, leading to more calls for additional research.

Citing one more thread in this forum alone feels like beating a dead horse, but there are a number of papers here that either discuss decision analysis in a medical context, or study EBM criteria empirically and find it flawed.

> [@Article: Going from evidence to recommendations: Can GRADE get us there?](https://discourse.datamethods.org/t/article-going-from-evidence-to-recommendations-can-grade-get-us-there/5445):
>
> I thought the following was worth reading: Mercuri, M, Baigrie, B, Upshur, REG. Going from evidence to recommendations: Can GRADE get us there? J Eval Clin Pract. 2018; 24: 1232– 1239. [https://doi.org/10.1111/jep.12857](https://doi.org/10.1111/jep.12857) From the abstract: Blockquote In this paper, we reveal several issues with the underlying logic of GRADE that warrant further discussion. First, the definitions of the “grades of evidence” provided by GRADE, while explicit, are functionally vague. Second, the “criteria for a…

> **[De Testimonio: on the evidence for decisions about the use of therapeutic...](https://pmc.ncbi.nlm.nih.gov/articles/PMC4954394/)**
>
> Decisions about the use of therapeutic interventions, whether for individuals or entire healthcare systems, should be based on the totality of the available evidence. The notion that evidence can be reliably or usefully placed in ‘hierarchies’ is ...

From the abstract:

> The notion that evidence can be reliably or usefully placed in ‘hierarchies’ is illusory. Rather, decision makers need to exercise judgement about whether (and when) evidence gathered from experimental or observational sources is fit for purpose.

[https://www.sciencedirect.com/science/article/abs/pii/S0895435616001475](https://www.sciencedirect.com/science/article/abs/pii/S0895435616001475)

From the abstract:

> As EBM became more influential, it was also hijacked to serve agendas different from what it originally aimed for. Influential randomized trials are largely done by and for the benefit of the industry. **Meta-analyses and guidelines have become a factory, mostly also serving vested interests.**

> **[Assessing the Predictive Validity of Strength of Evidence Grades: A...](https://www.ncbi.nlm.nih.gov/books/NBK321518/)**
>
> We sought to determine the predictive validity of the U.S. Evidence-based Practice Center (EPC) approach to GRADE (Grading of Recommendations Assessment, Development and Evaluation) by examining how reliably it can predict the likelihood that...

From the abstract:

> The limited predictive validity of the EPC approach to GRADE seems to reflect a mismatch between expected and observed changes in treatment effects as bodies of evidence advance from insufficient to high SOE. In addition, many low or insufficient grades appear to be too strict.

---

_[View the full topic](https://discourse.datamethods.org/t/clinicians-blaming-evidence-syntheses-for-research-waste/6537)._