# Standardized mean difference on very small dataset

**URL:** <https://discourse.datamethods.org/t/standardized-mean-difference-on-very-small-dataset/3638>\
**Category:** meta-analysis\
**Created:** [August 8, 2020, 1:57pm UTC](https://discourse.datamethods.org/t/standardized-mean-difference-on-very-small-dataset/3638 "2020-08-08T13:57:33Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Victoria](https://discourse.datamethods.org/user_avatar/discourse.datamethods.org/victoria/32/1735_2.png) [@Victoria](https://discourse.datamethods.org/u/Victoria)\
**Post date:** [August 8, 2020, 1:57pm UTC](https://discourse.datamethods.org/t/standardized-mean-difference-on-very-small-dataset/3638/1 "2020-08-08T13:57:33Z")

</div>

Hello everyone,

I am currently working on a meta-analysis for my internship.

I want to know if the difference observed between the bioaccessibility of As in raw product (control) vs in cooked product (treatment) is consistent across articles. My first idea was to calculate a Cohen’s D for each cooking treatment in my article (raw vs grilled, raw vs boiled … ) and then to compute a forest plot for each of these cooking treatments.

The problem is that I’m limited by the structure and the scarcity of my dataset. In fact, my bioaccessibility mean was obtained from a triplicate of measure only.

So, to resume my calculation of the Cohen’s D results from the difference between mean obtain from a triplicate.  
In a statistical point of view, I’m not sure of the reliability of my calculation. What do you think about that ?

Thank you for your answers.

Victoria

---

<div class="post-metadata">

**Author:** ![R\_cubed](https://discourse.datamethods.org/user_avatar/discourse.datamethods.org/r_cubed/32/1518_2.png) [@R\_cubed](https://discourse.datamethods.org/u/R_cubed)\
**Post date:** [August 10, 2020, 7:15pm UTC](https://discourse.datamethods.org/t/standardized-mean-difference-on-very-small-dataset/3638/2 "2020-08-10T19:15:37Z")

</div>

Think carefully about using the standardized mean difference, as it confounds the objective effects of the intervention with the design of the study. Heterogeneity is always a critical factor in the conclusions you can draw from retrospective meta-analyses.

If your measurement has a natural unit, it is generally agreed that this is a valid way to aggregate different studies.

You would not know this from the easily accessible literature on how to do a meta-analysis. I’ve collected a lot of papers in a separate thread, but the most important ones are described here:

> [@Rationale for the high level of evidence for meta-analyses](https://discourse.datamethods.org/t/rationale-for-the-high-level-of-evidence-for-meta-analyses/3223/10):
>
> I’ve collected a large number of papers on meta-analysis in this thread. But I especially found the following articles (2 by Stephen Senn) helpful understanding the issues. That will place the following paper in a better context: The main things I’ve gotten from them: Retrospective meta-analysis is a very limited technique if we desire to come up with an aggregate effect. It almost certainly does not deserve to be at the top of any “evidence pyramid.” If the outcome does not have…