Subgroup analysis and Meta-regression in Meta-analysis

[R_cubed]

Without overwhelming the OP, I’ll try to briefly sketch out the disagreement I have
with Doi’s representation of the literature on this issue.

Starting from first principles – I take Herman Chernoff’s philosophy as a worthwhile
perspective:

Blockquote
With the help of theory, I have developed insights and intuitions that prevent me from
giving weight to data dredging and other forms of statistical heresy. This feeling of freedom
and ease does not exist until I have a decision theoretic, Bayesian view of the problem
I am a Bayesian decision theorist in spite of my use of Fisherian tools.

For Bayesians, use of decision theory as a formal tool even applies to the design of experiments.
I view meta-analysis as a tool to derive the most informative experiment, given goals and
resource constraints.

Who should define reasonable priors and how?

I thought I’d bump this thread after finding a valuable discussion between Andrew Gelman and Sander Greenland on this precise issue of setting priors. Specifying a prior distribution for a clinical trial: What would Sander Greenland do? https://statmodeling.stat.columbia.edu/2008/02/05/specifying_a_pr_1/ Specifying a prior distribution for a clinical trial https://statmodeling.stat.columbia.edu/2008/01/24/specifying_a_pr/ My posts in this thread were some speculations I had about how 2 hone…

With this outlook, I find classical meta-analytic methods excessively reliant on the
metaphor of a “population” of studies, and the assumption of normality. [1][2]

Gene Glass, a pioneer in meta-analysis wrote: [1]

Blockquote
Third, the conception of our work that held that “studies” are the basic, fundamental unit of a research program may be the single most counterproductive influence of all. This idea that we design a “study,” and that a study culminates in the test of a hypothesis and that a hypothesis comes from a theory this idea has done more to retard progress in educational research than any other single notion.

Summary of Criticisms

1. In his dismissal of the logit method, he failed to distinguish between the classic 2 step proportion combination procedures, and the more recently proposed 1 step GLMMs (Generalized Linear Mixed Models) [3-5]. This is directly relevant as the OP mentioned meta-regression, with [4] providing a good example on how to proceed.

2. Limiting the discussion to classic 2 step methods, his advocacy of the Freeman-Tukey double arcsine variance stabilization transformation is inadequate. In order for the synthesis to be useful, the estimate must be converted back from the combination scale to the [0-1] interval. This is trivial for the closest competitor – the arcsine transform – which is never mentioned in his papers, but is discussed in [3-5] and recommended by the authors in [6].

The Freeman-Tukey transformation converges to the arcsine in large samples, but is not defined in a meta-analytic context with multiple proportions, as the authors mentioned in [7] (this paper was also noted above). Considering Glass’s quote above, the fact this transformation is so reliant on how sample sizes are averaged leads me to skepticism of its value in this context.

I’d agree variance stabilization can be valuable, but the double arcsine is too complicated without clear
benefit over the arcsine.

References

1. Glass, Gene Meta-analysis at 25. Self-published Jan 2000 Archived at:
   https://web.archive.org/web/20210916234827/www.gvglass.info/papers/meta25.html

2. Jackson, D, White, IR. When should meta-analysis avoid making hidden normality assumptions?
   Biometrical Journal. 2018; 60: 1040 1058. https://doi.org/10.1002/bimj.201800071

3. Lin, L, Xu, C. Arcsine-based transformations for meta-analysis of proportions: Pros, cons, and alternatives. Health Sci Rep. 2020; 9999:e178. https://doi.org/10.1002/hsr2.178

4. P. J. Shi, H. S. Sand Hu, H. J. Xiao “Logistic Regression is a better Method of Analysis Than Linear Regressionof Arcsine Square Root Transformed Proportional Diapause Data of Pieris melete (Lepidoptera: Pieridae),” Florida Entomologist, 96(3), 1183-1185, (1 September 2013)
   Logistic Regression is a better Method of Analysis Than Linear Regression of Arcsine Square Root Transformed Proportional Diapause Data of Pieris melete (Lepidoptera: Pieridae)

5. Lin L, Chu H. Meta-analysis of Proportions Using Generalized Linear Mixed Models.
   pidemiology. 2020 Sep;31(5):713-717. doi: 10.1097/EDE.0000000000001232. PMID: 32657954; PMCID: PMC7398826.

6. Kulinskaya, E., Morgenthaler S., Stadute R. Meta Analysis: A Guide to Calibrating and Combining Statistical Evidence. Wiley 2008

7. Röver C, Friede T. Double arcsine transform not appropriate for meta-analysis. Res Synth Methods. 2022 Jul 15. [2203.04773] Double arcsine transform not appropriate for meta-analysis