Subgroup analysis and Meta-regression in Meta-analysis

[R_cubed]

@s_doi

Rover and Friede are correct but the error is at the extremes and is so small that it lacks practical significance

The variance reduction vs the single arcsine is going to be very small in the vast majority of cases, compared to other error sources. This is a case of taking normal theory (which is only an approximation) too literally. The chance of a combined estimate that is not even in the range of the data requires the analyst to check if the implied sample size used by the Freeman–Tukey method is sensible.

On simple mini-max grounds, the FTT fails to dominate the arcsine as there exist cases where the latter is better. If errors are weighted, a small chance of a large, embarrassing error with the FTT doesn’t seem worth the effort to.use it, considering the notion of a single “sample size” for multiple proportions has no theoretical justification.

Rover and Friede produce an example where the FTT reverses the order of two data points on the combination scale. This reminds me of the problem I noted on the use of parametric models on ordinal data:

Blockquote
The early critics of parametric models on ordinal data noted that arbitrary scale transformations could change the observed sign of the effect…
The implication is that no information is communicated by parametric models on ordinal data.

I don’t think this method is as bad as that, but I see no reason to use something that has a small chance of changing the ordering of the data points, which destroys information.

Preprint: Analysis of Likert Type Data Using Metric Methods

Dennis Boos and Judy Chen (2022). Analysis of Likert-Type Data Using Metric Methods. I edited this to bring the main points earlier, and the decision theoretic justification later in the post. Introduction The authors defend the common practice of using parametric models on ordinal data on the basis of pragmatics (ie.“easier” to for researchers understand), and perceived robustness in that parametric methods on empirical or simulated data sets are not sufficiently inaccurate that skepti…

[s_doi]

@s_doi

I have a philosophical position on meta-analysis of prevalence

@Junaid I had said I will address the philosophical question last. Basically meta-analysis is a weighted average and there are many types of weights one could use for weighted averaging. Only error (variance) weights make a weighted average a meta-analysis. However, in this situation there must be an underpinning unknown common population parameter otherwise variance weights are inappropriate. In most burden of disease meta-analyses this underpinning unknown common population parameter is absent and thus meta-analysis is not appropriate for such studies. See An Updated Method for Risk Adjustment in Outcomes Research

[Junaid]

Dear @s_doi and @R_cubed, I hope you both are well. I apologise, as I travelled abroad and couldn’t respond to your comments. I am working on subgroup analysis and metaregression and will get back to you for your feedback. Thank you for bearing with me.

[Junaid]

@s_doi

regress FTT x1 x2 [aw=1/v], vce(robust)

@s_doi
Dear Prof. Doi,

I used the “regress FTT x1 x2 [aw=1/v], vce(robust)” command and the Stata said, “option ftt not allowed.” Any suggestions, please?

[s_doi]

The command

regress year samplesize, ftt vce(robust)

is not correct and should be

regress _ES year samplesize [aw=1/(_seES^2)], vce(robust)

Both _ES and _seES are created after you run metan and are on the FTT scale if you selected the FTT transform in metan