Nonparametric Effect Size estimation, Likelihood Methods, and Meta Analysis

I’m adding at least 2 new links as they are related to this problem.

Here is an interesting question where there isn’t an easily accessible answer in the standard meta-analysis texts:

https://www.researchgate.net/post/How_to_run_a_meta-analysis_of_medians_and_IQR

After reading Senn’s article (posted above) on using overlap measures as effect sizes, in a meta-analysis of medians, I’d calculate the Mann, Whitney, Wilcoxon log odds as described in the Missing Medians paper (link above) if the U statistics were available. If not, I’d have to calculate the log odds assuming a shift in the logistic distribution.

Here is a pre-print that recommends the use of weighted median effect sizes at the meta-analytic level to reduce the need to model publication bias. I had thought this was a plausible idea, but I think the HL estimate would be even better.

Here is an interesting critique of “quality scores” for individual studies in meta-analysis.
https://academic.oup.com/biostatistics/article/2/4/463/321492

The following is a great paper in my field of rehabilitation, that describes the problems with making naive resource allocation decisions or drawing conclusions about effectiveness from improper analyses of ordinal data. Everything the author warned of had come true. I don’t know why I had not found it sooner.

One of the many key quotes:

Blockquote
The regular use of rating scales by many institutions tends to give them credence. Administratively and editorially sanctioned mathematical mistreatment of these data gives the appearance that such treatment has been found to be scientifically valid. Despite the most noble of intentions, the misuse of rating scale data encourages retention of misinformation. Even if we are astute enough to recognize methodologic errors while reading an article, we will often remember the author’s conclusions. In clinical practice, we tend to remember the numbers associated with a specific patient, even though we know them to be of questionable validity. This cannot help but shape our thinking and practice.

https://www.researchgate.net/publication/20619750_Ordinal_scale_and_foundations_of_misinference

1 Like