Nonparametric Effect Size estimation, Likelihood Methods, and Meta Analysis

[R_cubed]

The aggregation of effect sizes on anything other than interval and ratio scaled variables is much more complicated than I initially anticipated. You would not know this from most of the readily available texts on meta-analysis, as far as I can tell.

This discussion hinted at the problems:

Does data-based variable reduction lead to confounding?

Sander Greenland and colleagues do a good job explaining how the standardization of predictors leads to confounded estimates of both correlational and causal effects: because standardization depends on the eccentricities of the training data. I wonder if, for similar reasons, data reduction methods (e.g., principal components regression) have the same issue and why.

Relevant article by Stephen Senn on Effect size measurements, particularly non-parametric ones.

Stephen Senn (2011) U is for Unease: Reasons for Mistrusting Overlap Measures for Reporting Clinical Trials

Stehpen Senn (2000) The Many Modes of Meta

Some issues related to standardized effect:
Peter Cummings MD, MPH (2004) Meta-analysis Based on Standardized Effects Is Unreliable

Peter Cummings MD, MPH (2011) Arguments for and Against Standardized Mean Differences (Effect Sizes)

Milo A Puhan, Irene Soesilo, Gordon H Guyatt & Holger J Schünemann (2006) Combining scores from different patient reported outcome measures in meta-analyses: when is it justified?