Should one derive risk difference from the odds ratio?

[R_cubed]

@Sander

To assert empirical claims like “…and the OR is portable across baseline prevalence of an outcome…” based on math results is an example of reification of abstractions that at best are theories or hypotheses that need to be evaluated against data.

It seems like there is a general agreement that no modelling strategy dominates others in all cases. So while there may be reasonably strong prior (or expert) information that suggests the logistic model is a good default (ie. the opinion of @f2harrell ), it might not reflect all uncertainty that a skeptical audience might have (@AndersHuitfeldt).

@f2harrell

My current thinking is that one can (1) pick any model form they like and (2) use it too estimate any effect measure they like.

Wouldn’t a principled way to decide this issue be based on model averaging or model selection techniques whether Bayesian or Frequentist? How would someone specify the data analysis plan for this methodology?

I ask this after looking at the application of Bayesian Model Averaging to the issue of fixed vs random effect meta-analysis, and came across this R package: metaBMA: Bayesian (or Frequentist) Model Averaging for Fixed and Random Effects Meta-Analysis.

Related Threads:

RMS Describing, Resampling, Validating, and Simplifying the Model

Despite being published just over 20 years ago, I think this paper on Bayesian model averaging complements the discussion and recommendations in RMS. It explains the problems with prediction based on single models (and by implication step down methods which select the “best” model using the data) in the introduction. Their discussion of frequentist solutions in the last section is an apt description of RMS and much has been done from this perspective since that paper was written. It may have …

Many Analysts, One Data Set – many conclusions

Medscape psychiatry has a good write-up of a very interesting paper by Silberzahn and colleagues in which they got 29 teams, involving 61 analysts, who used the same data set to address the same research question: whether soccer referees are more likely to give red cards to dark-skin-toned players than to light-skin-toned players. Analytic approaches varied widely across the teams, and the estimated effect sizes ranged from 0.89 to 2.93 (Mdn = 1.31) in odds-ratio units. They report that the 29 …

Bayesian Biostatistical Modeling Plan

I’m a clinician, researcher, and statistics enthusiast. I try to be principled in my analyses, but since I don’t have a deep mathematical background, workflows help me to maintain good practice throughout a project. I find @f2harrell’s body of work extremely helpful because of how principled it is. Frank recently published a Biostatistical Modeling Plan for frequentist prediction models on his blog, which is a nice distillation of his advice in his RMS course. I’ve been looking for something s…

Biostatistical Modeling Plan – Statistical Thinking

Related Papers:

The following discusses the issue from a frequentist perspective, and includes other techniques (penalization). It cites some of the older Bayesian Model Averaging papers mentioned in the link to the Data Methods thread above.

Sylvain Arlot, Alain Celisse (2010). “A survey of cross-validation procedures for model selection,” Statistics Surveys, Statist. Surv. 4, 40-79

https://projecteuclid.org/journals/statistics-surveys/volume-4/issue-none/A-survey-of-cross-validation-procedures-for-model-selection/10.1214/09-SS054.full

Here is another informative paper from a frequentist perspective.

Hansen, B. E., & Racine, J. S. (2012). Jackknife model averaging. Journal of Econometrics, 167(1), 38-46.

https://www.sciencedirect.com/science/article/abs/pii/S0304407611002405

Buckland, S. T., Burnham, K. P., & Augustin, N. H. (1997). Model Selection: An Integral Part of Inference. Biometrics, 53(2), 603–618. Model Selection: An Integral Part of Inference on JSTOR