Use of machine learning experts and statisticians as referees

[R_cubed]

@kiwiskiNZ

Q1: Does anyone know of any literature that looks at refereeing of ML in the medical literature?

Failing that:
Q2: Any literature on the use of statisticians as referees?

Q3: Also, what is your favourite paper reporting on common statistical errors in the medical literature?

RE: Q1. Frank is listed on these 2 recent papers that compare “statistical” vs “machine learning” in a health context. I don’t know of anything specific towards refereeing.

1. Austin, P. C., Harrell, F. E., Lee, D. S., & Steyerberg, E. W. (2022). Empirical analyses and simulations showed that different machine and statistical learning methods had differing performance for predicting blood pressure. Scientific Reports, 12(1), 1-11. link
2. Austin, P. C., Harrell Jr, F. E., & Steyerberg, E. W. (2021). Predictive performance of machine and statistical learning methods: Impact of data-generating processes on external validity in the “large N, small p” setting. Statistical methods in medical research, 30(6), 1465-1483. link

I’m reminded of this thread that was critical of using the term ML to make a paper seem more sophisticated.

Rebranding of "Statistics" as "Machine Learning" to Sound More Impactful & Negative Fallout

This article recently came to my attention: It is my opinion that calling this a “Machine Learning” approach is yet another example of stretching the term in a naked effort to make a paper seem more sophisticated. The “machine learning” approach here, chi-squared automated interaction detection, has been around for almost 40 years and seems nearly identical to classification and regression tree analysis / recursive partitioning (I realize that some of these terms are used interchangeably dep…

RE Q3. This data methods thread has lots of relevant citations:

Reference Collection to push back against "Common Statistical Myths"

Note: This topic is a wiki, meaning that this main body of the topic can be edited by others. Use the Reply button only to post questions or comments about material contained in the body, or to suggest new statistical myths you’d like to see someone write about. I am not claiming to be the leading authority on any or all of the things listed below, but several of us on Twitter have repeatedly floated the idea of creating a list of references that may be used to argue against some common stati…