Tim Morris asked about “when a new statistical method is fit-for-use and how we get them used?”
“Fit for use” is a complex topic worth much more discussion than I’m able to provide, but to start we should consider at least these attributes:
- the analytic method should use all the information in the data it uses
- the analytical algorithm provides a reproducible numeric solution to within simulation error
- the method is shown to work either theoretically or by extensive realistic simulation
- by simulation the method is demonstrated to recover the hidden truth in the data generating mechanism for sample sizes that are typical of those available when the method will be used
- the method should have Bayesian or frequentist power or precision that is better than currently easily available methods
In terms of how to get a newer statistical method to be used in practice, here are some considerations:
- make sure that the method addresses a problem that is somewhat frequently encountered
- emphasize development of methods that have fewer hidden assumptions; prefer full model specification over weighting or estimating equations
- competently implement the analytical algorithm in a free highly available software platform such as R or python
- provide good documentation
- provide a template for the analytical workflow using
markdown
with ample comments - provide a template for the user to easily run alternative simulation studies using
markdown
with ample comments - use social media to publicize the method and make it easy to download free software
Some of the biggest holdups to getting a new method to be used even within the scientific team of which the statistician is a part are
- conservatism of the non-statisticians in the team (“but we’ve always used method x in our past publications …”)
- a belief among your co-authors that the subject matter journal will not accept a new method
Here the remedy is
- persistence of the statistician
- inclusion of nice graphics and appropriate references in the manuscript
- the statistician remembering that her job is never to give people what they want to rather to give people what they need
- never accepting as a reason “We’ve always done it this way”
Note: I have never had a situation where ultimately a journal prevailed in wanting only traditional analyses and graphics to be used in a manuscript.