I was looking at some threads over at Andrew Gelman’s blog and found an important paper recommended by @Sander in the comments to a discussion on modelling:
https://statmodeling.stat.columbia.edu/2020/12/04/discussion-of-uncertainties-in-the-coronavirus-mask-study-leads-us-to-think-about-some-issues/#comment-1604678
Link to actual paper:
Stark, P (2016) Pay No Attention to the Model Behind the Curtain preprint (pdf)
It’s a tough paper regardless of your statistical philosophy, as he raises important issues about the credibility of models in practice.
His criticisms of decision models to guide policy are especially cogent:
Blockquote
Cost-benefit analyses are an example. It’s widely claimed that the only rational basis for policy
is a quantitative cost-benefit analysis. But if there’s no rational basis for its quantitative inputs,
how can a cost-benefit analysis be a rational basis for anything?
Much of the focus in other forums was on the philosophy of probability. While there are certain philosophical nits to pick regarding the interpretation of probability, but that doesn’t mean his main thesis is off target:
Blockquote
The little man behind the curtain is an apt metaphor for the role of statistical models in science
and policy. In a vast number of cases—most, perhaps—impressive computer results and
quantitative statements about probability, risk, expected costs, and other putatively solid facts
are actually controlled by a model behind the curtain, a model that nobody’s supposed to look
at carefully or pay attention to … And just like the Wizard’s machine, the models have the power to persuade and intimidate, but not the power to predict or to control.