I came across this important article by Philip Stark I had previously read, on the use of statistical models in practice. He echos the concerns expressed by @ChristopherTong
Blockquote
But not all uncertainties can be represented as probabilities.
I think this assertion requires a bit more elaboration. It seems to smuggle in the idea that one must have some physical model before invoking “probability” as a tool. Jaynes would call this the “mind projection fallacy” and takes substantial effort to ground probability as an expression of prior knowledge (not mere subjective belief).
I still highly recommend reading (and re-reading) this from time to time. I think his observation about mathematical models being used to “persuade and intimidate” rather than predict is close enough to the mark to give him credit for a bullseye hit on the target.