@f2harrell wrote:
Blockquote
Those that are prospectively planned with a significant amount of resources used to collect high-quality data without much missing data, for which it is still likely that important confounding by indication can ruin the study,
For the sake of discussion, I’d like to explore the relative merits of random experiments vs observational research from a decision theoretic viewpoint.
Dennis V. Lindley wrote the following on the relevance of randomization that deserves more attention. (The Role of Exchangeability in Inference)
Blockquote
We therefore see that randomization can play an important role even In the personalistic, Bayesian view of inference. This is contrary to the opinion resulting from the basic theorem in decision theory, that for any randomized decision procedure there exists a nonrandomized one which is not worse than it, to the effect that randomization is unnecessary in the Bayesian approach. The reason for the difference is that the use of a random mechanism is not necessary, it is merely useful. [my emphasis]
Would it be fair to say that there are certain contexts that arise in medical research that Bayesian purists like ET Jaynes have not accounted for, that make randomization more than a “merely useful” device?
I’d concede that randomization is a very useful device, but I’m not going to insist all causal claims require an RCT. The pragmatic view treats positive assertions of beneficial treatments (those require RCTs, in addition to other preliminary studies) differently than assertions about harms from previously approved treatments (ie. post-marketing surveillance is inherently observational).
For a strong defense of randomization, Senn’s Fisher’s Game with the Devil, (pdf) is worth reading.
For a Bayesian perspective on the role of randomization in a rigorous decision theoretic framework for experimental design and analysis, this paper by Dennis Lindley is essential reading.
Lindley, DV (1982) The Role of Randomization in Inference. Proceedings of the Biennial Meeting of the Philosophy of Science Association (link)