One use of Bayesian and closely related methods such as penalization is for reduction of small-sample instabilities and artefacts via the introduction of “shrinkage” priors, as reviewed for example in
Greenland S, Mansournia MA, Altman DG (2016). Sparse-data bias: A problem hiding in plain sight. BMJ 353:i1981, 1-6,
https://www.bmj.com/content/352/bmj.i1981.
Bayesian and related methods are central to probabilistic bias analysis (PBA), especially but not only for observational studies. PBA extends the model for the data-generating mechanism beyond the usual forms (e.g., logistic, log-linear, proportional-hazards, structural) to include models for uncontrolled bias sources such as nonrandom selection, confounding, misclassification and measurement error. The parameters in these model extensions are given prior distributions in order to identify the effect of interest, which can then be estimated using the full expanded model and priors via simulation techniques (which may be of Bayesian form).
There is now a large literature on PBA. Here is a book and some chapters that provide coverage:
Fox MP, MacLehose RF, Lash TL (2021) Applying quantitative bias analysis to epidemiologic data, 2nd edn. Springer, New York
Greenland S, Lash TL (2008) Bias analysis. Ch. 19 in Rothman KJ, Greenland S, Lash TL (eds) Modern epidemiology, 3rd edn. Lippincott-Williams-Wilkins, Philadelphia, 345–380
Greenland S (2014). Sensitivity analysis and bias analysis. Ch. 19 in Ahrens W, Pigeot I, eds. Handbook of Epidemiology, 2nd edn. Springer, New York, 685-706
Here are some introductory overviews on PBA which you may find in JSTOR or I can supply on request:
Greenland S (2005) Multiple-bias modeling for observational studies (with discussion). J R Stat Soc Ser A 168:267–308
Greenland S (2009) Bayesian perspectives for epidemiologic research. III. Bias analysis via missing-data methods. Int J Epidemiol 38:1662–1673, corrigendum (2010) Int J Epidemiol 39:1116
Some guidelines and cautionary discussions for bias analyses:
Lash TL, Fox MP, MacLehose RF, Maldonado G, McCandless LC, Greenland S (2014) Good practices for quantitative bias analysis. Int J Epidemiol 43:1969–1985
Lash TL, Ahern TP, Collin LJ, Collin LJ, Fox MP, MacLehose RF (2021) Bias analysis gone bad. Am J Epidemiol 190:1604–1612
Greenland S (2021) Dealing with the inevitable deficiencies of bias analysis – and all analyses. Am J Epidemiol 190:1617–1621
MacLehose RF, Ahern TP, Lash TL, Poole C, Greenland S (2021) The importance of making assumptions in bias analysis. Epidemiology 32:617–624