I think this video by Michael I Jordan is extremely helpful for understanding how Bayes and Frequentist methods complement each other, with the key tool being decision theory. (slides for video)
Blockquote
Decision theory is an extremely useful perspective in thinking about fundamentals of statistical inference. (10:20-10:44)
This video with Jim Berger, Sander Greenland, and Robert Matthews provided me with the information needed to correct a lot of misconceptions I had about the relation between Frequentist and Bayesian methods. Matthew’s resurrection of Reverse Bayesian methods creates a great opportunity to combine the insights of Bayesians with those of the Frequentists.
Finally, more recent work on so-called “fiducial inference” via the notion of “Confidence” distributions (a generalization of confidence intervals) can show relationships between Bayesian posteriors, bootstrap distributions, and “Fiducial” distributions. A whole collection of papers on this unifying perspective (known as BFF: Bayes, Frequentist, Fiducial → Best Friends Forever) can be found here (most require “mathematical maturity”):
https://stat.rutgers.edu/home/mxie/SelectedPapers.htm
A good intro to confidence distributions can be found in this paper by @COOLSerdash as well as searching past threads.
https://onlinelibrary.wiley.com/doi/abs/10.1002/sim.8293
Related thread: