Bayesian Biostatistical Modeling Plan

[R_cubed]

I think a willingness to grapple with mathematical tools will pay large dividends as it will help in reading the foundational papers in Bayesian Decision Theory as well as information theory.

Quoting from Bayesian Analysis (Bernardo and Smith, 2004, p.67)

Blockquote
We have shown that the simple decision problem structure introduced … suffice for the analysis of more complex, sequential problems which appear, at first sight, to go beyond that simple structure. In particular, we have seen the important problem of experimental design can be analysed in the sequential decision problem framework. [my emphasis] We shall now use this framework to analyse the very special problem of statistical inference, [italics in original] thus establishing the fundamental relevance of these foundational arguments to statistical theory and practice.

I don’t think frequentist methods can truly be understood without understanding Bayesian Decision Theory. I deeply appreciate this perspective by Herman Chernoff in his comment on Bradley Efron’s 1985 paper “Why isn’t everyone Bayesian?”

Blockquote
With the help of theory, I have developed insights and intuitions that prevent me from giving weight to data dredging and other forms of statistical heresy. This feeling of freedom and ease does not exist until I have a decision theoretic, Bayesian view of the problem … I am a Bayesian decision theorist in spite of my use of Fisherian tools.

When you conceptualize your experiment or study with the goal of maximizing information (or designing a communication channel with the highest signal/noise ratio), things become clearer. Much of the advice in RMS can be understood from this point of view.

RE: Clinical utility of prediction models: search data methods for “decision curves” for the most rigorous evaluation of predictive models.

Validity of decision curve analysis in uncalibrated models with good discrimination

I’m doing an external validation of a prognostic model. The model shows good discrimination (AUC 0.82), but uncalibrated (O/E 1.58 (95%CI 1.3 to 1.8), Calibration Intercept 0.2 (95%CI -0.1 to 0.5 ), Calibration Slope 0.8 (95%CI 0.1 to 0.96). My doubt is if it is valid to run a decision curve analysis to assess clinical utility in this uncalibrated model or should I do it only on the recalibrated model? I would appreciate any insights into this or, if possible, literature that touches on the sub…

RE: Missing Data and Imputation. Stef van Buuren has an online text on this topic. I think I found it in Frank’s notes or bibliographies somewhere.

Flexible Imputation of Missing Data (2018). Chapman Hall/CRC Press https://stefvanbuuren.name/fimd/

See also:

A Bayesian Perspective on Missing Data Imputation

When thinking about workflows, Jeroen Janssens has published a freely available text on how to use traditional Unix/Linux command line tools as well as R. There is also a discussion of CLI machine learning tools.

Welcome | Data Science at the Command Line, 2e

The framework discussed has the acronym OSEMN (pronounced “awesome”):

1. Obtain: (Study design and prospective data collection goes here).
2. Scrub: (much of the work of “data wrangling” - ie. getting various data sources in a usable format)
3. Explore: (looking at distributions, missing data, etc. Imputation could be done here).
4. Model: (Computing likelihoods, posteriors, robustness checks, decision curves etc.)
5. iNterpret: Draw conclusions and recommendations for practice and future research.

Building upon Shannon/Weaver theory of communication, I’d place any SAP (statistical analysis plan) on the encoding and decoding ends of the channel. At that point, we cannot increase any received information, but it is easy to lose it.

Shannon and Weaver Model of Communication

Related Threads

Definition of statistics

Blockquote Experimental design and understanding properties of measures are key elements of statistics. I would agree, but I hasten to point out that design of experiments is fundamentally an application of the Bayesian decision theoretic machinery (in principle at least). I don’t comprehend this resistance to a broad conception of decision theory. Most objections to it hold up the simplistic Neyman-Pearson behavioristic approach as if that is the only conception around. In economics, th…

Information, Evidence, and Statistics for critical research appraisal

In another post about understanding p-values, ESMD made an important observation: ESMD wrote: I suspect hat much of the confusion among physicians learning critical appraisal stems from the fact that our teachers are often (?usually) practising MDs, who might not understand the historical foundations of statistics well enough to address questions like this. As a result, these types of questions get glossed over. In that thread, @Sander_Greenland mentioned Richard Royall – the biostatistic…

Who should define reasonable priors and how?

It is hard for me to imagine doing any experiment without considering the value of the information. Methodology scholars have long complained that experiments have been “under powered”, and that much clinical research is not useful [1]. IJ Good considered information as something he called a “quasi-utility.” [2, p.40-41] This was also seen by Lindley [3], Bernardo [4], Degroot [5], and others throughout the decades after Shannon wrote his classic on information theory. In terms of what you ha…