I would earnestly appreciate clarification or elaboration of the admonition in BBR section 5.12.5 (minute 40.35 of the video): “Never use change from baseline as the response variable except in a non-randomized pre-post design (the weakest of all designs)”. This reiterates the emphatic caution about change-from-basellne (CfB) expressed elsewhere in BBR; RMS; Altman in BMJ, 2001; etc. But, as expressed here, the qualification ‘except in non-randomized pre-post design’ seems to imply permission or exemption in the case of observational pre-post designs—which perplexes/confuses me.
Observational pre-post designs with CfB analysis are very prevalent in health research and Pharma’ (especially now with expanding interest in the impact of treatment on PROs), and likely a great source of scientific noise, confusion and waste. But like significance testing, p-values, etc., CfB is so prevalent it is assumed to be natural, a conventional standard, and legitimate; and making a cogent case for eschewing CfB and embracing alternative approaches is a challenge for advocates of progressive scientific methodology.
I should note that many/most of these CfB analyses are in uncontrolled, single arm observational cohorts. While it is almost axiomatic that CfB is NOT = treatment effect; many construe CfB as indicative and significant, even in uncontrolled studies with pre-post- comparisons.
In the case of ‘non-randomized pre-post designs’ modeling the raw outcome variable of interest (post) with adjustment for baseline (pre) and other covariates to account for heterogeneity and extraneous variance (ANCOVA) is recommended by Frank and others.
Q1: is there a reason for the qualification “except in a non-randomized pre-post design”----is there something I do not understand that makes this instance extenuating?
Q2: are there other approaches that should be considered or advocated for in addition to ANCOVA?
Q4: What are these essential /major principles underlying the criticism of CfB that will most trenchantly discourage attachment to this practice?
Q5: Can anyone share/direct me to simulations that help illustrate the defects, risks and principles underlying the criticism of CfB that anyone is aware of? (e.g., https://rpsychologist.com/treatment-response-subgroup) Or compelling parables: examples of epic failures of CfB.
It will be a difficult campaign to effectively change understanding and practice. I am very grateful for advice, guidance, and input.
I am very very grateful for BBR!!