Sorry to hear this, @daszlosek. I have precious little experience working through the FDA regulatory process, so Iâm afraid that I can offer little constructive commentary beyond things you already know, and of course Iâm always open to having my mind changed if something I say below is wrong. ButâŚ
First, try Frankâs intuitive explanation about the quantity of interest being the difference between the groups after treatment, using adjustment for baseline score, so the statistical model essentially asks: âfor 2 patients with the same baseline score, one receiving treatment and the other placebo, what is the expected difference in their follow-up scores?â
Second, send the BMJ Stats Notes paper explaining that raw-score-adjusted-for-baseline has increased power versus using change scores. (EDIT: and this paper National Institutes of Health Stroke Scale as an Outcome in Stroke Research: Value of ANCOVA Over Analyzing Change From Baseline - PubMed (nih.gov)
Third, use simulations to show performance advantages of this approach over using change scores (this ought to be more convincing than it actually seems to be in practice; unfortunately it seems an appeal to authority, like #2, is often more effective than actually illustrating something yourself).
You have probably done all that, unfortunately, and still meet resistance. My sympathies. I do wonder if other statisticians or consultants with regulatory experience will be able to chime in, though perhaps they cannot always reveal details depending on the nature of their work and agreements with clients.
Both of the arguments you listed are indeed weak - neither is a scientific or statistical justification; both are simply arguing to do it because âthis is the way weâve always done itâ (and while sometimes there may be a good scientific or statistical reason for âthis is the way weâve always done itâ - then thatâs the argument we should hear, not simply âthis is the way we have always done itâ by itself).
The second point is particularly frustrating to me, perhaps more than the first. Why is it a requirement for a new study to be âcomparableâ to a previous study? The goal ought to be designing a new trial to achieve its objective(s) as efficiently as possible. Saying that any new study must use a particular statistical approach just to be âcomparableâ to previous studies is ridiculous (by that standard every new trial must be designed the same as any previous trial, we may as well cease research into any new statistical methods for clinical trials - every new trial simply must be done with the same design as predecessor trials).
I, too, am sympathetic (if frustrated) that the attitude from a company/companyâs statistician would be the conservative âWe want our study design to be accepted by FDA and any other reviewers without too many arguments, so weâd rather be less efficient but guarantee that our drug will be approved if it meets the primary endpointâ rather than proposing something even slightly âdifferentâ (how amusing that even a fairly simplistic analysis is perceived as too risky). I have not done much consulting for industry sponsors, but in the little experience I have, those vibes are very strong - would rather use a statistical design that they already know FDA is accepting of. The endgame is to guarantee approval. Potentially needing fewer patients (and therefore saving a few dollars on the trial) seems to be viewed as a relative pittance if it puts the approval at risk for any reason.
Itâs not a regulatory setting, but in one trial that I am currently working on, our first meeting with the DSMB had several similar comments from the DSMB statistician. We will have a baseline measure and then a 6-month assessment and our statistical analysis plan is, effectively, this:
Follow-Up Outcome = b0 + b1*(Treatment) + b2*(Baseline Measure)
b1 will tell us the treatment effect on follow-up outcome, adjusted for baseline. Simple, right?
The DSMB statistician asked why we were not doing a t-test comparing the change scores between groups, and despite my best efforts to explain the above, seemed unconvinced. This person did not really have authority to mandate changes to our statistical plan, but it was frustrating to encounter a dogmatic, outdated view. Frank has previously noted that perhaps some statisticians are resistant to change because it feels like critiquing their past selves - for which I have some mild sympathy, but it does not seem a good reason to demand others follow your preferred approach if the alternative has a strong statistical justification, as would be the case here.
Anyways, happy to hear more educated opinions from others, and apologies that I could offer mostly just âcommiserationâ rather than constructive suggestions.
). Unfortunately as a consultant you have to heed the advice of your client. Itâs a tough position to be in, you might notice when you raise an issue the other consultants in the room keep quiet, thatâs what i noticed when consulting, they need the work, theyre self-employed. The problem you have is that the very best argument to persuade the client is to say âitâs what the regulator want/expectâ (itâs not using technical statistical arguments), but the statistician at the company has diffused that argument.