When discussing the strengths and weaknesses of randomized controlled trials versus observational studies, I often point out that observational studies can typically achieve larger sample sizes and longer follow-up periods than RCTs. To illustrate this point, I wanted to provide a general overview of the size and duration of contemporary clinical trials, based on empirical data.
To my surprise, I couldn’t find a good source for this information. I was particularly surprised given the existence of ClinicalTrials.gov, a comprehensive, publicly accessible database, so I had assumed this question would be straightforward to investigate.
So, my first question is: am I simply overlooking an existing resource? Is there a paper or website that captures this information?
In the meantime, I put together a quick and rough analysis, which you can find here: GitHub - tamas-ferenci/clinical-trial-size-duration: Sample size and duration of follow-up of clinical trials (from clinicaltrials.gov).. If you find it interesting, I’d greatly appreciate any suggestions or criticism!
4 Likes
Outstanding report. It would be good to add in the intro section that many observational studies have a very small effective sample size once bias is accounted for.
2 Likes
Thank you very much! That’s a great idea, I’ll add a sentence or two about this! Do you have any good citation for this point that I can refer to in the text?
Ah, I see! Thank you very much. I thought you had something similar in your mind, but this post made it absolutely clear. I updated the introduction with a short reference to this. Thank you again!
1 Like
Great work, very interesting to see this formally contrasted between RWD & RCTs. Besides the “time frame” field you used, I suppose the start and end date of the study might work as another proxy? Maybe not for observational studies. I am not sure.
In all cases, I think studying this via actual published results is ideal. Even then, there is no standard way that people quantify follow up in, and the estimator is rarely reported too.
Rufibach et al. argue that even quantifying follow up is unnecessary: https://onlinelibrary.wiley.com/doi/10.1002/pst.2300
While observational studies would surely have more follow up, many of them include some immortal time at the beginning which you wouldn’t have in RCTs, among other issues.
2 Likes
Thank you very much!
Besides the “time frame” field you used, I suppose the start and end date of the study might work as another proxy?
Hm, that’s an interesting idea! I don’t have time to investigate it right now, so I added it as a possibility for further development.
In all cases, I think studying this via actual published results is ideal.
Exactly! I completely agree; that would be much better than the “time frame of the outcome” that we have now. Problem is, I have absolutely no idea on how we could scrape it from the publications… (And we must do that, i.e., look up the publications, because clinicaltrials.gov does not include such machine-processable field, not even among the results for trials that have results posted.)
Even then, there is no standard way that people quantify follow up in, and the estimator is rarely reported too. Rufibach et al. argue that even quantifying follow up is unnecessary: While observational studies would surely have more follow up, many of them include some immortal time at the beginning which you wouldn’t have in RCTs, among other issues.
I updated the text with a reference to this (I even cited this paper, thanks!).
2 Likes
Update: I managed to extract the therapeutic areas. So now we can have figures like this:
I thought it might be interesting; see the details in my Github-repository.
1 Like
Excellent work. Would be good to see by other factors also, e.g., single-center vs. multi-center trial, year of study start, medical vs surgical vs device, possibly region or multinational vs US only.
IMHO, at this point, you should set up a team and investigate this 
This is interesting (and the GitHub page is neat too, to see this develop). I see you’re working to exclude cluster RCTs, though there may need to be relatively nuanced search criteria to screen those out, and other things like stepped-wedge trials that may or may not be caught under a cluster term in the records.
My other thought (possibly beyond what could be done with large-scale scraping) is seeing whether the [primary] outcome is binary or continuous as a major driver of sample size – this would be most useful when considering comparison with observational studies, where studies based on administrative/registry type observational data (in contrast to prospective observational studies with primary data collection, a la classic cohort studies) would be unlikely to have continuous outcomes.
See this for @EvZ 's fantastic work on sample size for binary vs. continuous outcome trials.
Adding to James’ excellent thoughts, a reason that stratifying by pragmatic/cluster-randomized vs. traditional RCTs is important is that cluster-randomized trials need much larger sample sizes to have the same power as individual-randomized trials. It is also much easier to accrue patients since they usually don’t use informed consent. That is related to another criterion: first-in-man RCTs vs. RCTs of accepted practices / treatments / repurposed drugs.
