Exactly - I should have mentioned this. Perhaps replace explained with predictable?
1 Like
Maybe “predictable in this data set by a standard, off-the-shelf model”?
2 Likes
For “risk factors” I have a problem interpreting the goal of the authors this paper helped:
Explicitly distinguishing between the different purposes of observational biomedical studies and explicitly matching the approach, interpretation and wording to the researcher’s intent will enable more focused and productive use of research resources. Avoiding the imprecise term “risk factor” and using a word, such as ‘predictor’, in risk stratification studies and ‘explanatory’ factor in causal studies might bring clarity of thought and thereby reduce unwarranted assumptions in biomedical research.
2 Likes
This study seems to interpret risk factors as both predictive and explanatory:
Late to the thread, I want to build on what the DAG does not cover. To strengthen ESMD points:
- Baseline ascertainment: This study lacks temporality to establish the exposure → outcome relationship, especially for outcomes like depressions. Usually with databases study, authors usually leave out 1-3 years at the start as baseline ascertainment period, seems like 4 months for this one, which likely inadequate.
- The control groups - they chose to compare within benign prostatic hyperplasia individuals, those who took medication and those who did not. This should raise everyone eyebrows, clearly one group has less disease burden than the other, so that they were not on medication. Or that they could just be wrongly coded as one time based on their symptoms
- What funny is the authors seem to know 1 and 2 which can be seen on sensitivity analysis. They compared risk of dementia/depression between drug groups and the results are less dramatic but chose to leave it out on the supplementary file. Seems like the main results are for optics.
Number 2 is particularly relevant within drug safety literature and I’ve seen many patients stressing over it… Especially with depression, it becomes a self-reinforce vicious cycle.
3 Likes
Hi Minh
I agree completely. In fact, there are many medical conditions for which administrative data will be extremely misleading if used for the purpose of trying to establish chicken/egg relationships (i.e., establishing which came first, the exposure or the outcome). Databases can be pretty reliable with regard to timing the onset of sudden/acute and highly symptomatic conditions (e.g., heart attack, stroke), but often completely useless for establishing the time of onset of more insidious conditions.
Most cases of dementing illness will be fairly insidious in onset, with many patients spending variable lengths of time (sometimes several years) in the “Mild Cognitive Impairment” (MCI) phase before their cognitive function objectively deteriorates. Sometimes physicians will be able to identify patients while they are in the MCI phase, usually because a family member/friend/visiting care worker expresses concern or because the physician notices certain problems (e.g., forgetting to refill prescriptions, repetitiveness, failure to recall previous discussions). These suspicions then prompt the physician to perform formal cognitive testing. Physicians will often re-test patients with MCI regularly in order not to miss the onset of worsening impairment, particularly if the patient is still driving. But if the patient is not driving, he might not attend for regular reassessment and might not present again until his impairment is a lot more advanced. And, not uncommonly (especially for patients without close friends/relatives or with very infrequent physician contact), patients will fly completely under the radar, only coming to attention once their cognitive impairment is fairly advanced. Often, impairment is first detected during a hospitalization for acute medical illness (when patients with underlying, but previously unidentified, more chronic impairment develop a superimposed delirium).
So unless a patient has been subjected to frequent formal serial cognitive tests over a period of many years (including at least one “baseline” cognitive test when he was cognitively asymptomatic), we will often never be confident when his decline started. But since we don’t do “routine” cognitive testing in older patients who are not reporting any symptoms (or whose families are not reporting symptoms, or in whom we don’t have reason to suspect cognitive impairment), we almost never have this type of historical objective information available. Usually, the first test isn’t performed until someone raises a concern; the result tells us where the patient is at this point in time, but not where he was 6 months/1 year/2 years/5 years ago.
For all these reasons, administrative databases will not be able to tell us, reliably, whether cognitive impairment was absent at the time an older male patient was first started on treatment for his BPH.
NEW- There’s an interesting parallel between: 1) non medically-trained researchers failing to appreciate the complexity of clinical practice/diagnosis/patient management, dredging administrative databases for “risk factors,” trumpeting these findings in the media and somehow expecting their “findings” to impact clinical practice; and 2) non statistically-trained physicians feeling that they can “do their own stats” or that AI can take the place of an experienced applied statistician. At the root of both phenomena is insufficient respect for the expertise of other professionals.
3 Likes
Yeah, I don’t think there is enough discussion around how these commercial database arose, especially related to the practice of coding.
Related to the specific paper we discussed, there is evidence that the authors knew about the nature of diagnosis we talked about here- they conducted a sensitivity analysis to shift the dementia diagnosis back three years and removed anyone with the new diganosis > the first day of initiating the medication
What funny is that they willingly left this less dramatic result into the supplementary materials for the optics of the more dramatic results.
Also concur to the last point, I used to be the clinical practitioner that ignorance to the practice of statistics. I think we need to encourage the exchangeability between these two populations, which might be hard because the Venn’s diagrams of these expertises hardly overlapped.
2 Likes
To clarify, in order to be fair to the authors, their paper does NOT actually claim a causal role for BPH medications in the development of dementia- quite the opposite, in fact. Rather, their paper discusses evidence they found that might explain why such observed associations could be erroneously interpreted if due caution is not exercised. I also agree that they acknowledge the limitations of database analyses. But I think the authors can be faulted for making the phrasing in their Discussion incredibly difficult to parse- too many instances of “we found this…HOWEVER…” And I wonder whether this garbled presentation might have caused the author of the original post in this thread to infer that the authors WERE actually implying a causal role for dutasteride/finasteride in cognitive decline (?)
1 Like
The “Table 2 fallacy” at the core based on the specificity of confounders to its causal paths. A confounder in one causal path might not be one in another. This paper gives off the vibe that the authors using the same set of covariates to adjust for different outcomes (dementia/depression), which could introduce biases if one of these covariate is confounder in one path and collider in another path. I think this is where the concern of this post came from.
I would not fault the authors trying to move-up the ladder of causality by the language in their discussion. They tried to establish qualities of causality such as temporality and strength of association in this paper. It would had been fine, in my opinions, if they did not exaggerate the strength of association by reporting numbers from in-appropriate control groups.
I think we should not shy away from saying causality when there is evidence to say so, in the end, that what we are trying to do in every day as human. 
1 Like
However a fundamental requirement is having a defensible control group. How should we give the authors credit as you say when this was not handled properly?
1 Like
Hi Minh
My problem with most observational drug safety studies - and, I suspect, the problem that most physicians have with them - is that their publication in prominent clinical journals doesn’t jibe with their limited or nonexistent clinical actionability.
What could be the main goals of research like this?
- To explore whether anecdotal patient reports of cognitive effects of BPH medications are substantiated by administrative databases?
- To explore whether these anecdotal reports might represent the tip of a massive iceberg i.e., could these reports involving commonly-used medications be responsible for a large fraction of dementing/depressive illness among older men (a major public health issue)?
- To further refine previous work on this topic by analyzing the data in different ways?
- To inform the field of neuroscience i.e., maybe the mechanistic neurologic effects of these medications are more important than we previously thought for cognitive function?
All of the above goals seem, superficially, to be defensible. But what’s left out of the equation here is consideration of what the patient suffering from intractable BPH symptoms is supposed to do if the above exploratory process terrifies him out of accepting treatment for his BPH…His other main options are as follows: 1) put up with his symptoms, risking acute urinary retention/kidney injury (every family physician has a few older male patients each year who suffer this complication); 2) put up with his symptoms and accept that he’ll need to go to the bathroom every hour for the rest of his life because his bladder is always full and never empties properly; 3) ask his urologist to do a TURP [though TURP is usually restricted to men with intractable symptoms in spite of BPH medications or men who can’t tolerate the medications (and the overwhelming majority of men will tolerate their BPH medications)]. Researchers also need to consider second-order effects of large numbers of men being scared (potentially unnecessarily) away from these medications. BPH is such a common problem among older men that, if enough refuse to use them, urologists would likely spend 24/7 doing nothing but TURPs. Surgeries for urologic cancers would be delayed due to overwhelming demand for TURP. And, without enough urologists to do all these TURPs, huge numbers of older men will end up walking around with permanent indwelling catheters…And TURP comes with its own list of potentially significant complications, including perioperative complications like bleeding/infection, urethral stricture, retrograde ejaculation, (rarely) erectile dysfunction or urinary incontinence etc…In short, withholding BPH medications from symptomatic men is not, at this point in history, a clinical option.
It’s important to try to unravel the mechanisms underlying common/complex/likely multifactorial human diseases like dementia and depression- this has not proven to be easy. Multiple lines of evidence will be needed. However, this imperative doesn’t change the fact that most exploratory observational drug safety research doesn’t belong in JAMA- a clinical journal. Most of these studies are not even remotely ready for clinical “prime time.” They belong in neuroscience or epidemiology journals, not splashed across the headlines of the evening news because they “made it” into a premiere clinical journal.
Researchers should not embark on this type of research before they consider all the potential ramifications of what they might find. Specifically, they should have a plan as to how they will react to a weak association that doesn’t disappear with various types of adjustment/sensitivity analysis. Will they, out of an altruistic fear of dangerously downplaying what they might view as a potentially monumental public health discovery, feel compelled to rush to a clinical journal for publication ? If so, then they should think again, because their actions might instead end up doing a lot more harm than good. Or will they, before embarking on their study, do a diligent survey of clinicians who care for patients who might be affected by their results, then decide whether to proceed and how they might appropriately contextualize their findings (a much more responsible approach)?
4 Likes
Frank (@f2harrell ) and Erin (@ESMD), appreciate your responses. I’m working to be more direct on what I wanted to say.
Let me establish my position again, I’m against the results of this paper. So against, in fact I wrote a whole meta-analysis to raise the issue of the control group regarding to depression outcomes since it has brought me (and others) so much trouble in practice to explain the depression risk associated with 5-alpha reductase inhibitors. Here is the abstract, full version is submitted to AJE:
"We conducted a systematic review and meta-analysis to examine the association between 5-alpha-reductase inhibitors (5-ARIs) and depression, focusing on reasons for heterogeneous findings, using Scopus, Embase, and MEDLINE from database inception to January 2025. Peer-reviewed studies reporting depression outcomes associated with 5-ARI use were included, excluding those with insufficient data or non-English publications. Study characteristics and relative risk estimates were extracted and pooled via random-effects meta-analysis, with stratification by control group and 5-ARI type to explore heterogeneity, following PRISMA guidelines. Five longitudinal studies (n=2,517,859, effect size=8) from the US, UK, Canada, and South Korea (1992–2018) showed a 31% increased depression risk with 5-ARI use (HR 1.31, 95% CI 0.98–1.76), with high heterogeneity (I²=95.5%, τ²=0.0984, P<0.0001). Stratification revealed elevated risk with non-drug controls (HR 1.61, 95% CI 1.20–2.16, I²=94.4%, τ²=0.0635, P<0.0001) but reduced risk with alpha-blocker controls (HR 0.89, 95% CI 0.86–0.92, I²=0%, τ²=0, P=0.9711). Comparator choice largely explained heterogeneity, while 5-ARI type showed consistent results. These findings highlight that inappropriate control groups in pharmacoepidemiologic studies may overstate 5-ARI depression risks, emphasizing the need for careful study design.
"
It has been awkward talking to patients on this issue, on top of the whole post finasteride syndrome thing.
3 Likes
Erin, agree with all counts you presented here.
We need to push back against these publications by publishing these conversation/against evidence in more formal avenues. These publications on side effects can be used as legal documents on medical lawsuits and basis for patients claim. The trend of them appearing in medical journals does not seem to stop.
On this, I think there is no stopping people to continue to use these databases to conduct research and the logic/methodological behind it is not so shaky. So, I’m not against the spirit of using these evidence to move-up the causality ladder, it just needs to be done in a correct way. If something is not done properly and in a bad faith manner, it needed to be called out.
3 Likes