These (to me) do not mean anything in terms of supporting the notion that a single likelihood ratio has any utility as a measure of discrimination - I think a data example is needed to get your point across
Your own example shows that ratios of probabilities when framed in terms of power and type I error can be constant, while converting those same probabilities to odds cannot be.
The fundamental relationship between frequentist alpha levels and Bayesian update from prior to posterior is shown via simple algebra.
Starting from Berger et al.s Rejection Ratio paper, the pre-posterior odds needed for an experiment is:
O_{pre} = \frac{\pi_{0}}{\pi_1} \times \frac{1 - \bar\beta }{\alpha}
The term with \pi can be thought of as a skeptical odds ratio \lt 1, with the entire ratio being a proportion of true to false rejections of the reference null hypothesis.
O_{pre} can be seen as the poster odds ratio \gt 1 needed to confidently reject the null hypothesis based not only on the design of this single study, but also on prior assessment of the relative merit of the hypotheses.
\alpha = \frac{1-\beta}{Odds(\theta|data) \times Odds(\theta)}
In Berger’s GWAS example, the scientists did not need for the poster odds to be greater than 1 in order to claim a discovery; simply going from \frac{1}{100,000} prior odds on a gene - disease relation to \frac{1}{10} (leaving high posterior odds that there no relationship even after the data were seen) was considered worthwhile from a scientific decision analysis.
Are you disputing that the Bayes factor is a ratio of frequentist error probabilities? If so, why?
The expression you have typed is incorrect - please have a close look at our paper. Also using alpha and beta will be confusing for everyone so I recommend sticking to TPR and FPR. Our example shows no such thing - if you think it does please explain how so.
- Which expression is incorrect?
- The only one confused here is you. Your entire framing of this problem is wrong.
1 Like
[quote=“R_cubed, post:607, topic:4403”]
α=1−β/(post x prior)[/quote]
The above is clearly incorrect. I am not going to make a decision about who is confused - the expression speaks for itself…
It literally takes high school level algebra to re-arrange the equations in this paper, that I have posted at least 2 times in this exhausting thread. Before making a blanket claim that I am wrong, you need to read it.
The authors are all top notch statisticians. I’m confident they can do high school level algebra.
From the highlights:
Pre-experimentally, these odds are the power divided by the Type I error.
When the prior and posterior are in odds format, this is precisely how \alpha was calculated in a GWAS study where James Berger was a consultant statistician.
You need to read the Rejection Ratio paper (cited above) in order to understand the argument, and why your claim about risk ratios vs ratios of odds ratios is wrong.
He describes his reasoning in this video (from about 5:00 to 30:00)
(The relation among posterior odds, prior odds and power to \alpha described at 25:00 - 28:00 mark).
I used similar reasoning in a re-analysis of a “null” meta-analysis claiming “no association” between excess knee ROM and risk of ACL injury for amateur athletes.
There is nothing special about diagnostic testing that has not already been explored in the hypothesis testing literature, where likelihood ratios are derived from frequentist error probabilities.
Related References
You can derive adaptive p values (significance threshold decreases as sample size increases) by minimizing the a linear combination of the error probabilities.
Luis Pericchi, Carlos Pereira “Adaptative significance levels using optimal decision rules: Balancing by weighting the error probabilities,” Brazilian Journal of Probability and Statistics, Braz. J. Probab. Stat. 30(1), 70-90, (February 2016)
1 Like
@R_cubed I understand your frustration but I’m not sure it’s even worth it to pursue this line of argument. As @AndersHuitfeldt has pointed out, the whole idea that the RR (or its interpretation) might be in violation of Bayes’ theorem is just a category error.
@s_doi In the section “Application of the OR versus the conventional RR to risk prediction”, you make frequent reference to “Method 2” (essentially the RR) being inconsistent with Bayes rule. I must ask again, what is your precise definition of being inconsistent with Bayes rule? It is certainly not defined in the paper. Furthermore, your references to “updating” probabilities have nothing to do with the standard idea of Bayesian updating. In the examples you consider, we are “updating” from a probability P(Y | X =1 , S = s) to another probability P(Y | X = 1, S = t) for some values s and t, where s and t stand for different populations of interest. Here we generalize to another population by changing the value of the covariate representing population that we are conditioning on. Bayesian updating, on the other hand, strictly deals with rearrangements of conditional probabilities keeping the conditioned on quantities the same: P(Y = y | X = x) = P(X = x | Y = y) P(Y = y) /P(X = x).
2 Likes
This talk tomorrow at the Online Causal Inference Seminar may be of interest to participants and readers in this discussion:
Hello everyone,
The Online Causal Inference Seminar is excited to welcome two student speakers, Benedicte Colnet from INRIA, and Keegan Harris from CMU, to present at our student seminar. The titles and abstracts are copied below. The seminar is on Tuesday, May 30 at 8:30am PT / 11:30am ET / 4:30pm London / 6:30pm Tel Aviv / 11:30pm Beijing .
You can join the webinar on Zoom here (webinar ID: 996 2837 2037). The password is 386638.
As a reminder, you may suggest a speaker or propose to speak for future seminars here.
We look forward to seeing you on Tuesday!
Best wishes,
Organizers of Online Causal Inference Seminar- Student speaker 1: Benedicte Colnet (INRIA)
Title: Risk ratio, odds ratio, risk difference… Which causal measure is easier to generalize?
Abstract: There are many measures to report so-called treatment or causal effect: absolute difference, ratio, odds ratio, number needed to treat, and so on. The choice of a measure, e.g. absolute versus relative, is often debated because it leads to different appreciations of the same phenomenon; but it also implies different heterogeneity of treatment effect. In addition some measures – but not all – have appealing properties such as collapsibility, matching the intuition of a population summary. We review common measures and their pros and cons typically brought forward. Doing so, we clarify notions of collapsibility and treatment effect heterogeneity, unifying different existing definitions. Our main contribution is to propose to reverse the thinking: rather than starting from the measure, we start from a non-parametric generative model of the outcome. Depending on the nature of the outcome, some causal measures disentangle treatment modulations from baseline risk. Therefore, our analysis outlines an understanding what heterogeneity and homogeneity of treatment effect mean, not through the lens of the measure, but through the lens of the covariates. Our goal is the generalization of causal measures. We show that different sets of covariates are needed to generalize an effect to a different target population depending on (i) the causal measure of interest, (ii) the nature of the outcome, and (iii) the generalization’s method itself (generalizing either conditional outcome or local effects).
2 Likes
Why should they have any thing to do with what your standard idea is? Why should our notion of updating align with your understanding of the terminology?. One of the good things about our papers is that we use data examples to indicate exactly what we mean so there is no confusion. The confusion is created by vested interests when scientific jargon is pursued in an attempt to justify what cannot be justified or perhaps to obfuscate. Also there seems to be a lot of misunderstanding of what the expressions mean as seen with the use of expressions by @R_cubed - 1-beta/alpha x prior = posterior - simple algebra indeed but clouded by what we want to prove regardless of what has been shown.
Regarding the relationship between Bayes factors, rejection ratios, and frequentist error probabilities:
You are certainly correct. I did not entirely understand the claims made by Doi until I read that statement regarding likelihoods being ratios of ORs rather than ratios of conditional probabilities.
The only claim in contradiction to Bayes Theorem, and basic concepts in information theory, is Doi’s. This can be easily seen by careful examination of the equations in Berger et. al’s paper, where LRs are ratios of frequentist error probabilities.
This can also bee seen in Doi’s own example
In order for a likelihood ratio to behave as expected, constants of proportionality can be ignored. That cannot be done when converting a ratio of probabilites into ratio of odds ratios as his own chart shows.
If ratios of odds ratios appear anywhere in re-arrangements of Bayes theorem, it is on the left side of the equation, after converting prior and posterior target probabilities to odds, as in the relative belief representation. From that, it is easy to see how \alpha relates to Bayesian updating via power considerations.
I am grateful that @AndersHuitfeldt persisted in this discussion. I’ve learned much from considering the arguments in his papers and following up on some citations. I think there is merit in the proposed switch risk ratio, although I see no objection to using methods recommended in RMS to compute it.
1 Like
Hi Suhail
Thanks for responding. One comment about your paper: I wonder if the distinction between the Odds Ratio as used in the observational study context versus in the RCT context should be highlighted more clearly (?) Maybe my understanding of the term “clinical epidemiology” is too narrow, but when I hear the word “epidemiology,” I automatically think of observational studies rather than RCTs…
When I took introductory epidemiology courses many moons ago as a medical student and resident (and later as a job requirement), my instructors were either practicing epidemiologists or physicians with some additional epidemiology training. This is what I recall being taught:
- Confounding was explained as a potential source of bias. A “confounder” was a factor that was associated with BOTH the patient’s likelihood of receiving a certain treatment AND the outcome. In observational studies, one way to test to see if a factor was acting as a confounder was to see if the point estimate moved by more than a certain amount if you adjusted for that factor. This led to a view that movement of a point estimate in response to an adjustment procedure signalled that “bias” was present in the study (and therefore it was a good thing you did the adjustment);
- If you failed to adjust for confounders, then the point estimate from your study would have been “biased” (i.e., farther from the “truth”).
Maybe the above teaching wasn’t off base in the context of analyzing observational studies (?) But physicians primarily use RCTs to guide treatment decisions (when available). Students who are taught critical appraisal by epidemiologists will extrapolate epidemiologists’ definitions of “confounding” and “bias” to their appraisal of RCTs. Confusingly, statisticians seem to have their own understanding of “bias” and “confounding” and how these terms apply in an RCT context. And when outcomes of an RCT are binary and Odds Ratios come into the picture, all hope for understanding by novices is lost. I suspect these are the reasons why students get so confused:
- In at least some epidemiology circles (or at least in the minds of students they teach), movement of a point estimate as a result of an adjustment procedure seems to have been conflated with the notion that “bias was present;”
- Epidemiologists (but not statisticians?) use the term “random/chance confounding;"
- Hapless medical students and residents trying to understand how to appraise RCTs wonder how it can be true that “confounding is addressed by randomization,” yet epidemiologists still discuss “random confounding…”
My understanding (?maybe incorrect):
-
Statisticians seem to focus on the “in expectation” part of the randomization concept as being most important when discussing the potential for confounding to be present (or not). In other words, statisticians implicitly tack on the phrase “in expectation” when defining a confounder- they view a confounder as a factor that is associated (“in expectation”) with both the likelihood of receiving a particular treatment and the likelihood of the outcome of interest (?) If the “expectation” is not present, EITHER because there’s a lack of prior evidence that the factor in question is associated with exposure (an important consideration when analyzing observational datasets) OR because the act of treatment allocation has been divorced from the likelihood of receiving a certain treatment (through the act of random allocation), then the factor will not exert confounding bias;
-
In contrast to statisticians, epidemiologists seem to focus not on the “in expectation” angle to randomization, but rather on the outcome of the randomization process (i.e., how the covariates ended up actually getting distributed between arms as a result of randomization i.e., how the randomization procedure actually “turned out”). I realize that there are probably decades of published studies on “random confounding,” but it seems like a very difficult concept for students to reconcile with the definition of a “confounder” that seems to be preferred by statisticians.
-
Statisticians consider that valid inference hinges most fundamentally on proper expression of uncertainty in a study result. A trial result must accurately reflect the things we know and the things we don’t know about the phenomenon under study. If we know that certain factors are prognostically important AND if we have measured these factors, then we must adjust for them in our analysis in order for our resulting inference to be valid. Conversely, if either we are unaware that certain factors are prognostically important OR if we are aware but unable to measure prognostically important factors (with both scenarios leading to a failure to “adjust” for these factors in our analysis), then our inference will remain valid. However, since our study will have been suboptimally “efficient” at identifying any underlying signal of treatment efficacy, we could be faulted for an unethical trial design (exposing more patients to an experimental treatment than is necessary to detect the efficacy signal). Before running an RCT, we should thoroughly understand the condition being treated, making every effort to measure known important prognostic factors (?)
Would this be a fair summary?:
-
It’s important to understand the distinction between confounders and prognostic factors;
-
When interpreting different types of effect measures, it’s important to recognize how each one can be affected by confounders and prognostic factors and to consider whether the study design is observational or experimental;
-
Epidemiologists and statisticians don’t seem to agree that “random confounding” exists (?);
-
Both confounders and prognostic factors can affect the OR in observational studies, whereas it will primarily be the latter (i.e., prognostic factors) that affect the OR in RCTs (?);
-
When we adjust for important prognostic factors in an RCT that is using the OR as the effect measure, the OR tends to move farther from the null, while confidence intervals become wider. Overall, the adjustment for important prognostic factors tends to confer greater power/efficiency to the study, since the point estimate moves away from the null more rapidly than the rate at which the confidence interval widens (?);
1 Like
I think we’re getting a bit far from the original intent of this topic, and I for one do not enjoy hearing any more about Bayes’ factors, \alpha, \beta, likelihood ratios (personal opinion), and what constraints in one metric lead to on another metric.
Hi Erin, these are great questions that take us back to what is important – decision making in practice which is after all the real goal of all these methodological discussions which sometimes get railroaded by vested interests. In response:
-
Agree, it is indeed critical to understand the distinction between confounders and prognostic factors; The former leads to bias if ignored and the latter to averages that apply to no one if ignored. Note that all confounders are also prognostic for the outcome so if we say ‘prognostic factors’ we mean non-confounding prognostic factors.
-
Absolutely right, interpretation of different effect measures needs recognition of how each one can be affected by confounders and prognostic factors and consideration of RCT or not; However, it is also important to understand what ultimately an effect measure is intending to measure and whether it measures the intended association or something additional to it.
-
I am not sure if it is that clear cut – what I meant previously is that covariate imbalance can occur by chance in a RCT but we do not need to adjust for these unless it is a strong prognostic factor. There is no ‘random confounding’ – confounding is due to differential distribution of covariates that influence both treatment and outcome (independent of treatment) in the compared groups and leads to real associations (or absence thereof) except that they are spurious. I doubt there is a divide on this between epidemiologists and statisticians.
-
Absolutely right for properly conducted RCTs
-
Yes, that is right. While there have been several calls for covariate adjustment in individually randomized trials, this has been raised in relation to the statistical perspective which links prognostic covariate adjustment with increase in statistical power to detect a treatment effect yet (paradoxically) decreased precision for the treatment effect estimator.This only happens with baseline covariate adjustment in RCTs analyzed using models that are noncollapsible and this is not a paradox because the precision comparison is made for estimators of different estimands while the conditional and marginal estimands share the same null.
1 Like
True, we see that all the time. This was because the 10% rule got accepted quite rapidly. Then when logistic regression did not really align with that (noncollapsibility) people began discussing how to distinguish between a change in estimate due to noncollapsibility from confounding. In my view, the change in estimate criterion for confounding should be completely dropped and we should move on to some DAG based procedure but not the sort suggested by Pearl in his ‘Book of why’ for reasons outlined here
1 Like
I think you may be right about this because we are dealing with HTE in the experimental context and confounding/colliding in the observational context. I will request Jazeel to look into this.
Regarding clinical epidemiology, I would consider this ‘the science behind good clinical research and evidence based decision making’. Sometimes clinician researchers get carried away by their concept of ‘clinical research’ not realizing that all clinical research is epidemiological research. An interesting read about this is my paper titled ‘Angry scientists, angry analysts and angry novelists’.
Addendum
This is a relevant paper to this discussion because it addresses application to practice. I sent in a rapid response as follows:
Dear Editor
Murad et al. compare four methods of deriving the RD from a meta-analysis. Of note, neither the RD nor the RR [1, 2] are reliable effect measures for use in meta-analysis and the same authors also suggest that the bivariate random effects model makes distributional assumptions that are unlikely to be tenable in practice [3]. Therefore none of the four methods they describe seem acceptable in practice. In my view a fifth method should be used. As we have previously described [4], the odds ratio (OR) can be viewed as a likelihood ratio that can be used to update the baseline risk (r0; on stenting) to that after endarterectomy (r1). The meta-analytic OR (IVhet model [5]) is: 0.724 (95%CI 0.595, 0.881) and therefore using this method the RD’s are as follows:
1% baseline risk: RD is 3 fewer (1 fewer to 4 fewer)
5.3% baseline risk: RD is 14 fewer (6 fewer to 21 fewer)
10% baseline risk: RD is 26 fewer (11 fewer to 38 fewer)
The values of r1 at the three values of r0 are 0.7%, 3.9% and 7.4% respectively. These results can be compared to the results they present in Table 1. These computations can be done in Stata using the logittorisk package [6].
References
- Doi SA, Furuya-Kanamori L, Xu C, Lin L, Chivese T, Thalib L. Controversy and Debate: Questionable utility of the relative risk in clinical research: Paper 1: A call for change to practice. J Clin Epidemiol. 2022 Feb;142:271-279.
- Bakbergenuly I, Hoaglin DC, Kulinskaya E. Pitfalls of using the risk ratio in meta-analysis. Res Synth Methods. 2019 Sep;10(3):398-419.
- Liu Z, Al Amer FM, Xiao M, Xu C, Furuya-Kanamori L, Hong H, Siegel L, Lin L. The normality assumption on between-study random effects was questionable in a considerable number of Cochrane meta-analyses. BMC Med. 2023 Mar 29;21(1):112.1.
- Doi SAR, Kostoulas P, Glasziou P. Likelihood ratio interpretation of the relative risk. BMJ Evid Based Med. 2022 Aug 11:bmjebm-2022-111979. doi: 10.1136/bmjebm-2022-111979.
- Doi SA, Barendregt JJ, Khan S, Thalib L, Williams GM. Advances in the meta-analysis of heterogeneous clinical trials I: The inverse variance heterogeneity model. Contemp Clin Trials. 2015 Nov;45(Pt A):130-8.
- Furuya-Kanamori L & Doi SAR, 2020. “LOGITTORISK: Stata module for conversion of logistic regression output to differences and ratios of risk,” Statistical Software Components S458886, Boston College Department of Economics, revised 16 Oct 2021.
LOGITTORISK: Stata module for conversion of logistic regression output to differences and ratios of risk
They did mention this fifth method I allude to in Box 1 but just dismissed it so this rapid response (if accepted) will bring focus back to this issue.
1 Like
Hi Suhail
There is no ‘random confounding’
https://twitter.com/Lester_Domes/status/1269085065067151362
NEW: From the Twitter thread linked above:
That is, in randomized trials, we are better off ADJUSTING for prognostic factors that happen to be imbalanced between groups.
It might seem simplistic to keep harping about the divide between statistics and epidemiology (and how often the two fields seem to use the same terms to mean different things). But there really does seem to be a major “Tower of Babel” problem here. If experts can’t even agree on the meaning of basic terminology, students’ understanding of fundamental concepts will be a function of whether they were taught by an epidemiologist or a statistician.
The overwhelming majority of physicians (me included) will have no hope of understanding this thread. And that’s okay/not surprising- this is, after all, a forum for methods experts to debate each other, not for clinicians.
The only purpose of my long post above is to highlight the potential implications of these disagreements for how clinicians interpret published RCTs. The implications are significant, as shown by the never-ending debates around interpretation of the evidence base for thrombolysis in acute ischemic stroke (as outlined in another thread).
Maybe I should stop worrying. After all, most physicians these days use pre-digested sources of evidence anyway (e.g., UptoDate), rather than appraising RCTs themselves (the approach to EBM I was taught in the early-mid 1990s, when UptoDate didn’t exist…). Maybe it no longer matters whether they understand concepts like bias, confounding, and Odds Ratios (?)…But as an observer, I’m just trying to point out that the communication/ language divides between statistics and epidemiology seem profound and foundational. Until they are sorted out, maybe it would be better not to even attempt to teach clinical students these concepts, rather sending them mixed messages…
3 Likes
Hi Erin, I share your frustration as that has happened with me too. First, what I meant by no random confounding referred to the covariate and that is why I said that you can get confounding by chance (finite sample) to avoid implying that confounding is not a property of the covariate. I am sure that is what Sander meant too except that he used a different choice of wording. Unless we give applied examples this will remain the case. Take for example the use of risk, rate and proportion - different people mean different things. True positive rate is common terminology but in reality it is true positive proportion but people need to see the examples to find out what is meant.
In this thread and in my papers I used ‘portability across baseline prevalence’ as a term for what others have called ‘variation independence’. The epidemiologists who rebutted my paper did not bother to examine the examples I gave and the whole rebuttal was about transportability (extent to which the treatment effect holds across various characteristics of persons and settings) rather than ‘variation independence’ and also because these are trials, complained about ‘prevalence’. The reality is that I was not discussing generalizability at all but rather variation dependence or not and the fact that I was referring to prevalence in a sample at the end of follow-up was ignored. Yes, incident outcomes occur in trials but the variation independence I was talking about related to prevalence at the end of follow-up in a sample. No discussant was willing to examine this critically and a lot of comments were made about this too. Variation dependence is however related to prevalence and not to baseline risk.
Another example is the very recent comment by sscogges when I was discussing updating of proportions using Bayes’ rule. He latched on to his concept of Bayesian updating as defined for other purposes and without thinking about what I mean went on to say that there was a problem - the applied example was there, but people are so buried in their misconceptions about what they know that they refuse to relate to the problem at hand.
Is this a problem of a divide between epidemiology and biostatistics? No. I think this is simply scientific arrogance - we are so confident about what we know that we refuse to consider what has been said. Basic terminology can be overcome through applied examples that show what we mean but no one wants to do so. There are of course those on this thread who make makes sure they understand before any judgment is offered but that usually happens when there is no vested interest. When we do not have a career interest or reputational interest in the discussion that is when we focus and try to understand and unfortunately many papers out in the literature are there simply to bolster careers and therefore ‘experts’ tend to call each other out on terminology rather than core concepts.
I agree with you that UpToDate is a core resource and has the predigested interpretations from the research literature - but this is limited to common scenarios. As soon as we get an exception, we need to weigh the literature ourselves and reach a decision. It matters a lot whether clinicians understand the effect measures, bias and statistical results and we must teach this to medical students. If we do not do so, what is the alternative? Do we put fortune tellers in the clinics and rely on them for decision making? I think its not biostatisticians and epidemiologists that have to decide for clinicians how to think about research output. We need to make the necessary decisions but for that we need to understand the core concept clearly so that we are not swayed. A common example given is that there are no green cows. However if I understand the concept of cow clearly, then if someone paints a cow green and shows this to me and asks what is this, I should be able to say ‘its a cow, but I dont know why it is green’. Same applies to a patients presentation - not many are classical and the same applies to the choices we make after reading the research literature.
Addendum
I just read the twitter link fully but I can see that the responder was as frustrated as we both are: ‘I’ll take “terminologic dogmatism” over inventing nonsensical oxymorons for things that already have names.’ To be clear, this was a bit harsh and Sanders response was equally harsh
2 Likes
Of course you’re free to define a procedure and call it updating or producing a predicting probability or whatever else you want to call it. But you are going far beyond that when you 1.) claim that particular ways of performing that updating are consistent/inconsistent with Bayes’ rule and then 2.) use that supposed consistency/inconsistency to advocate for one method over another. If you’re making those sorts of claims, you actually have to provide justification. My simple point, from the beginning, has been that neither the OR-based or the RR-based methods for updating, as you put it, have any substantive reliance on Bayes rule, and that to talk about one or the other being inconsistent with Bayes rule in the first place is nonsensical. So why is that the case? Well, we’re obviously in agreement that neither involves updating in the Bayesian sense. You would probably counter that the OR-based method is consistent with Bayes rule because, in the way you’ve written it, it uses Bayes rule in the calculations. But in fact, the OR-based method doesn’t really rely on Bayes rule, a point that Huw made in this perceptive comment:
In other words, Bayes rule is not needed at all for the OR-based method of updating that you advocate for. All you’ve done is shoehorn Bayes rule into the calculation in a completely superfluous way, and then claimed that this is some grand result demonstrating the superiority of the OR over the RR.
To claim that your critics are quibbling over terminology instead of engaging your claims is also completely off base. Besides the efforts of Sander, Anders and others, I went through the OR-based and RR-based methods in detail in previous posts, demonstrating why they actually produce different results and showing the conditions under which each one would provide a reliable basis for updating, as you put it. Again, it had nothing to do with one method or the other violating Bayes rule and everything to do with more fundamental assumptions about stability of effect measures across groups. I even illustrated this by providing an example where updating based on the OR actually fails but updating based on the RR succeeds. Despite your professed love of empirical examples, you refused to comment on it, offering as an excuse that you had already addressed it in your previous conversation with Huw. When asked to actually point to where in that conversation you had addressed this, you offered…silence.
Finally, you have repeatedly refused to offer basic clarifications about what you mean by things like a method being consistent with Bayes rule or in violation of Bayes rule, despite these sorts of claims appearing over and over again in your writings. According to you, it would be better if people somehow intuit what this means through examination of your numerical examples. Let’s assume for the sake of argument that this is true. We have in this thread a collection of experienced clinicians, statisticians and epidemiologists. I’m genuinely curious to know: does anyone in this thread feel like they’ve understood from his examples the point that s_doi is trying to make about the OR, the RR, and Bayes rule? If I am missing something then please, enlighten me!
5 Likes
First, thank you for acknowledging that I can use the terms I would like to use to define a procedure that I describe (with data examples). For the claims I have made, I have used applied data examples throughout and in addition produced a preprint based on the discussions that contains step-by-step introductions to the calculations, results and interpretation. On the basis of the latter I state the conclusions 1) & 2) that you disagree with.
Your main argument is that the OR-based method for updating that I describe do not have any substantive reliance on Bayes’ rule. The only way you can assert that this is false is to take the expressions from the paper and demonstrate that the relationships do not hold as stated in the data examples I provided – the fact that I have written it up means that you now have a perfect opportunity to rebut it – why is that so difficult for you?
The reference you made to Huw demonstrates that you have not bothered to read the paper at all – when Huw says ‘Actually, I did not calculate the RLR as you do but got them directly by dividing one odds by another.’ has no relationship whatsoever to whether the OR based method relies on Bayes’ rule or not because each LR is a ratio of odds and the RLR is therefore itself directly computable by dividing one odds by another. I still find it puzzling why you call out a method you have not read or understood and my guess is that you must be speaking from your confidence in the status quo rather than any scientific point of view.
I once again come back to the same point – yes my critics (yourself included) are indeed quibbling over terminology and not engaging science. You have not gone through my data example at all (the only person who did that was Huw). Yes, indeed, each method produces different results, hence they both cannot be correct. As I said in my previous post, the debate here is not about transportability, it is about variation dependence and the process of updating risks. I do not recall any real effort on your part in this thread of your providing a data example. If I was silent it was because you asserted something theoretical that did not make sense (to me).
Since you have dogmatically asserted that the OR has nothing to do with Bayes’ rule and are calling out others for clarification let me pose a question:
Study in a low risk stratum 15% prevalence of death (odds=0.176)
Heart transplant: r0=0.1 and r1=0.2 (r0 risk under no transplant and r1 risk under transplant). Therefore RR = 2 and OR = 2.25
Apply to a high risk stratum 85% prevalence of death (odds=5.67)
Using Bayes’ rule:
Od(r1) = 5.67 x LR1 (aka pLR)
Od(r0) = 5.67 x LR2 (aka nLR)
Od(r1)/Od(r0) = LR1/LR2 = 2.25 = OR (2.25 from previous study)
Thus when I update from a new r0 of say 0.8, I get (4 x 2.25)/(1+(4x2.25)) = r1= 0.9
This is exactly what happens also in logistic regression
Can you now explain to the readers of this thread how ‘Bayes rule is not needed at all for the OR-based method of updating’ that I have presented here?
So many of you have tried to reach @s_doi – he is beyond reach.
Remember that @sander said:”… I may be more exasperated with your resistance to stepping outside your framework, and how you seem committed to defend your framework as sufficient for understanding what we are writing about. I have encountered many others who seemed to be similarly “cognitively locked” …”
2 Likes