You are right - they must add to 1. When I look back I realize that what I put as p(C|O) in the previous post is actually p(T|nO). I will edit this and put the changes in bold and then respond again after I think this through
1 Like
@HuwLlewelyn I think I have figured it out and you can examine the below and suggest any flaws:
First, we agree that when p(O|C) goes from 0.4 to 0.8 due to change in severity then p(C|O) and p(T|O) do not change after using the RR in its conventional interpretation. But that is not important because what we want to know is if RR × p(O|C) = 0.4 = p(O|T) is in keeping with or in violation of Bayes’ rule (if we are to think about this as consistent with the diagnostic paradigm for your book).
To check that we compute the two LR’s and take advantage of the fact that the ratio of the two likelihood ratios is mathematically exactly the same when interpreted as:
a) connecting p(T|nO) to p(T|O) = 0.375 (we know this)
b) connecting p(O|C) to p(O|T) = 0.375 (we assume this)
(this can be checked using simple algebra on a 2×2 table)
So, from Bayes’ theorem, under the new level of severity and using the ratio of likelihood ratios to compute posterior probabilities then:
p(O|T) = [(0.8/0.2) × 0.375] / (1+ [(0.8/0.2) × 0.375]) = 0.6
Thus the p(O|T)=0.4 with the conventional RR interpretation violates Bayes’ rule.
The OR however is unable to violate Bayes’ rule on posterior probabilities for the simple reason that the ratio of the two LR’s that we use to compute the posterior probabilities is indeed equal to the OR
1 Like
Would you please explain why we know that connecting p(T|nO) to p(T|O) = 0.375.
From Bayes’ perspective, if outcome is considered a test of treatment status:
LR(O) = p(O|T) / p(O|C) & LR(nO) =1- p(O|T) / 1- p(O|C) [1]
Rearranging and simplifying
LR(O)/LR(nO) = OR(O)
In this example trial LR(O) is the RR=0.5 and LR(nO) is 0.8/0.6 = 1.33
and 0.5/1.33 = 0.375 which is indeed the OR
While the LR connects an unconditional odds to a posterior conditional odds via Bayes’ rule, the ratio of LR’s (odds ratio) connects two conditional odds and thus
Od(T|nO) × OR(O) = Od(T|O)
Given that algebraically LR(O)/LR(nO) = LR(T)/LR(nT)
Then OR(O) = OR(T) = 0.375
and thus Od(O|C) × OR(O) = Od(O|T) [2]
Applying expression [2] to the new baseline risk using the same OR
(0.8/0.2) × 0.375 = Od(O|T) = 1.5 and p(O|T) = 1.5/2.5 = 0.6
The ratio of LR’s (aka the OR) thus connects the control group risk to the treatment group risk (or the no-outcome group risk to the outcome group risk depending on what is viewed as test and what is viewed as outcome) in line with Bayes’ rule.
That is what I meant by connecting p(T|nO) to p(T|O) = 0.375
i.e. the OR of 0.375 is a likelihood ratio connecting two conditional odds and thus connects the two probabilities of interest
2 Likes
I thought a simple summary of the discussion thus far will be helpful. Based on the evaluation of the example trial at two baseline risk levels (p(O|C)=0.4 & p(O|C)=0.8) based on AER we reach the following conclusions:
a) The relative risk and its complement (RR(outcome) and RR(no-outcome) are also likelihood ratios LR+ & LR-) if outcome is deemed a test of the treatment status
b) The ratio of complementary RR’s or LR’s equals the odds ratio
c) If we follow Bayes’ rule then the OR as a ratio of two likelihood ratios is itself a likelihood ratio such that Od(O|C) × OR = Od(O|T)
d) For this trial, the OR is 0.375 so at baseline odds of 0.4/0.6, Od(O|T)=0.25 [p(O|T)=0.2] and at baseline odds of 0.8/0.2 , Od(O|T)=1.5 [p(O|T)=0.6]
e) If we agree with Bayes’ rule then we must agree that the RR is 0.5 at baseline risk of 0.4 and 0.75 at baseline risk of 0.8
f) If we switch this around and hold the RR constant and compute the OR, we must agree that we are violating Bayes’ rule
3 Likes
OK. Thank you for your detailed explanation. Your objective is to estimate the interesting probability that a patient was on treatment or control conditional on an outcome. This is totally different to my objective and the purpose of my calculations, which is to predict an outcome conditional on disease severity with treatment or control. What I think of odds and likelihood ratios have to be transposed by you. You also regard all individuals in a trial as belonging to a single universal set to which you apply Bayes rule. You point out that this can lead to inconsistencies (e.g. when applying risk ratios). This is different to my approach of considering several universal sets, within which such inconsistencies cannot arise. I am interested in the relationships between these universal sets and the extent to which they can be modelled accurately using risk and odds ratios.
I regard randomisation as a process of creating two or more universal sets that represent counterfactual situations – the set of those treated and the set not treated or treated differently e.g. different dosages of a drug or placebo. Bayes rule is then applied within each universal set and comparisons are made between these universal sets in the form of risk ratios or odds ratios.
In order that risk ratios are collapsible between sets incorporating a degree of severity (and provide a coherent model), then the prior probability of the degree of severity must be the same in the control and treatment set and also the likelihood of the degree of severity conditional on the outcome must be the same in the control and treatment set.
In order that odds ratios are collapsible between sets incorporating a degree of severity (and provide a coherent model), then the likelihood of the degree of severity conditional on the outcome must be the same in the control and treatment set and also the likelihood of the degree of severity conditional on those without the outcome must be the same in the control and treatment set. This inevitably means that the prior probability of the degree of severity is different in the control and treatment set, suggesting interaction.
These conditions for risk ratio and odds ratio are rarely met precisely but often met approximately enough for the purposes of estimating probabilities for dichotomous variables (e.g. severe/not severe). They are also helpful in comparing clinical trials when there is variation in the baseline risk of an outcome in different trials. Risk ratios may work at very low outcome probabilities in RCTs but odds ratios appear to provide more consistent comparisons at low and high probabilities as would be expected theoretically. However, when it comes to modelling quantitative degrees of severity they may be too imprecise. This is the point that I was making in my post.
Figure 5 (just above) shows the distribution of albumin excretion rates in those with the outcome of nephropathy on treatment and placebo when logistic regression functions are fitted to the treatment and placebo data independently. They are not identical, which is a requirement for the odds ratio or risk ratio to be transportable over the full range of baseline risks due to disease severity. However, looking at the raw data after dichotomisation into those with an AER above 80mcg/min, the proportion with an AER over 80mcg/min in those with nephropathy on placebo was 20/30 = 0.67 and the proportion with an AER over 80mcg/min in those with nephropathy on treatment was 19/29 = 0.66. The proportion with an AER over 80mcg/min in those WITHOUT nephropathy on placebo was 42/166 = 0.25. The proportion with an AER over 80mcg/min in those WITHOUT nephropathy on treatment was 93/346 = 0.27.
Based on dichotomizing the data into those above and not above an AER of 80mcg/min, the closeness of 0.67 and 0.66 and 0.25 and 0.27 in the presence of stochastic variation in the observed proportions satisfies the criterion for the odds ratio being transportable from the placebo set to the treatment set. Furthermore the proportion with an AER over 80mcg/min in the placebo limb was 62/196 = 0.316 and the proportion with an AER over 80mcg/in in the treatment limb was identical at 112/355 = 0.315 (which is what one would expect after randomisation) suggesting that the risk ratio was also transportable from the placebo set to the treatment set.
The observed proportion with nephropathy in those with an AER over 80mcg/min in those on treatment was 19/111 = 0.171. If we were to assume transportable risk ratio from the placebo set to the treatment set it would be calculated as (20/62)(29/375)/(30/196) = 0.163 (an under-estimate for 0.171). If we were to assume a transportable odds ratio it would be ((20/62)/(1-20/62))(29/375)/(1-29/375)/((30/196)/(1-30/196))/(1+((20/62)/(1-20/62))(29/375)/(1-29/375)/((30/196)/(1-30/196)) = 0.181 (an over-estimate for 0.171).
This means that for this particular data set, both the risk ratio and odds ratio provide reasonable estimates of the probabilities of an outcome on treatment if we know the probability of the outcome on placebo when we deal with dichotomised AER of above and not above 80mcg/min. However, as shown in Figure 2 of my first post they are not transportable for a range of AER values representing degrees of disease severity.
1 Like
In this thread there was a discussion by @AndersHuitfeldt about the utility of the switch risk ratio and I argued that this does not solve the problem. This has now been elucidated in detail in this paper and brings some closure to this issue.
1 Like
Claiming “closure” to this discussion on the basis of a paper which does not even engage with my argument, is a curious rhetorical strategy.
At this stage, Suhail, I am going to ask you as a professional courtesy to please refrain from discussing my work publicly until you have understood the structure of the argument. Feel free to contact me by email if I can help clear up any confusion.
Other than that, I am going to tap out of this discussion for fear of saying something that I might regret.
Anders, we are not discussing your argument in this paper, we are discussing the proper interpretation of the switch risk ratio. If you feel that we have not brought closure to the interpretation of this ratio, please feel free to point out any potential flaws in our proofs as that is the purpose of this thread - scientific dialogue that expands our collective understanding of the subject.
Anders, we are not discussing your argument in this paper, we are discussing the proper interpretation of the switch risk ratio.
I am basically the only person who is actively arguing for the switch relative risk. You write a whole paper to refute it, without ever considering the argument I am using to support my case (and without citing my work)?
And how is this paper even about the switch relative risk? The only sentence in which you even mention it, is this one:
It also explains why the RR divided by the complementary or switch
RR equals the OR
This sentence is not even correct. If the RR is divided by the switch risk ratio and the risk among the exposed is lower than the baseline risk, you will get 1 and not the OR. This makes me seriously wonder whether you have at all understood what the switch risk ratio is; it is not the same thing as what we call the survival ratio (and which you call the complementary risk ratio)
The only thing this paper does, is repeat an incoherent and irrelevant argument that you have made repeatedly in this thread, about the connection between risk ratios and the likelihood ratios for a hypothetical diagnostic test. You have never explained why this connection is relevant to choice of effect measure.
Your paper makes this claim:
⇒ The risk ratio (relative risk) is a ratio of two risks that is interpreted as connecting the intervention conditional risks in a clinical trial.
⇒ It is demonstrated that the conventional interpretation of the risk ratio is in conflict with Bayes’ theorem.
Bayes theorem follows logically from the Kolmogorov Axioms. Earlier in this thread, I gave a very simple proof (if you can even call it that) that the risk ratio is indeed a ratio of probabilities. It looks like you have discovered an inconsistency in the Kolmogorov axioms. Congratulations, my dude! Get back to me when the mathematical community rewards you for this great achievement with a Fields medal.
Due to the publication of this paper, which happened without my knowledge or involvement, I have resigned my position on the editorial board of BMJ Evidence Based Medicine.
1 Like
As I said previously, this paper is not about your ideas regarding the switch RR - rather it is about the principle underpinning the concept of a complementary or switch RR. Thus we use the “complementary RR” or “switch RR” as synonyms.
If
RR(Y) = r1/r0
then the complementary or switch RR is given by
RR(not_Y) = (1-r1)/(1-r0)
This paper is about both of these RRs (not just the switch RR) and we have concluded that these are just likelihood ratios
Thus RR(Y) / RR(not_Y) = OR(Y)
It is easy to see that this is not in doubt at all by creating any 2x2 table and doing the math - if any RR is divided by its complement the result will always be the OR.
Dude. We have spent the last 300 posts in this thread arguing about an object called the switch risk ratio, an effect parameter which becomes the risk ratio when treatment reduces risk, and becomes the complementary risk ratio (survival ratio) when treatment increases risk.
You then write a paper where you arbitrary add “the switch risk ratio” as a new name to the complementary risk ratio, and claim that you have won the discussion about the switch risk ratio, which has been argued to closure. Before this paper, nobody has ever used the term “switch risk ratio” to refer to the complimentary risk ratio. The complementary risk ratio has absolutely no need for a new name.
I couldn’t find anything useful to quote from your post so a generic response. We use the term switch as a synonym for the complementary RR for a reason which must be obvious to most … epidemiology is full of the same thing so we are not setting a precedent
This paper (in my humble opinion) puts closure on anything that emanates from the RR or its complement that claims portability (across baseline risk) because both are likelihood ratios and as such their interpretation as ratios of risk instead of ratios of odds is naive - the maths speaks for itself
We are not here to win or lose - we need the best evidence to guide patient management and when useless concepts abound, shrouded by sophisticated justifications that ignore the maths, that bothers me.
Sincere question for the statistical establishment: Do you all see what is happening here? Can someone please go on record to clarify for the public that Suhail makes absolutely no sense?
In particular, it would be helpful if anyone could corroborate the following claims, which are not easily disputed:
(1) Suhail claimed to have conclusively won the discussion against me, based on a paper in which he defined the “switch risk ratio” to be equal to the complementary risk ratio
(2) This illustrates that Suhail has never even understood what we mean when we use the term “switch risk ratio”, and that he clearly has made no attempt to understand our argument
(3) Suhail has never tried to explain what an “interpretation” is, how an interpretation can be “correct” or “incorrect”, and why this matters to choice of effect measure
(4) Suhail has made absolutely no attempt to explain why the connection between RCTs and diagnostic tests matters to choice of effect measure in RCTs
It would also be helpful if someone could clarify to the public that my arguments, while simple, are structured as proper mathematics (i.e. it is clear what my assumptions are, and how the conclusions follow from the assumptions) whereas Suhail’s paper is an unstructured mess.
I worry a lot that, due to Suhail being a professor and me being a nobody, that he is able to use his academic authority to win by default in the court of public opinion. Now, I would be the first person to recognize that in a hypothetical world where I am wrong, I am acting like a crazy person. So if he wins in the court of public opinion, most people will conclude I am crazy. It is kind of important that people who see what is happening here come to my defense.
Suhail, would you be interested in borrowing a method of conflict resolution from the judicial system, ie. some sort of mock trial, a live streamed event where we could both give presentations to a jury of statisticians, and call witnesses to corroborate our claims?
1 Like
@s_doi’s paper is a valuable contribution to the literature focusing on something that he is interested in. But it is true that the paper does not discuss @AndersHuitfeldt’s switch RR or his related papers, which is a distinct line of investigation that should be continued independently. A practical solution to clear the air may be to issue an addendum removing the term “switch RR” (used only once in the manuscript) and use only the term “complementary RR” throughout.
5 Likes
I am also a “nobody”, and I am not a member of the “statistical establishment” (I’m not even a statistician). But I recommend that you tone down the language. I don’t know @s_doi personally (or his work), but I do know that it is inappropriate and unnecessary to state that he “does not have the required intelligence to take part in this discussion”. Even if that were true, it is not the way to get people to “come to your defense”. I’m exposing a personal bias of mine, but I tend to discount people who interact that rudely. Even if, say, @s_doi was to argue that the earth is flat, I would hesitate to side with someone being so antagonistic.
3 Likes
I think its best to retain the word switch as a synonym for complementary as in any case proponents of the so called switch RR also base this on the probability of the complement of the outcome event anyway and there should be no copyright on the English word switch - which implies that outcomes are being switched aka the complementary outcome
Thank you, but I am actually in favor of the earth being round 
In that case it would have been more appropriate to cite @AndersHuitfeldt’s work and directly engage with his arguments in your manuscript.
7 Likes
This was discussed by the author group after it was raised by the reviewer and editors but then we realized that since we are suggesting that the RR (or complementary RR) should be interpreted as ratios of odds then there is nothing to engage anymore with this argument as they become invalid under the constraints we set for the interpretation of the ratio.