Just saw this, and although, I prefer to avoid dichotomization of inputs, I thought the talk was interesting as Dr. Wolf seems to hint at the situation many statisticians are faced with: the clinician wants a cut point.
Interested to hear thoughts (especially critiques) on her simulations about which method to use and how to see if cutpoints might exist.
An analyst who agrees to search for something that does not exist is IMHO not well serving the collaborators. This invariably leads to no one being to replicate the cutpoint on a new sample. The other way to think about the futility of this exercise is do to a formal assessment of goodness of fit of the resulting piecewise flat relationship. It won’t.
I agree with you and have read your writings/expositions (as well as others such as Senn).
In her video she sets out the assumption that if a cutpoint exists, and then she asks how might we find it. I was curious if you had commentary on the methodology proposed for finding it, if one exists (I.e. if time were a predictor, where a true cut point might exist more so than another type of x-variable like blood pressure). In that case, what thoughts would you have on the methodology?
I haven’t watched the video yet, but I thought we already had procedures to answer this type of question if it is looked at from the more general POV of dose-response estimation or risk estimation. Cut points just reinforce poor intuition regarding decision making under uncertainty.
The decision to bet or not bet has vastly different consequences in no limit vs. limit poker. There isn’t a rule valid for all versions of the game, which seems to be the “holy grail” of these cut point search methods.
In some areas (ie. econometrics), there is only a binary x for some questions (ie. effect of a policy implementation), so this most closely matches your specific question. In that case, interrupted time series can be used.
I have challenged medical researchers for 20 years to demonstrate the existence of a cutpoint on a continuous variable. No one has met the challenge yet. To develop a method assuming something exists that never exists is not a good use of anyone’s time IMHO. And many algorithms for finding “optimal cutpoints” have been published over the last 20-30 years anyway. Forcing a cutpoint forces you to add more variables to the model to make up for the information lost, in addition to creating a demonstrable lack of fit.
Even the freezing of water doesn’t have a distinct temperature cutpoint. The freezing point depends on the presence of contaminants in the water.
Again, I agree with you. It does seem like the “if it did exist” discussion could lead people to end up using something that is inappropriate overall.
I agree. I am not a fan of cut points; I think they strip the decision maker of at least some responsibility to think critically and assess the utility function (even if it means simply presenting a risk estimate to a patient in a usable format). Different people would bet different wagers on the same risk estimate. This is well rooted in behavioral finance and economics.
This recent paper by @llynn is relevant to the issue of cut points being misleading:
From the actual paper
Blockquote
When a RCT is used to test the efficacy of a protocol applied to a syndrome comprised of a mix of diseases (or sub-phenotypes), the result only provides evidence of the average treatment effect on the mix under test as a whole, not whether the treatment used in the RCT will be beneficial or harmful for any particular disease or subphenotype within the mix.[29] In this regard, a problem posed by the construction of syndromes, which may not have been evident to those selecting the criteria, is that if the criteria are broad enough to include markedly different diseases, an RCT applied in the study of treatment of the syndrome may suffer from amplification of the HTEs. This is a fragile state, highly dependent on the mix of the diseases present. In 2020, the pandemic changed that mix of ARDS cases in a dramatic way and exposed the weakness of generalization of RCT results to disease populations which were not sufficiently represented in the RCT and worse, were not represented at all.
When you visit your GP, has he/she ever told you, your Cholesterol is high, or perfectly normal based on the fact that it is below or above some number (e.g. 200)? How about BP? How about creatinine level? Just stop being a hypocrite. Whether we statisticians like it or not, they do it all the time. So, our burden is to make sure to let them know how it affects practice. Wolf’s work is not saying it is okay to dichotomize. It is giving a rationale for when (and when not) it is kosher. Dichotomizing a predictor does not always lead to lack of efficiency in prediction.
That is not convincing. Give an example.