As much as I can tell there are 5 poopular curves for Performance Metrics, and the only one that includes PPV includes Sensitivity as well.
In terms of prediction (and not detection) it would be more insightfull to see the tradeoff between PPV and Predicted Postive Conditional Rate, I wonder why it’s not the norm? What Am I missing?
“Positive” and things derived from it like PPV only work in the special case where the test has an all-or-nothing binary result.
Sorry for the late response, but the same is true for Lift right?
If we have a resource constraint and we do not worry about potential treatment harm, isn’t it reasonable to make a curve of ppv on the y axis and ppcr on the x axis?
PPV refers to the probability of disease when a diagnostic indicator is “positive”. Because it treats the diagnosis as positive or negative whereas most diagnostic tests provide continuous or multiple outputs, this is a very narrow focus. Lift curves are more general than that since they are based on continuous probabilities of disease no matter how you got there — whether by multiple moderately positive indicators, one very positive indicator, or advanced age.
I’m trying hard to understand this sentence but I still miss something.
Lift is just PPV divided by Prevalence and Lift Curve is Lift on the y axis and PPCR (predicted positive condition rate) on the x axis.
PPV on the y-axis and PPCR on the x-axis will have the exact same shape as a Lift Curve. I don’t see much difference besides that a Lift Curve refers to a Random Guess (the Prevalence) as a reference (which also might be misleading, especially if the target population is not well defined).
Decode “predictive value positive” (PPV). The word “positive” can only be used if two things are true: the test result is binary and there is no pre-test probability, i.e., there are no variables relevant to outcome probability other than the test. So PPV is almost never applicable.
So Lift is more applicable because of the added information donated by the Prevalence of the Target Population?
Because Lift also requires Dichotimization of the estimated probability which leads to a loss of valuable information.
Life requires no dichotomization and is more applicable because it works for multivariable (not just single binary variable) models.
How come Lift requires no dichotomization? Using ranking for prioritization is not some sort of dichotomization?
In order to use Lift one might use ranking of estimated probabilities and stop according to the relevant resource constraint. Why not doing the exact same thing with PPV?
It is more intuitive for me and the result in terms of “which model should I choose” will be identical.
Once again, PPV is not applicable to this situation in any way except for the most trivial of models. PPV = P(outcome | test +) which is not what you are analyzing.
Life requires no dichotomization. It’s used a lot in marketing. Sometimes the marketing group has $100k to spend and they will choose the N most likely customers such that the total cost is $100k. At another time they may have $50k and will chose the most likely purchasers M where M < N.
If I understand correctly one can use Lift the same way in healthcare but in the context of how an organization makes a decision when there is a resource constraint: Allocation of an important medicine that is scarce. Right?
Yes, and there are other uses, e.g., enrolling top high-risk patients in a randomized trial.
I should be more clear about different kinds of motivation:
For choosing between candidate models given a resource constraint and no treatment harm choosing the model with the highest Lift is the same thing as choosing the model with the highest PPV and the decision will always be identical.
If you don’t consider resource constraint as a type of dichotomization (maybe I have a language barrier) than I don’t see the difference in the context of decision making.
I’m sorry to have to repeat myself multiple times. PPV is not applicable here. Once again, it is P(outcome | positive binary test and given that no covariates are available).
Resource constraints are not binary thresholds because the constraints vary from day to day.
But you should be using predictive measures that have higher statistical information and statistical power: Statistical Thinking - Statistically Efficient Ways to Quantify Added Predictive Value of New Measurements