Hello - thank you for your time. As part of a medical education pathway program at my univeristy, I will lecture my peers (medical students) on medical prediction. I have been studying this topic, although I am not expert. I have read “Clinicians’ Misunderstanding of Probabilities Makes Them Like Backwards Probabilities Such As Sensitivity, Specificity, and Type I Error” (http://www.fharrell.com/post/backwards-probs/). It seems that this is concerning and should be fixed, and I might have an opportunity to fix it, in a tiny way. Over 5-some years, I will lecture 500 students. Some of the things that I hope to cover, mostly from the mentioned blog post, deviate from what’s emphasized on board exams, but I think they’re too important to ignore (I might appreciate some reinforcement, if available, since I might feel somewhat unorthodox up there saying some of these things). If you would be amenable, I would also be interested to hear about anything else YOU might want MDs-in-training to know about medical prediction so they can better interact with and critically evaluate the tools that you build. Thank you again.
-Probability, conditional probability
-Decision theory (classification v prediction)
Careful of tools that embed decision thresholds, they might disempower patients and physicians
Ergo, for prediction, pay attention to calibration in place of metrics requiring thresholds
-Briefly, how are coefficients estimated
They depend on data — is the patient you’re treating similar to the data used to develop the risk score?
-Uncertainty of coefficient estimates, predictions
What happens with small sample sizes? What if papers don’t report this?
-Benefits of risk scores
Done right, they augment decisions for one patient with data from many others
Continually updating, more granular tools on the horizon
-derive “AI” model from linear regression
Important to show students, I think, that AI of today is not really “intelligent”
nonlinear methods can capture interactions, but rarely needed
-black box vs. explainable