I am working on validating a 1-year risk of an event in EMR data. The previously developed model (which is binary logistic) is already integrated in the EMR, which calculates 1-year risk of the event every time a patient has a visit (using info from that visit and previous 180 days). But a patient could have several visits in a year and each time the risk is calculated. So, what I have is a patient-visit data with likely different risk scores at different visits. I am basically censoring the patient-visit observation whenever next visit happens and a new predicted risk is added. How do I go about validating these one-year risks in this time-to-event type of setting with censoring? Do I:
-
Follow the external validation techniques for survival model (following for example, this paper: https://www.acpjournals.org/doi/10.7326/M22-0844). So, converting 1-year risk to risk at each follow-up time for each observation (assuming constant hazard), and estimating O/E, calibration slope/plot, Harrell’s c-statistic..; OR
-
Do I use inverse probability of censoring weights (IPCW) and apply the approaches for discrimination, calibration for binary prediction with the IPCW weights?
I am struggling to figure out what the right approach is in such situations.