How to update a prognostic model of COVID-19 for the effect of vaccination

We have just put up uploaded a pre-print (https://www.medrxiv.org/content/10.1101/2021.07.19.21260759v1) that describes the derivation and validation of a prognostic model for Covid-19 severity in a cohort of undifferentiated adults at the time of diagnosis.

In Iceland, all individuals who were SARS-CoV-2-positive were prospectively enrolled into a telehealth service at a median of <24 hours from their positive test. We derived a prognostic model using data collected during the enrollment interview of 1,625 persons in the first-wave of infections and then later validated on the 3,131 individuals who were diagnosed during the second and third waves. We found (what we consider) excellent discrimination and validation. We also performed decision curve analysis and provided a case-study on how we suggest the model could (should?) be used.

Almost immediately after we finished our manuscript and uploaded to medrxiv, another wave of infections began in Iceland. The difference is that now, roughly 90% of Icelanders 16 years of age and older have been vaccinated [1]. As we describe in our paper, we will use the prognostic model to decide who to enroll into the telehealth service using a cut-off of predictive risk that weighs the risks omitting enrollment with the benefits of not overwhelming the telehealth service. We have provided a interactive calculator that is built upon our prognostic model for this purpose (Prognostic model for COVID-19). However, we would like to display a rough estimate of the calculated risks given that the individual is vaccinated.

Given that an unvaccinated individual has a predicted risk of hospitalization xunvaccinated and given that we find a publication of a randomized controlled trial that that describes the odds ratio of hospitalization of vaccinated SARS-CoV-2 infected individuals compared to unvaccinated SARS-CoV-2 infected individuals, ORvac/unvac, how would we approximate xvaccinated? Could anyone point me to papers or textbooks that describe this kind of adjustment. Is my understanding correct that odds ratios would be the only transportable effect size? Given that the calculator strongly warns that xvaccinated is approximated and provides a citation for the odds ratio, are there any obvious reasons why this would be a poor idea?

P.S. Any and all critique of the prognostic model, its presentation and the manuscript is very well appreciated. It is currently undergoing peer review but I am a strong believer in open review. We tried to be as transparent as possible and followed the TRIPOD guidelines. All statistical code along with all the code output is available at OSF | Development and validation of a prognostic model for COVID-19: a population-based cohort study in Iceland

[1] Statistical information vaccine

2 Likes

i quote from your preprint: “the aim of the study was to develop a prognostic model to predict the severity of COVID-19 at the time of diagnosis and determine risk factors for severe disease.”
this new review has many references, maybe something in it useful, although i’m not so sure: Review on COVID-19 diagnosis models based on machine learning and deep learning approaches “this paper provides a comprehensive review of the most recent DL and ML techniques for COVID-19 diagnosis.”

1 Like

Outside of the approach you are suggesting, is it perhaps possible to collect information similar to what you have used in your development and validation phase, but additionally collect information on vaccination status? I was thinking you might use the data from this ‘vaccinated wave’ to extend the model with an additional predictor. The Clinical Prediction Models book from Steyerberg has a nice chapter (Ch.20 in the 2nd edition) on updating prediction models and describes one situation that might be similar to yours (you want to keep the original predictors, but potentially evaluate/validate their performance in the new situation and extend your model using an additional predictor).

1 Like

We were hoping to use the model during this current wave of infections among the vaccinated as our resources are being overwhelmed. I do however think this is the only viable option.