I am part of a team medical students and doctors working on validating a crowding score for the ED department in hospital in the south of the England.
We are trying to validate the NEDOCS and ICMED score, neither of which have previously been fully validated in the UK. Our ‘gold-standard’ outcome of crowding is clinician opinion of crowding, sampled via a survey.
Recent research has identified a problem with validating crowding measures is the large degree of temporal correlation between short-spaced sampling times which means the observations are not independent, as assumed by many statistical methods. https://emj.bmj.com/content/emermed/33/5/307.full.pdf
We were planning on using ROC curves, as has often been done in crowding research, where the scores would be the diagnostic test and opinion of crowding would be the true ‘diseased’ state. I am struggling to understand whether ROC curves would be affected by the temporal correlation, whether the calculation of the Area Under the Curve assumes independence of observations. We sampled the data every two hours, but we are thinking about using samples every 6 hours to space the observations to account for this temporal correlation and as suggested by Boyle et al.
i worked on a slightly similar proj ie validating a tool to predict the no. of nurses needed, but maybe we sampled once a night. how often does ‘crowding’ occur? one event a day? you might simply amalgamate events and consider it a single crowding event… but i haven’t read the literature you point to