Comparing pre/post variance by cluster

I am wondering if anyone can suggest a way forward with an analysis I am perplexed with.

CTAS is a 5-point ordinal (some could argue categorical) scale used in many emergency departments for establishing the priority by which patients should be assessed.The scale delineates 5 levels of acuity: level 1 (resuscitation), level 2 (emergent), level 3 (urgent), level 4 (less urgent) and level 5 (nonurgent).

We implemented an electronic algorithm (eCTAS) to help triage nurses assign this score. We want to see if there was a reduction in variability in CTAS scores after the implementation of eCTAS in 35 hospitals across the province. So I am interested in both within hospital variance and between hospital variance for 15 different presenting complaints (chest pain, fever, etc).

I think I need something that can assess the variance across the CTAS distribution for a given complaint, while accounting for clustering (hospital), pre and post eCTAS implementation. I am stuck wondering what statistic I could use for this.

Any suggestions?

1 Like

What is your concept of this variation? Are you trying to inquire into the entropy of eCTAS vs older CTAS? I would wonder if some triage nurses might subtly recalibrate a subjective grading system to how busy the emergency department is at the moment and what the distribution of acuity is at the moment, whereas an algorithm might not do this. Is that what you’re after? Do you have data that enable you to investigate these matters at the level of individual triage nurses, and to examine interactions with volume (and staffing levels)?

Hi David. This is administrative level data on >360,000 triage encounters pre and post implementation of eCTAS. The hypothesis is that eCTAS reduces the variability/spread of the distribution of scores on a 5 level scale for 15 different presenting complaints. Each hospital has to act as their own control. It has nothing to do with accuracy, more so variance, but the scale is ordinal (or arguably categorical) and the variance will be very small.

Hi, Shelly. This sounds like a quality improvement (QI) inititative. What bearing does your stated hypothesis have on quality of care? Also, why not treat individual triage nurses as their own controls?

We don’t have data for each individual triage nurse. Each hospital will act as their own control. We would like to know if eCTAS has reduced the distribution in triage scores for a given presenting complaint. For example, pre implementation, “chest pain cardiac features” might have had a distribution such as CTAS 1=8%; CTAS 2= 81%; CTAS 3=6%; CTAS 4=4%; CTAS 5=1%. After the implementation of eCTAS, perhaps the distribution was less variable: CTAS 1=10%; CTAS 2= 88%; CTAS 3=2%; CTAS 4=0%; CTAS 5=0%. I need to compare both within and between 35 hospitals.