Rmsb: Relative Explained Variation of Clustering Variable

Hello everyone, I have some reviewer comments back on a paper where we used rmsb to assess some predictors of survival for patients on ECMO due to COVID-19. One aspect of the analysis that the reviewers (and my co-authors) seem to like was the inclusion of the relative explained variation (REV) for each predictor/covariate, obtained using Anova(fit). One reviewer has asked that we provide the REV for the clustering variable we used, site of ECMO treatment. Is this something that rmsb or some workaround can provide?

Thanks,

Eric

2 Likes

for survival dont you want the GCE?: https://onlinelibrary.wiley.com/doi/10.1002/sim.7188

Thanks, Paul. Sorry I wasn’t more clear, we’re modeling a binary outcome (successful wean from ECMO and survival to discharge); it’s a logisitic model, not a time to event or survival analysis.

did you include site as random? if so, i think that case (random effects logistic regression) motivated the paper linked above, they refer to papers by Larsen et al. Eg, when they say: “An advantage to the [median odds ratio] is that it permits the analyst to present the between-cluster variation as a measure of association (i.e., an odds ratio) and thereby allows the comparison of the GCE with the fixed effect of the covariates in the model.”
but i’m not very familiar with these quantities ie REV, VPC, GCE etc, so best to wait for another to offer their thoughts …

The REV in rmsb is experimental and it doesn’t cover clusters. REV is equates posterior covariances to covariances of \hat{\beta} in a frequentist model and uses Wald \chi^2 to guage relative explained variation. I suggest supplementing that with a Stan leave-out-one cross-validated log-likelihood measure. Wish I had that worked out.

2 Likes

Thank you, Frank, for the quick reply and for rmsb! I’m looking into the log-likelihood measure you mention; I assume I might find tools for what you’re referring to somewhere within the ‘loo’ package?

How likely would you say it might be for REV to move out of experimental status in the next few years?

Re: the clusters, I am thinking of going the route proposed by Ben Goodrich (similar in spirit to an intraclass correlation coefficient) here:

Yes, something loo-based. No timetable for REV.

Hi Dr Harrell,
What is the definition of Relative Explained Variation?
And what is the best laymans definition so that I can explain it to clinicians?

A general definition is the proportion of variance in Y that is explained by a subset of X. For example if a model has 5 predictors and achieves R^{2}=0.5 and 2 of the predictors have a semipartial R^2 of 0.1, their REV is \frac{1}{5}.

In rmsb I use a very rough analogy based on partial \chi^2 statistics and equating posterior imprecision with sampling variation.