Within an RCT, I’m interested in the interaction of time (factor, 1,2,3) and group (control vs. treatment, 0,1) in a mixed model. Therefore, I fit a mixed model using R.
I struggle with the theoretical interpretation of my interactions versus the results using marginal means and am asking myself whether there is any sense in the former.
For example, the time*group interactions shows that:
time1:group1: estimate = 0.58, p=0.03
time2:group1: estimate = 0.33, p = 0.30
time3:group1: estimate = 0.30, p = 0.29
Using marginal means, the estimates remain the same, while the p-values do not, e.g.:
time1 (group0-group1): estimate = 0.58, p = 0.17
time2 (group0-group1): estimate = 0.33, p = 0.45
time3(group0-group1): estimate = 0.30, p = 0.48
My hypothesis of treatment effects at each timepoint is pre-specified. However, is there any sense in interpreting the model-derived interaction (i.e., in comparison to the reference groups) or is it usually better to interpret the marginal means? It also seems that the differences between marginal means and model-derived interaction is quite large, which may be another point of discussion?
Thanks a lot for any answers.
1 Like
I don’t know the answers to your questions, but I have two observations. First, did you draw the semivariogram to check that the mixed model assumptions about the correlation patterns match the actual correlation patterns in the longitudinal
data? Second, the first thing to do is to get chunk (multiple degree of freedom) tests for (1) time x treatment interaction and (2) overall treatment effect. The latter tests, with a perfect multiplicity adjustment, whether treatment has an effect at any time.
Are the times of data collection really regimented with no room for being off a few days? That is required when analyzing time as discrete as you did.
1 Like
Hi Frank, thanks a lot for your time and help. As usually, this is really much appreciated. 1) yes, I did the semivariogram check and also checked via model selection techniques (AIC/BIC, though I am aware that the focus is on the semivariogram and you referenced to Keselman et al. (1998) regarding AIC selection). 2) I now did the chunk test. I compared one model with the Intercept+time versus a model with the intercept only (using the anova function in R, mixed models are fit using nlme). I did not add any other covariates in the model. Is this the correct way to proceed? If so, the test slightly fails the significance threshold (p=0.054).
Regarding discrete time-points: Theoretically, there is a possibiltiy that participants complete assessments +/- 1-2 days around each timepoint. What would you recommend?
1 Like
I always prefer continuous-time correlation structure that goes along with continuous-time mean response profiles.
What is a ‘significance threshold’ and why use the word ‘failed’?
1 Like
Thanks a lot for the recommendation. And sorry for the imprecise speech regarding significances. Maybe that’s a rather dumb question, but regarding the interaction effect, my question would be how to interpret the time:group interaction terms, when I adjust for Baseline as a covariate (thus not having it in the outcome). Using dummy-coding, a time*group interaction with baseline as reference group for time seems more interpretable; with T2 as reference group, the interpretation changes and isn’t of much interest.
A recent (2022) paper reviewing Nat. neuroscience and Neuron states that there are many pitfalls and mistakes and “post-hoc pairwise comparisons present several shortcomings, like failure to acknowledge the interaction as a global effect, the need for multiple comparisons correction, the impossibility to test the absence of difference, and the use of a redundant statistical evaluation” (link). In my estimation, some of these arguments also apply for planned contrasts. E.g., when I look at the difference in estimated marginal means between group1 and group2 at timepoint 2, this also fails to acknowledge the interaction effect (i.e., simple difference instead of difference of differences). Maybe I’m too confused at the moment, but any help resolving this issue would be appreciated; thanks a lot.
I’m not getting what is complex about interpreting the time interaction effect.