Dear Dr. Harrell,
Thank you very much for your fast reply.
-I completely agree with your comment about statistical significance. I might miss-expressed myself, as I am not “looking for” a p-value as much as knowing if I can claim that there is a non-trivial difference between the summer/ winter differences. Yet, if that would come together with a p-value, that would be convenient for possible publication.
- I see that the correct approach would be computing a confidence interval for the double difference. In fact, that is what I have read about confidence intervals of means’ differences. However, I am not sure how to conduct such a calculation in this case.
-To get help with the previous, I should probably provide more context. I apologize for not having done it in the first instance and I hope to make it better now (please, correct me if I missunderstood which information I should provide).
my dependent variable is the score in a psychometric scale which provides a continuous score between 0 an 100. I have used the qcomhd function for R to calculate the differences between two groups at specified quantiles (in this case, the quartiles) of the obtained scores. Therefore, I obtained a bootstrap estimate of the value of each quartile in each group and also an estimate of the groups’ difference at each quartile with its 95%CI. As mentioned in my original post, I evaluated my sample in two different moments (summer/ winter) and I would like to provide information about the following questions
A) Differences between groups at each quartile in summer or winter. This is “solved” because that is what qcomhd calculates.
B) To draw valid conclusions about whether the between group differences for a quantile (e.g. the median) are similar/ different to those observed at other quantiles. I (probably uncorrectly) assumed that I could do that by checking whether or not the confidence intervals of the median contain the point estimate of other quantile (e.g. Q1). That is, are differences at the median similar or different than those observed at Q1?
C) Obtain similar conclusions when comparing the same quantile in the summer/ winter measurements. For instance, is the between-groups difference at the medians similar in the summer than in the winter?
I apologize for the extension of this reply and for any error or confusing expression it may contain.