It seems to be very common these days to see study results dismissed because the effect size is below a clinically important difference. I’m very skeptical about the relevance and use of this value in interpreting study results for a number of reasons. First is the way in which the MCID seems to be derived. It is my understanding that the tool for which you want to derive a MCID is compared to an anchor question. But in the cases I’ve seen the anchor question itself is not validated and even on face value it would not appear to detect the minimum difference even if it can detect a meaningful difference. Then there is the application of that value to compare cohorts. Taking a typical interventional study where one group is treated and the other isn’t, or gets a placebo. If the treatment has some effect then the results from the treated group are likely to be made up of those for whom the treatment worked, those who improved despite the treatment and those who did not. The magnitude of the difference between groups will thus also be diminished by those who did not respond to the treatment. While the treatment may have been very successful for a meaningful percentage of participants, the magnitude could be dismissed as being below the MCID.
Alternatively, to think of this in a different way, if we repeatedly run methodologically rigorous trials which consistently show a significant difference albeit one that is below the MCID, what is it that we are finding?
Surely the MCID is only useful when applied to individuals within a trial to see if the treatment has been successful for them (as a secondary analysis perhaps)? That is assuming the MCID can be successfully derived.
In my mind the more important value is the minimum difference a measuring tool can detect, but perhaps more so in the design of the trial rather than interpretation of the outcome.
Is there something I am missing or misunderstanding about this issue?
MCID cannot be applied to individual patients unless doing a crossover study. MCID as usually used refers to the treatment effect in terms of things like the difference in means between groups (ideally, covariate adjusted). When this difference is much below the MCID we do worry that the treatment is not effective enough. Though usually assessed with p-values this would all be better assessing using a Bayesian posterior probability of treatment benefit > MCID and a second probability for benefit > MCID/2.
We need to assess the treatment effect against costs, adverse events, comparable treatments etc to decide its worth. But given my previous argument that the mean difference between groups is a mix of those receiving some benefit and those receiving no benefit I don’t understand the value of the MCID. Perhaps as a secondary analysis we can compare the MCID to the change seen only in those who improved?
I tried to explain that there are problems with the way you look at individual patients in a non-crossover study. MCID applies to what we need it to apply to.
I didn’t read @micah as trying to identify individual responders of different degrees as much as just trying to capture that your probability of having some desirable value is higher on treatment than control? For example maybe the mean difference between two groups does not meet MCID but if you fit a PO model you’d find that eg the treatment has a higher proportion of individuals with better scores than control at any given threshold? I suppose you could even go a little further and fit a PO model (or a distributional model) and make decisions based on the difference in probability that Y is greater than some threshold of interest?
I think it’s strange when there’s some absolute threshold on a scale where people can agree that even a 2-5% change would be meaningful (eg because patients find no additional utility when scores are reduced below x) but then a mean difference which implies/supports a risk difference of 2-5% at that threshold could be crudely dismissed as not meaningful?
I agree with you that the idea MCID is scale dependent is ludicrous. What is the minimum that would be clinically important depends on what you have to do to get it. You sometimes see an MCID for a given scale as e.g. 3.1 points. This suggests, absurdly, that a simple, safe, inexpensive intervention that led to an improvement of 3 points would be “clinically irrelevant” whereas an invasive, toxic and expensive intervention that led to an improvement of 3.2 point would be worth using.
Additionally an anchor question with a different number of response levels will lead to a different MCID and thus different conclusions on clinical relevance…