Between Subjects Variance and Within Subjects Variance in ICC

[Sudhi_Upadhyaya]

Sorry this may be a naïve question. Generally ICC is considered an indicator of reproducibility. Higher ICC is interpreted as good reproducibility , lower ICC not so much.

ICC as such is a ratio of between subjects variances/(between + within subjects variance). So if higher ICC is preferred, that means the numerator, higher between subjects variances is a good signal.

In chemicals , especially those related to phthalates and parabens, how can higher between subjects variance indicate a high reproducibility. Higher variance in phenols and chemical concentration values between subjects should indicate poor reproducibility , is it not ? Please advise.

[tkaczye]

Hello Sudhi, indeed this is a fundamental concern with methods like ICC – a measurement in a heterogeneous population appears more reliable than the exact same measurement applied to a homogeneous population. I have essentially stopped using ICCs for this reason, and followed prof Harrell’s advice to rely on U-statistics. Related post:

Mixed vs Random Effects ICC

Thank you for this interesting discussion. We will also look to U statistics in our future papers, but previously have always relied on ICCs as the norm for measuring repeatability. I am often confused by the varying formulas and am glad for the above discussion and helpful article referenced above by Koo and Li to interpret McGraw&Wong’s structure better. Other authors have advocated that a minimal detectable change (MDC95) corresponds to the real world better. This is a bit odd since the MD…

[Sudhi_Upadhyaya]

@Mixed vs Random Effects ICC

o look to U statistics in our future papers, but previously have always relied on ICCs as the norm for measuring repeatability. I am often confused by the varying formulas and am glad for the above discussion and helpful article referenced above by Koo and Li to interpret McGraw&Wong’s structure better. Other authors have advocated that a minimal detectable change (MDC95) corresponds to the real world better. This is a bit odd since the MD…

Ah, I heard about it but I forgot , I will dig into this. Thanks.