Responder Analysis: Loser x 4

After much personal study, I ended up asking myself that very question, and coming to the conclusion that nonparametrics are to be preferred, if you don’t want to go the Bayesian route.
There have been Monte Carlo studies going back to the 60’s documenting the sometimes large power advantages of rank tests to parametric counterparts in all cases except for a small (5%) loss of efficiency under strict normality, and about a 14% loss with tails thinner than a normal distribution.

Shlomo Sawilowski and many of his students have published similar results in recent times. An interesting historical POV is in the essay devoted to his colleague R Clifford Blair, who came to similar conclusions after comparing the textbook recommendations to his own Fortran simulations. (Pages 5 to 15 are most relevant). Both worked in the educational assessment realm, IIRC.

The emphasis in the textbooks on normal theory, suggests such a strong prior belief in the validity of parametric assumptions, that they should be incorporated into the model formally, and use a Bayesian analysis. But if one does not want to go the Bayesian route, ordinal regression seems to be the way to go.

In terms of using parametric techniques on Likert scales, there is a huge amount of question begging literature attempting to justify the practice, along with improper appeals to the central limit theorem. Proponents basically assume what needs to be proved – that there is a finite variance to the scale. It is truly amazing to me that the practice not only has gone on for decades, but continues.

Update: I was able to find a draft of the paper linked to – it should be required reading for anyone involved in data analysis.

2 Likes