# Effect sizes for power and evidence for effectiveness

**URL:** <https://discourse.datamethods.org/t/effect-sizes-for-power-and-evidence-for-effectiveness/17118>\
**Category:** design\
**Tags:** effect-measures, mcid\
**Created:** [July 5, 2024, 2:43pm UTC](https://discourse.datamethods.org/t/effect-sizes-for-power-and-evidence-for-effectiveness/17118 "2024-07-05T14:43:04Z")\
**Posts on this page:** 1\
**Showing post:** 6

<div class="post-metadata">

**Author:** ![R\_cubed](https://discourse.datamethods.org/user_avatar/discourse.datamethods.org/r_cubed/32/1518_2.png) [@R\_cubed](https://discourse.datamethods.org/u/R_cubed)\
**Post date:** [July 7, 2024, 7:18pm UTC](https://discourse.datamethods.org/t/effect-sizes-for-power-and-evidence-for-effectiveness/17118/6 "2024-07-07T19:18:23Z")

</div>

> [@Pavlos\_Msaouel](#):
>
> Many such outcomes would more appropriately be treated as ordinal but [empirical strategies](https://pubmed.ncbi.nlm.nih.gov/2818162/) can be used to convert them to interval.

Before accepting the claims of psychometricians at face value, I recommend a careful study of the scholarship of Joel Michell, a psychologist and critic of psychometric methodology as “pathological science”

**Michell, J. (2008).** Is Psychometrics Pathological Science? _Measurement: Interdisciplinary Research and Perspectives_, _6_(1–2), 7–24. [https://doi.org/10.1080/15366360802035489](https://doi.org/10.1080/15366360802035489)

Pretty much all of his papers and responses to psychometric proponents are worth reading. Here is a response in Michell’s defense by mathematical psychologist Steven Barrett:

**Barrett, P. (2008).** The Consequence of Sustaining a Pathology: Scientific Stagnation—a Commentary on the Target Article “Is Psychometrics a Pathological Science?” by Joel Michell. _Measurement_, _6_, 78-123. [(PDF)](https://www.pbarrett.net/publications/Measurement_Scientific_Stagnation_commentary_Barrett_2008.pdf)

> As Michell notes in his target article, the response from psychologists and psychometricians alike has been largely silence. From my own attempts to relay his thesis in papers, conferences, and academic seminars, I have been met with a mixture of disbelief, disinterest, anger, accusations of intellectual nihilism, ridicule, and generally a steadfast refusal to seriously question the facts. And, make no mistake, Michell deals with facts about measurement. The only coherent responses I have read where the author made a passable attempt at really engaging with Michell’s work was in the book Measurement: Theory and Practice by Hand (2004) a mathematical statistician and machine learning expert…

> There are real-world consequences to his thesis, and we are living through them right now. The most important of these is that while psychometricians have advanced their thinking and technical sophistication in leaps and bounds over the past 40 years or so, the practical consequences have been almost nonexistent except in the domain of educational testing and various examination scenarios.

Outside of Barrett’s citations, the only other attempt I’ve seen to deal with this problem of needing interval/ratio quantities on inherently ordinal data is the work of Thomas Saaty – a mathematician who worked on multi-objective decision problems, and developed the analytic hierarchy process:

**Saaty, T. L. (2008).** Relative Measurement and Its Generalization in Decision Making Why Pairwise Comparisons are Central in Mathematics for the Measurement of Intangible Factors The Analytic Hierarchy/Network Process. _Estadística_, _102_(2), 251-318.

More sources can be found in my old thread on treating ordinal as metric. Rasch is more sophisticated, but still has problems, that Michell addresses in other papers. From a stats POV, I don’t think these scores have the statistical properties the inventors think they do outside of very specific contexts.

> [@Preprint: Analysis of Likert Type Data Using Metric Methods](https://discourse.datamethods.org/t/preprint-analysis-of-likert-type-data-using-metric-methods/5536):
>
> Dennis Boos and Judy Chen (2022). Analysis of Likert-Type Data Using Metric Methods. new I edited this to bring the main points earlier, and the decision theoretic justification later in the post. Introduction The authors defend the common practice of using parametric models on ordinal data on the basis of pragmatics (ie.“easier” to for researchers understand), and perceived robustness in that parametric methods on empirical or simulated data sets are not sufficiently inaccurate that skepti…

🆕 For a reasonable example of AHP in a health context, the following paper describes the creation of a Quality of Life questionaire

**Prusak, A., & Sikora, T. (2017).** THE APPLICATION OF THE ANALYTIC HIERARCHY PROCESS IN ASSESSING THE QUALITY OF LIFE. _Center for Quality_. [(PDF)](http://cqm.rs/2017/cd1/pdf/papers/focus_1/16.pdf)

---

_[View the full topic](https://discourse.datamethods.org/t/effect-sizes-for-power-and-evidence-for-effectiveness/17118)._