# Random sampling versus random allocation/randomization- implications for p-value interpretation

**URL:** <https://discourse.datamethods.org/t/random-sampling-versus-random-allocation-randomization-implications-for-p-value-interpretation/6177>\
**Category:** general\
**Created:** [November 6, 2022, 9:30pm UTC](https://discourse.datamethods.org/t/random-sampling-versus-random-allocation-randomization-implications-for-p-value-interpretation/6177 "2022-11-06T21:30:48Z")\
**Posts on this page:** 1\
**Showing post:** 35

<div class="post-metadata">

**Author:** ![R\_cubed](https://discourse.datamethods.org/user_avatar/discourse.datamethods.org/r_cubed/32/1518_2.png) [@R\_cubed](https://discourse.datamethods.org/u/R_cubed)\
**Post date:** [November 13, 2022, 8:12am UTC](https://discourse.datamethods.org/t/random-sampling-versus-random-allocation-randomization-implications-for-p-value-interpretation/6177/35 "2022-11-13T08:12:50Z")

</div>

> [@ChristopherTong](#):
>
> This would also completely defuse the more aggressive statements made by the permutation test people, such as Berger et al’s claim that “We conclude that the frequent preference for parametric analyses over exact analyses is without merit.”

I fail to see how that follows from what Sander wrote. The quote from Ernst in the Permutation Methods paper (who was also a co-author of the initial paper I linked to) described the problem: parametric assumptions that require large sample properties to hold for our finite, often very small sample, but we don’t know how quickly we converge to the limit.

Philip Good in his book **Permutation, Parametric, and Boostrap Tests of Hypotheses** (3rd ed, p. 153-154) gave a real world example (categorical data examined via Chi-square statistics) where the permutation p values and the large sample approximations differed by a factor of 10! The permutation test detected differences that asymptotic approximations did not.

I think it is unfortunate permutation _methods_ are invariably perceived as _tests_; interval estimates and entire compatibility distributions (more commonly called “confidence” distributions) can be created from them. In the case of RCTs they substitute an unverifiable assumption with a design based proposition that can be taken as correct by construction (if you accept the research report as described).

Related article that @ESMD might appreciate:  
[https://escholarship.org/uc/item/6hb3k0nz](https://escholarship.org/uc/item/6hb3k0nz)

---

_[View the full topic](https://discourse.datamethods.org/t/random-sampling-versus-random-allocation-randomization-implications-for-p-value-interpretation/6177)._