# Language for communicating frequentist results about treatment effects

**URL:** <https://discourse.datamethods.org/t/language-for-communicating-frequentist-results-about-treatment-effects/934>\
**Category:** study interpretation\
**Tags:** writing, confidence-interval, journal, rct, p-value\
**Created:** [November 13, 2018, 5:41pm UTC](https://discourse.datamethods.org/t/language-for-communicating-frequentist-results-about-treatment-effects/934 "2018-11-13T17:41:34Z")\
**Posts on this page:** 1\
**Showing post:** 17

<div class="post-metadata">

**Author:** ![zad](https://discourse.datamethods.org/user_avatar/discourse.datamethods.org/zad/32/44_2.png) [@zad](https://discourse.datamethods.org/u/zad)\
**Post date:** [November 15, 2018, 7:07am UTC](https://discourse.datamethods.org/t/language-for-communicating-frequentist-results-about-treatment-effects/934/17 "2018-11-15T07:07:11Z")

</div>

I believe @Sander raises some important points, especially this,

> [@Sander](#):
>
> “Assuming the study’s sampling scheme, experimental design, **and analysis protocol** , the probability is 0.4 that another study would yield a test statistic for comparing two means that is **as or** more impressive that what we observed in our study, if treatment B had exactly the same true mean as treatment A **and all statistical modeling assumptions used to get p are correct or harmless**.”

The common way to mention p is to discuss the assumption of the null hypothesis being true, but few definitions mention that **every model assumption** used to calculate p must be correct, including assumptions about randomization (assignment, sampling), chance alone, database errors, etc.

and also this point,

> [@Sander](#):
>
> Finally, if not clear from the above, I disagree that P-values are wisely dispensed with in favor of confidence intervals

I don’t believe abandoning p values is really helpful. Sure they are easy to misinterpret, but that doesn’t mean we abandon them. Perhaps instead, we can encourage the following guidelines:

- thinking of them as continuous **measures of compatibility** between the data and the model used to compute them. Larger p = higher compatibility with the model, smaller p= less compatibility with the model

- converting them into S values to find how much information is embedded in the test statistic computed from the model, which supplies information against the test hypothesis

- calculate the p value for the alternative hypothesis, and the S value for that too

---

_[View the full topic](https://discourse.datamethods.org/t/language-for-communicating-frequentist-results-about-treatment-effects/934)._