Most reasonably hypothesised effects cannot be exactly zero?

[R_cubed]

@f2harrell

I wouldn’t say that point nulls are used as approximations. They represent exactly a zero effect, and with a large enough sample size you can have power to detect any effect. Hence if the intervention is actually experienced by the subjects randomized to it in any way (biological, psychological, Hawthorne effect, etc.), it is logical to assume H_{0} is false and save time by not collecting data. As stated in Cohen’s 1994 [“The Earth Is Round; P < 0.05”]

The nil hypothesis is always false. Tukey (1991) wrote that “It is foolish to ask ‘Are the effects of A and B different?’ They are always different—for some decimal place”. Schmidt (1992) … reminded researchers that, given the fact that the nill hypothesis is always false, the rate of Type I errors is 0%, not 5%, and that only Type II errors can be made.

I agree with everything you said/quoted from a declarative point of view; in the state of nature, the null is always false.

But from a decision perspective, those real differences might not be actionable. Agents do experiments to see if there are better decision rules than the ones they currently follow.

If an agent’s decision remains unchanged in spite of knowledge of a small departure the simple, null model, is it unreasonable to say “the agent acts as if H_{0} is true”?

It seems reasonable to me. That is all Neyman-Pearson theory promises – that we won’t too often go wrong, given our ignorance.

But I can see it is easy to confuse the two points of view. I had them confused for longer than I would care to admit.

Your post on communicating frequentist results, as well as that thread was helpful in that I have a better idea on how to communicate to others who might not have studied this topic as deeply as participants here.