Most reasonably hypothesised effects cannot be exactly zero?

It is easy for those of us who have never had to grapple with the logical issues discussed in a mathematical analysis class, to get totally confused on this point. I am fortunate enough to have mentors with Ph.D level mathematics education to help me get over the hurdle.

These 2 papers also helped me understand the issues much better.


Abstractly, the cardinality of the reals is so vast, that the probability of picking any particular real number is zero. That also makes it difficult to discuss issues of strict equality of 2 real numbers, except in certain cases.

This is all second nature to mathematicians, but it is too easy for the rest of us to treat real numbers as rationals, when we should not.

Realistic nulls are bounds around 0. That bound is context-sensitive. If you look at how equivalence testing is done, you must do two one sided tests (TOST). In this scenario, the null is formally expressed as a bound.

Good explanation of TOST (commercial site):

In the more common scenario, point nulls are used as approximations of what a skeptic might say the true effect might be (added after Prof. Harrell’s response). If the effect we are looking for is reasonably large, we should be able to detect it in “small” sample studies. How “small” or “large” depends upon what sample size we can collect.

For a realistic look at a field where the point null should be 0, look up the work done in parapsychology by Jessica Utts. IIRC, she was former president of the American Statistical Association not too long ago.

https://www.ics.uci.edu/~jutts/

Specifically, her article for Statistical Science on Replication and Meta analysis in Parapsychology is interesting: (PDF in link).

Either there is a small element of truth that effects studied by parapsychology exist, something wrong with our statistical test, our experiments, or our understanding of the assumptions.