Neyman advised setting different \alpha levels for different experiments. His test also formally used a likelihood ratio, not a p-value.
Neyman, J. (1977). Frequentist probability and frequentist statistics. Synthese, 36(1), 97-131. PDF
The elements common to all the situations typified by situation 5, will
be: (1) a hypothesis Ht to be tested against an alternative Ha and (2) a
subjective appraisal of the relative importance of the two kinds of error,
This gets overshadowed by idea that 0.05 is some “objective” line of demarcation, which is total nonsense and irrational. He was concerned about rules for rational behavior not reasoning under uncertainty, as Fisher was.
Fisher’s closely related, but distinct procedure only specifies 1 hypothesis and a sample size. He noted that p-values could be combined to detect departures from the reference null, even if there was no clear signal in any single study.
He was more willing to consider that the mathematical assumptions that specify the data generation process may not be entirely correct, reducing the ability to detect a signal at the individual study level, but not eliminating the information contained in the study.
The following has a quote from Fisher on combined tests:
On information grounds alone, a 2 sided p value ignores the sign, so we can halve the reported p value to 0.03.
Based on the base 2 log transformation of p values and the assumption that the statistics from the reported studies came from a distribution N(0,1), we have 3 \times -log_2(0.03) \approx 15 bits of information against that hypothesis.
Alternatively, we could say 15 bits of information is lost by using the null reference model to compress the observed data. That is a huge amount of information loss. The Higgs Boson discovery in physics reported 22 bits of information against the reference null.
It is not correct to say any study that “fails to reject” contains “no evidence” (aka. information).