It will be hard to improve on posts by @Sander, whose many papers have improved my understanding of foundational statistical concepts. I submit the following Data Methods threads as worthy of reflection and follow up of the links and citations.
Setting aside the issue of p-hacking, a major source of confusion involves conflating Neyman’s (pre-experimental) \alpha (error rate of the procedure) with Fisher’s evidential use of discrepancy measures to summarize information from (randomized) experiments. Readers of research reports grant too much epistemic weight to the experimenter’s (local) \alpha level.
Combining the many things I’ve learned from a number of papers, I used frequentist tools and Bayesian reasoning when reviewing a meta-analysis on predicting ACL injuries.