Methodology - Mediation, Moderation, Regression, MANOVA?

Consider formulating your research question in terms of competing or alternative theories, rather than in the manner of a title of a paper. In a wonderful essay [1] that every psychologist should grapple with, Paul Meehl says:

I consider it unnecessary to persuade you that most so-called “theories” in the soft areas of psychology (clinical, counseling, social, personality, community, and school psychology) are scientifically unimpressive and technologically worthless

Perhaps the easiest way to convince yourself is by scanning the literature of soft psychology over the last 30 years and noticing what happens to theories. Most of them suffer the fate that General MacArthur ascribed to old generals—They never die, they just slowly fade away.

Contrast this with the ‘Strong Inference’ concept elaborated by Platt in his famous 1964 essay [2,3]. A key ingredient of strong inference is sharply formulated theories that admit of falsification. It seems premature to consider choice of statistical techniques before you have in hand at least two clearly formulated alternative theories.

Given how intimately tangled sleep must be with aspects of ADHD and its pharmacological treatment, I have to wonder whether the usual type of (acyclic) causal diagram would apply. Perhaps “messy” diagrams are more likely to be faithful to the realities?

  1. Meehl PE. Theoretical Risks and Tabular Asterisks: Sir Karl, Sir Ronald, and the Slow Progress of Soft Psychology. Journal of Consulting and Clinical Psychiatry. 1978;46:806-834. PDF
  2. Platt JR. Strong Inference: Certain systematic methods of scientific thinking may produce much more rapid progress than others. Science. 1964;146(3642):347-353. doi:10.1126/science.146.3642.347 PDF
  3. Fudge DS. Fifty years of J. R. Platt’s strong inference. Journal of Experimental Biology. 2014;217(8):1202-1204. doi:10.1242/jeb.104976 PDF
3 Likes