Good questions Drew. Come to think of it I don’t like multiplicity corrections in general, and anything that derives from p-values. And is *false discovery rate* even the correct terminology? I’m forgetting my statistical history now—I can’t remember if it attempts to estimate the proportion of true non-null effects or the proportion of non-null *assertions*. At any rate, it’s not really a *rate* but is rather a proportion or probability.

*Regulator’s regret* is an interesting term that regulators for too long have assumed means type I error. But in fact, and apropos of the original posting above, it is really the condition of approving a drug that doesn’t work (there is also the opposite regret of missing a good drug). The probability of regulator’s regret is the probability the treatment has no effect or harmful effect, so it’s not type I error.

For large-scale problems my biggest concern with FDR is that it doesn’t actually work. It lulls researchers into a false sense of security, makes them miss real effects, and fails to recognize that the feature selection method being used has no chance of finding the “right” features.