Is there a rigorous but non-technical discussion of the differences for descriptive statistics and point estimation purposes (especially in RCT settings)? It seems to me that there is a giant gap in the perception of research results by stakeholders (possibly only in the post-Soviet space). A new cargo cult of evidence-based medicine is being created among Russian clinicians, not without the help of “instagram-statisticians”. In particular, when discussing the interpretation of RCT results, one can find a strong interest in the observed average outcomes in samples, but not in the ATE estimates. Related to this is the testing of hypotheses on the “equality of sample averages”. I understand that I can refer to the discussion of the Neyman–Rubin model, estimands, etc., but they are usually difficult enough for the perception of the target audience given their background.
I tried to find relevant discussions in Stigler’s books, The Art of Statistics, Dicing with Death, in Professor Harrell’s, Stephen Senn’s, Darren Dahly’s blogposts, but without success.
Welcome to datamethods Maksim. This is a huge topic. I would not be so quick to mention ATE because that may not apply to any particular patient. It is important to condition on what you know at baseline that explains outcome heterogeneity within treatment. Because of the need to explain easily explainable outcome variation within one treatment group, to get the model correct as well as to provide estimates that apply to individual patients, it is necessary to covariate adjust the estimates. Because of that, descriptive statistics are far less useful than assumed by most readers of papers reporting clinical trial results. Descriptive statistics are invariably unadjusted, thus ignoring outcome heterogeneity due to factors not being reported on.
Thank you very much, Professor. I have already tried to start a discussion about the covariates adjustments referencing your and Darren Dahly’s posts but to no avail. To describe the depth of the problem, Russian-speaking statistical bloggers recommend choosing descriptive statistics only after testing the normality hypothesis (in my opinion, they believe that they are testing the hypothesis of the normality of the sample distribution). It is especially scary that some of them studied at annual programs (including biostatistics and clinical trials) at Harvard and Yale and have great popularity and authority among clinicians.
is senn’s maxim: “clinical trials are comparative not representative” useful here? ie do away with the implicit assumption they insist upon?