Determine power to detect a misclassification event in rule-out pathways for myocardial infarction

[R_cubed]

@pmmbossuyt

@Frank: when recommending the likely outcomes of a testing strategy, it is inevitable to define action thresholds. If combined with fully informative reporting, this do not hinder progress, as other action thresholds can be explored post-hoc (depending on study design).

I’m having difficulty reconciling, in a rigorous statistical way, the above with the following insightful observation (also in the same post):

Blockquote
The clinical population in which the drug/test will be applied typically differs from the study group, both in RCT of tests and in test accuracy studies.

The participants in the RCTs will also differ from each other and are not sampled from any population, so arguments based on sample theory are out. But there is still valuable discriminant information in well done studies that can be useful.

Frank goes into detail in Chapter 18 of Biostatistcs for Biomedical Research on how data driven cut-points (aka decision thresholds) are doomed to not replicate. AFAICT, attempting to synthesize decisions instead of probabilities will add unnecessary statistical heterogeneity for any evidence synthesis of diagnostic studies, unless the signal is overwhelming. A study of threads by @llynn on the problems with 30+ years of sepis research would be valuable.

Mathematical behavior of the primary endpoint

I previously discussed critical care summation score of the APACHE class the discussion on CITRIS-ALI. I you missed the discussion I present some of the basics of APACHE class scoring in sections 18-20. I was in training when they were introduced. Prior to the 80s threshold thinking was widely discouraged as simplistic. Paulker embellished this approach with some goid concepts but the use of thresholds was too seductive. The growth of threshokd science (the use of guessed thresholds as gold s…

Statistics Applied Using Threshold Science Fails to Deliver Progression Toward Truth

A recent study addressing a very fundamental question of considerable importance to the emergency room physician demonstrates the failure of the statistics applied using “Threshold Science” to progress the science toward an answer. Here, the question of the utility of the band count dates back tot the turn of the century. This question is asked here. The fundamental controversy is . Are the presence of an elevated absolute band count and/or elevated % bands in blood predictive of excess a…

After much searching, I finally found this forest plot of 30 years of widely published sepsis studies. Credit also goes to Dr. Lawrence Lynn. This graphic really should be archived somewhere around here. A look shows nothing but heterogeneous control groups from the studies, centered around the null. The only improvement on the chart would be to list the studies in order of publication.

I concede that cut points for decision in practice are inevitable. But these are decisions based on cost/benefit (ie. utility) considerations, where values and assumptions should be explicit, with input from all stakeholders.

Utilities should not be smuggled in under the guise of “evidence synthesis”. This occurs when dichotomization of “objective” information is substituted for the raw information itself. That permits all sorts of gamesmanship that is well outside the domain of strict scientific assessment.

That task of determining decision thresholds is distinct from information collection and synthesis at the local study level or aggregate level, where mathematics and statistics can provide guidance.