@Stephen Senn provided this reference in a twitter discussion about RCT vs OBS studies.
So measurement has long been overlooked as a source of non-reproducibility in RCT. I have been intrigued by the limited focus on measurement quality and I believe this is due to a lack of understanding among trialists of the characteristics required. Measurements have not been the responsibility of statisticians rather they are generally derived from trialists. At least in the fields of critical care and sleep apnea, trialists have not developed guidelines or standards for measurements used in RCT. To begin to facilitate this process I would like to compile a list of the relevant papers which address the characteristics which are required to incorporate a measurement into an RCT.
To show the desperate need for guidelines and the search for articles I would like to show the scope of the problem “in the raw”: Seeing pathological consensus in the raw is much better than discussing the problem as extant, without being able to see how it happens.
Here I provide a rare opportunity to actually hear sleep apnea trialists discuss their guessed measurement for RCT (in this case, the AHI) which they still use and have mandated for research for 35+years. This candor may be due to the AHRQ, which failed to find any evidence that CPAP reduced chronic disease . .
The entire discussion is elucidating but all statisticians in med science should watch at least this 5 minute section of this remarkably candid ATS & European Respiratory Society Seminar to see the truth about the measurements trialist may present to you as a standard for a trial and for which they may ask you to apply statistical analysis
5 minutes: Scroll to 2200 & watch at least to 2730 of ATS & European Respiratory Society Seminar
http://ow.ly/pbaT50MJK66
The important thing to note is that despite the acknowledgement that the standard measurement AHI is highly variable with criteria guessed over a weekend, the AHI remains the standard measure for RCT and other sleep apnea research. .The same situation exists with critical care syndromes… .
So going forward the solution appears to be to prioritize the development of measurement guidelines. Please provide any references or existing guidelines. These also can address composite measurements.
Please forward