Review of Trial at the Center of the Firestorm Over Early Connection of Patients to Ventilators during the Pandemic

llynn · July 6, 2023, 7:46pm

Presently there is a disturbing and growing outcry in social media that physicians willfully placed patients with COVID pneumonia on invasive mechanical ventilators, negligently contributing to their death. This misguided narrative is hard to understand and, of course, completely false.

However, where did this bizarre idea come from? It appears that a plurality of social media influencers have advanced this retrospectively derived narrative, citing the approach, early in the pandemic, to proceed with early mechanical ventilation, which was later changed.

Of course, the narrative is absurd because, whenever possible, protocols and physicians actions are based on the evidence. However what if that evidence is unknowingly wrong? What if the consensus is unknowingly wrong? This is the danger of consensus. If it is wrong this magnifies the error, potentially worldwide but provides a false sense of security to stay the course despite failure.

Recall that Langmuir in 1953 described “Pathological Science” as derived from an early unrecognized error (upstream) in measurement unknowing causing false laboratory results. In contrast “Pathological Consensus” is where the error in measurement of Pathological Science is expanded by promulgation (and mandated for research), potentially worldwide, by consensus, so the false results are generated and applied worldwide, not just in the laboratory. So the greatest risk associated with a trial is that flawed results will be considered actionable leading to adverse outcomes on a worldwide scale. This is exactly what Pathological Consensus can cause.

Recall Pathological Consensus is the strange 20th century “science” (which is still the standard in critical care today) where guessed consensus sets of thresholds replace discovery and are the gold standard measurement for RCT (until the set of thresholds is changed every decade or so). .It is a domain in which critical care syndrome science operates and where variable SETS of different diseases with disparate average treatment effects are captured by these guessed sets of thresholds (i,e, measurements or criteria) causing non reproducibility of the RCT…

However, how does this relate to the issue at hand, specifically, early invasive mechanical ventilation in COVID pneumonia?

Consider this seemingly robust 3 year study (albeit observational) from 2016 which concluded a significant increased rate of failure occurred when patients with “ARDS” (and low P/F ratio), were initially treated with non-invasive ventilation (NIV) rather that invasive mechanical ventilation.(IMV). When COVID pneumonia emerged in 2020 and met the broad 2012 Berlin consensus measurements for “ARDS” this article provided a perceived “evidence basis” to proceed with early invasive mechanical ventilation This was later abandoned due to high death rates.

Now let us examine this trial. First we can’t fault the authors or the physicians who trusted the trial, because It has all the hallmarks of standard ubiquitous and indeed mandated critical care research to use measurements derived from “Pathological Consensus”: Remember the hallmark of Pathological Consensus is the use of guessed threshold sets as measurements for RCT and other trials. See link: What is a fake measurement tool and how are they used in RCT

When we examine the methods and conclusion of this trial we see all the standard characteristics of 20th century “Syndrome Science”.

The measurements are a guessed set of thresholds from 2012 (the Berlin Criteria)
A WIDE range of different diseases are captured by the consensus measurements
The captured diseases causing the “ARDS” were pulmonary in 58.8% & extrapulmonary in 32.9%
The syndrome (ARDS) is a cognitive bucket guessed in 1967 comprised of many disease with diverse pathophysiology including viral pneumonia.
The authors generalize their conclusions to “ARDS”, and therefore generalized the results to, all the diseases captured by the measurements of ARDS as is standard with “Syndrome Science”.

The authors conclude that a high rate of NIV failure occurred in patients with a “low baseline Pao2/Fio2 ratio” and “ARDS” so invasive mechanical ventilation will likely be required.

When the COVID pandemic developed (many having a low P/F ratio) this provided standard Syndrome Science level “evidence” to proceed with early intubation and invasive mechanical ventilation which has now appeared to be a less than optimal approach in at least a portion of the cases. This is the action taken up by the self serving social media provocateurs in their ill-conceived, follower deriving, rants against physicians.

However, here again we see the danger of using measurements for trials (RCT or otherwise) which first capture a range of diseases and, second, generalizes those results to any single disease (and all diseases) captured, especially a disease that was not included in the trial and worse, did not even exist at the time of the trial. However this is the hallmark of Syndrome Science, the standard science of critical care today…

How could that happen? How could invasive early mechanical ventilation be considered “evidence based” if no patient who actually had the disease under care was in the trial? Under 20th century “Syndrome Science” generalization of the results to diseases that were not in the trial is actually considered correct.

The fundamental axiom of the present critical care research standard (20th century “Syndrome Science”) is that the disease does not matter. The conclusions of a study applied to a guessed “syndrome” are applicable to all diseases which meet the guessed threshold set measurements of the syndrome. Indeed, the results of the trial even apply to those diseases which were not included in the trial but meet the measurements of that syndrome, and that includes a disease which did not exist at the time of the trial but retrospectively meets the measurements of that syndrome.

This is why the results of a trial of “ARDS” were generalized to a disease (COVID Pneumonia) which did not even exist a the time the measurements for ARDS were guessed (or the trial itself was performed). This last feature (applying conclusions to a disease which did not exist) is how one absolutely knows that Syndrome Science IS NOT valid science and must be soon discarded.

Remember this graph from the previous post. The End of the "Syndrome" in Critical Care

However, non-reproducibility of the results is not the primary risk. The primary risk is that the results will be considered actionable for all diseases which meet the guessed threshold set of measurements. This is what happened in the spring of 2020.

Its easy to see this in retrospect now but at the time, early in the pandemic, no good physician was going to sit by while the patients appeared that they would certainly die of this novel disease without ventilator support. Also, as indicated, there were other very valid reasons why physicians felt that early intubation and invasive mechanical ventilation was indicated. This of course was considered by virtually all considering the extant evidence to be the best care and was delivered by caring physicians risking their lives.

In critical care, all a physician can do is diligently strive to apply the best care, whether that turns out to be the optimal care can only be defined by the passing of time, sometimes the required time is measured in years or decades.

Even so, 40+ years of Syndrome Science of ARDS has proven to be a failure. Just say No to Pathological Consensus.

References:

**Origin of “Syndrome Science” **
1975 Tom Petty describes his new guessed syndrome (ARDS) as including viral pneumonia:

Rebuttal of Tom Petty’s guess in 1975 (we should have listed to this)

45 years later in 2020, conflation of Tom Petty’s guess (ARDS) and severe COVID (viral) pneumonia

A post pandemic “plea for honesty” as it relates to “Syndrome Science”

Global consequences of 40+ yrs of “Syndrome Science” & “Pathologic Consensus” on world health.

Discussion of “ARDS” as a “Sociological Construct” for trialists. All statisticians should understand this is the essence of “Pathological Consensus”. Fake measurements render useless statistics. Just say NO!

llynn · July 7, 2023, 4:57am

If you have followed at least a few of my posts you might know that I have for years (and before the pandemic) called for an end to RCT of 3 synthetic syndromes derived from “Pathological Consensus”. These are the “sleep apnea syndrome”, “acute respiratory distress syndrome” (ARDS), and the “sepsis syndrome”. This has been a one man, 20+ year long, campaign to end the cargo cult research of 20th century guessed syndrome science (after I discovered its pathological consensus nature in the 1990s). .

Earlier I showed that the fluidic Apnea Hypopnea index (AHI), the guessed set of thresholds as measurements for sleep apnea RCT, (the first Synthetic Syndrome I described) has now been accepted by the thought leaders to be inadequate (to say the least).

Then, in the above posts I have shown that leaders are accepting that guessed thresholds like Berlin based ARDS (the second Synthetic Syndrome) are simply artefactual constructs.

Finally, in the attached link you can see a leading sepsis researcher say "‘sepsis studies’ are outdated" They are abandoning RCT for the “sepsis syndrome” (the third synthetic syndrome) after 35 years of failure. This tweet is from the father of SOFA score, the guessed basis for Sepsis 3, (a set of guessed consensus thresholds used to measure and define sepsis for RCT)…

Its a great summer. We are witnessing the beginning of the end of an era of RCT using “Pathological Consensus” and of decades of worthless non reproducible RCT of the three guessed Synthetic Syndromes. Pathological Consensus based RCT has been stain on the clinical perception of RCT in critical care because (as I predicted) the non-reproducibility has been unrelenting for over 3 decades . This has produced skepticism of RCT in the clinical ranks and on twitter that not even the promulgation of the “statistical myths” could dissuade because the results spoke for themselves.

With tens of thousands of visits to the threads promulgating these antidogmatic points on this forum it is likely that this forum had much to do with the welcome collapse of this worthless research which will greatly benefit public health…

Please don’t contribute your statistical math talents to the prolongation of this welcome and long overdue passing.

I have had help (most of it silent) but the political expedience has waned and everyone should feel free to say what you believe. Thanks to all.

ESMD · July 8, 2023, 9:44pm

Hi Lawrence

Thanks for the interesting links. They really got me thinking about how medicine is both an art and a science. At the beginning of our careers, we are focused more on the scientific side of the profession. The longer we practice, the more we come to appreciate the art.

When we start practising, we obsess over learning what guidelines tell us we should be doing. But as the years go by, we gradually internalize the incredible complexity of human physiology and behaviour. We come to appreciate how impossible it is to pigeon-hole patients into neat categories and how important it is to treat each one as an individual. Guidelines might tell me that my patient should be getting 6 different medications for his 3 chronic medical conditions and that he is due for 4 different screening tests. But if he’s chronically depressed over the loss of his wife or can’t afford to feed himself, all the other imperatives go out the window. At that moment, he doesn’t care one whit about what the “guidelines” say about his care and my insistence on prioritizing guidelines will be a fast way to lose his trust.

Some important human conditions/diseases are easy to define and recognize clinically and present fairly consistently from one patient to the next. In such situations, we might be able to identify a safe, affordable intervention that could improve outcomes for many afflicted people. In these scenarios, we should try to develop treatments whose intrinsic efficacy we can test in randomized controlled trials. If a trial shows meaningful efficacy, then we can apply it judiciously to our patients (provided their circumstances allow). And if we don’t have solid evidence that a therapy works, then guidelines should refrain, as much as possible, from making strong recommendations for its use (except perhaps in very exceptional circumstances).

Other human conditions, however, will simply be too heterogeneous, or changeable, or have too many “moving parts” for us to be able to define them with any degree of reliability. In these situations, maybe we need to accept that we will be limited in our ability to learn, through formal experimentation via RCTs, how to improve outcomes. Maybe this is when we need to acknowledge the value of deep clinical expertise.

In listening to Dr.Tobin speak, the “art” of managing critically ill patients shines through. He uses physiologic reasoning and decades of experience (i.e., likely including painful lessons learned through his own errors and those of his colleagues) to make clinical decisions about when to intubate and how to adjust a ventilator in response to a patient’s changing condition. Critical care, as a field, seems to be asking how it can learn whether individualized clinical decision making, based on deep physiologic knowledge derived from decades of expertise, is actually superior to a “protocolized” approach to patient care.

I would not expect that every ICU has staff with expertise as deep as that of Dr.Tobin. And maybe, on average, a less expert clinician (e.g., a resident working the overnight shift in the ICU) will have better patient outcomes if he/she is following a protocol than if he/she is making decisions based on suboptimal physiologic knowledge or bedside experience (?) The crux of the problem is that, in order to justify using a “protocol” to manage a medical condition, we usually need experimental evidence that the protocol results in better outcomes, on average, than an alternate approach. But here’s where the rubber meets the road. It seems like this type of evidence has been very hard to come by in the field of critical care (?) And your threads suggest that the reason why the evidence has been hard to come by is that critical care clinical presentations simply have too many moving parts/too much heterogeneity to lend themselves to formal study through experimentation.

At the end of the day, would it be so horrible/unreasonable if the field of critical care were to concede that protocolized approaches to managing certain critical care problems (like respiratory failure/support) are untenable? Maybe the field could find ways to facilitate quick access to clinicians with deep relevant expertise when managing these problems (?) Just as stroke neurologists train intensively to be able to rapidly differentiate acute ischemic stroke from stroke “mimics” and to identify patients in whom the risk/benefit equation for thrombolysis or clot retrieval is most likely to be favourable, maybe critical care systems could be designed so that clinical experts are available on short notice to assist with key clinical decisions like intubation and ventilator adjustments (?) For example, if there’s sufficient time to call an expert, maybe it’s best for the on-call resident to do so, rather than trying to follow a cookbook recipe developed through experiments involving conditions that have been suboptimally defined/validated (?)

These are just random thoughts. I’m in no way qualified to speculate about next steps for your very complex field and won’t be at all surprised if you tell me that the suggestions above are hopelessly naive. I expect that the demand for deep critical care expertise far outstrips its supply, as is true for statistical input into medical publications…

llynn · July 9, 2023, 3:58am

Great comments. I agree, the expert is better than the protocol and its great to hear experts like Tobin speak but as you point out, expert prevalence is too low. However in our effort to find and protocolize broad treatments to deal with limited expertise, these syndromes were conceived.

Whereas basic patient heterogeneity is ubiquitous, RCTs are often capable of overcoming that heterogenity through the power of randomization.

Clearly, though, “Syndrome Science” provides a unique, anomalous magnification of heterogeneity by combining many diseases with different pathophysiologies into a set designated as a syndrome, the mix of which changes with each new RCT. This is a second layer of markedly fluidic heterogeneity which is a bridge too far.

The good news is that syndrome science is a 20th century artifact which can easily be abandoned and large RCT can be performed on specific diseases rather than sydromes. Then disease specific not syndrome specific protocolization can be implemented. Tobin hints at this in that he says that expert clinical care ignores the “syndrome” and treats the underlying disease. The research must mimic this.

If we find common target pathophysiology then diseases may be combined but this is best as a function of discovery from bottom up, not as top down guessed combinations as in “syndrome science”.

Thanks again for your excellent thoughts.

llynn · July 10, 2023, 3:10pm

To show both sides of this critical and active debate in critical care science here is a defense of Syndrome Science.

https://doi.org/10.1164/rccm.202211-2032LE

karlamoPA · July 13, 2023, 4:55pm

“Presently there is a disturbing and growing outcry in social media that physicians willfully placed patients with COVID pneumonia on invasive mechanical ventilators, negligently contributing to their death.”

My bet: the social media outcry is, mainly, a form of denial about the significance of the disease … that it was physician malpractice that caused many of the 1,134,300 deaths in the US. (There was no need to mask, or to get vaccinated…)

llynn · July 14, 2023, 1:10am

Absolutely, there is a component of antiscience. It is hard to take.

However, there was a major error made. So, of course, now we have to investigate in the interest of assuring the integrity of the relevant science and the public health.

Since all of us made the error (worldwide) this suggests the error is upstream. It suggests the error relates to the fundamental science which defined the “evidence based” protocol.

One view is that COVID pneumonia simply exposed the error of using one-size-fits-all protocols for a consensus SET of different diseases.

This view holds that “syndrome science” is a 20th century pseudoscience which emerged in the 80s where a threshold set of measurements generated by “pathological consensus” defined a syndrome which was then studied as if it was a disease by RCT. The argument is that syndrome science is a form of pseudoscience which became the standard RCT methodology in critical care and that COVID simply exposed the truth.

If this is true, syndrome science needs to be reformed, and promptly. As I noted previously, it’s already been a great summer of reform but there is much more to learn and do. Entrenched dogma are very difficult to displace.

In fact this “plea for honesty” article was written because a new consensus definition for ARDS was promulgated and it appeared that the, now well known, pitfalls of syndrome science had been swept under the rug. This is what some of the public think we do, but this time the counter articles (like this “plea for honesty”) are being published. The science is engaging in introspection.

In any case the social media outcry is background noise. The important thing is for us to ignore the antiscience crowd. They are irrelevant to the goals of learning the lessons taught by our collective mistakes and to reforming the science in the interest of the health of the public.

I look forward to your thoughts and the thoughts of others.

As it relates to the public health, there are few topics in statistical math and trial methodology as instantly important as this one.

karlamoPA · July 14, 2023, 12:37pm

I believe that patient and caregivers would appreciate a lay language explanation of what is known & unknown about the syndrome/condition and how it’s treated (including level of evidence) … and related risk factors (age, fitness, other conditions).

A plain language consent for select, common, emergency care conditions, reviewed by a CIRB - standardized across centers and revised as practice changes.

It sounds daunting but turnaround for CIRB review of trials during my tenure took 2 weeks. It might include a brief primer on the rationale for RCTs and the limitations of applying the findings.

As you know, the public is largely clueless about clinical research methods. The history of bias and the role of RCTs should be taught to everyone in high school!

ESMD · July 14, 2023, 10:59pm

Lawrence

Thinking back to the early days of COVID, I wonder what proportion of all decisions to intubate were driven by a belief that “evidence” supported mechanical ventilation for patients meeting certain criteria, versus the myriad logistical/manpower constraints created by the pandemic (?) As we know, hospitals were a war-zone at that point.

If I were a physician facing a tsunami of patients arriving in my ER with respiratory distress caused by a highly infectious disease (that could, if I’m not careful, take many of my staff out of commission), many of whom I anticipate (based on experience) will end up deteriorating to the point of needing intubation, and if I’m way too short-staffed to ensure ongoing close expert clinical monitoring of each unintubated patient, what should I have done? At the beginning of the pandemic, the benefit of steroids was uncertain, so there really wasn’t much to offer patients other than supportive care. And in that particular context, with insufficient monitoring capacity, was it really so unreasonable for physicians to intubate more liberally than under “normal” circumstances? The alternative might have been to risk multiple patients suddenly crashing (or being found dead by a nurse) while being managed non-invasively…

Decisions to intubate are, clinically-speaking, extremely critical. Under the clinical care of someone like Dr.Tobin, with decades of experience making decisions around the need (or not) for mechanical ventilation, maybe some COVID patients could have avoided intubation (and perhaps fared better as a result) (?) Alas, we didn’t have 50 Dr.Tobins in each ER at the start of the pandemic, nor the ability to provide 1:1 nursing/close clinical monitoring that’s needed for patients on the verge of intubation.

Early in the pandemic, every acute care MD must have struggled to stay on top of the latest “guidelines” for management of COVID-related respiratory distress. To the extent that such guidelines were based on suboptimal evidence (e.g., evidence from previous ARDS trials that long predated COVID), it’s important to introspect as to whether that was a good decision. But to the extent that guidelines might have also reflected an attempt to compensate for unfortunate logistical and pragmatic realities of a mass critical illness scenario, maybe we need to cut physicians some slack (?)

Two questions: 1) if there had been NO evidence (good or bad) related to management of acute hypoxic respiratory failure available at the time the pandemic started, what proportion of intubations do you think (realistically) might have been avoided? 25%? 50%? 75%? and 2) to what extent would ER and critical care physicians agree about this statistic?

llynn · July 15, 2023, 2:33pm

I fully agree and as I said in my original post, there were many reasons to intubate early at that time. Perhaps the percentage of intubations would not have been different.

However, that does not alter the fact that the evidence was wrong and this is the point. Over 3 decades of ARDS research. Massive effort and public expense. But when the world needed good evidence based medicine most, the evidence was wrong. How could that happen?

So the focus of the effort going forward. Why was the evidence wrong?
Why were the statistics wrong?

It appears to me that Tobin asks for honesty not because of excess lives lost but because the ARDS “experts” were going about the process of ostensibly abandoning “Pathological Consensus” without the deep introspection and candor required.

Here are two quotes from the linked article below to which Tobin responded with his editorial comment in a “plea for honesty”.

"The purpose of this article is to propose a change in how the critical care community has historically defined syndromes such as ARDS, from an approach based on a “consensus of experts” to a “scientific system of categorization,” "

“We believe that it is time to abandon the concept of a single definition for all purposes. It may be useful to develop different definitions for use in randomized trials in which consent is obtainable or for use in less resourced settings…”

On the surface this appears to be progress but anyone who examines the statistical and trial methodology of syndrome science without the blinders of dogma can understand Tobin’s outrage. Where is the introspection and failure mode analysis?

It take much courage to write a paper like Tobin’s but these are desperate times. If the pandemic does not trigger open introspection of the WHY (mathematically) with a deep and open failure mode analysis by statisticians & trialists then there is no hope for real reform.

In my opinion, Tobins courageous hammer seeking to drag critical care syndrome pseudoscience out of the 1980s has nothing to do with excess deaths but rather is a call for real open review of why the syndrome science got it wrong.

If you read this linked article (from which the above and below quotes are derived) you might have less hope than the above quotes convey. This quote might be particularly disconcerting to math grounded statisticians.

“We propose that attempts to revise the definition of ARDS should apply the framework originally developed by psychologists and social scientists and used by other medical disciplines to generate and assess definitions of clinical syndromes that do not have gold standards.”

Further, this aspirational article is full of the language of “grand theory” which is not far removed from the vacuous rhetoric of pseudoscience and is completely devoid of statistical math.

Most importantly, there is no understanding (mathematically) of the “WHY”. Where are the statisticians among these authors aspiring to pivotal new trial methodology?

What is grand theory (devoid of math) to the hard math based science of critical care trials? It is potentially the hallmark of another emerging 30 plus years of pseudoscience.

Obviously now is the time for the statisticians to step in and help them for there is no doubt they now know that 35+years of “Pathological Consensus” and “Syndrome Science” has failed the but they apparently do not know or are not telling (mathematically) why it failed.

Clearly they need guidance of statisticians.

llynn · July 15, 2023, 3:18pm

Absolutely. This is what is needed. The process and debates for generating them would be so powerful.

If we look toward the future maternal/paternal medicine has to become subservient to the formalization of medical glasnost.

I wonder though how this excellent idea might be converted to action. Who would lead such efforts?

karlamoPA · July 15, 2023, 4:35pm

Perhaps the US department of HHS would fund a pilot program for select conditions if pushed by experienced patient advocates … (just my first thought (not necessarily a good one.). Such a program might already fall within its approved mission?

However, i could not find a patient advocate program at HHS similar to patient consulting programs that exist at FDA, NCCN, and NCI.

A nonprofit group might attempt this - starting with common serious conditions lacking a consensus on best practice.

llynn · July 15, 2023, 9:50pm

ARDS SYNDROME FETISHISMS POST COVID-19`

The proposed new method for determining a new definition for “ARDS” (Ref.1) comprises “grand theory” as described by Mills in 1959. Mills points out that, "With the label of grand theory comes a whole syndrome of unfavorable designations like “fetishism of the concept” (Ref 2)

Here the instant fetishism is of “ARDS”, a failed syndrome guessed by one man or, at most, a few persons in 1967 and advanced by pathological consensus under “syndrome science” for 56 years.

This is the previously cited article to which Tobin responded with his “plea for honesty” (Ref. 3) and it describes methods borrowed from sociology for defining a new definition of ARDS. This article comprises a desperate call to salvage the 1967 guessed construct by the application of more integrated subjectivity. No wonder Tobin requested honesty. Below are quotes from the article describing the method for generating a new definition of ARDS

“…many of what physicians call ‘syndromes’ would be called ‘hypothetical constructs’ by psychologists”. ARDS fits into this schema, … An enormous body of literature exists for developing tools to measure these complex constructs such as intelligence, racism, and quality of life.” (for references see original)

Defining ARDS by the methods used to define “intelligence, racism, and quality of life” is a desperate proposal. It is proposed because ARDS cannot be defined in objective terms. So this, on its face, is just another (more complex) top-down approach which still relies on subjective imaginations which may for decades remain biased by “ARDS syndrome fetishisms”. Indeed, this would comprise an endless playground for those with ARDS fetishisms to publish. In the alternative, we already have a categorization available based on each objective disease causing hypoxic respiratory failure so we do not need ARDS. In fact, it was research directed to the specific disease, COVID pneumonia, and not ARDS (whatever that is) which made the critical discoveries.

Here the fetishisms of the concept or “construct” as they call it, of 20th century ARDS, is so strong they go backward into the subjective world of psychiatry where few objective measurements are available despite the fact that objective measurements (eg PCR) are ubiquitous in critical care. Since an objective overarching definition for ARDS has been elusive for 56 years, they go from the potential study of the time series matrix of objective data relating to specific diseases backward into the subjective imaginations, to define a guessed construct that has already failed the public…

Reading Petty describing his idea in 1975, (Ref.4) he was clearly just wrong as women and men often are. There is no other way to say it. Stop the fetishism. ARDS was just a bad idea that became dogma in the 90s and whose time has past.

Perhaps the ARDS terminology can live on in the clinical world subservient in each instance to the disease specific condition but it is a far to subjective and fluidic concept for the application of RCT. This may be a reasonable compromise.

Seeking help from the sociology literature, while failing to learn the lesson of Mills in 1959, cannot salvage this failed 1967 idea as a syndrome for RCT.

.

f2harrell · July 16, 2023, 12:16pm

Perhaps one little analysis that would demonstrate the lack of meaning of ARDS would help, e.g., a chart showing the variety of prognoses that exist with respect to physiologic measurements, and show the existence of meaningful patient types that are not officially qualifying as ARDS. Something to visually convey what oversimplification causes.

llynn · July 16, 2023, 12:55pm

Yes. This would be very helpful. Here is an example of a slightly different approach exploring the “syndrome” of sepsis. Here they present graphically the mortality of the controls of 65 RCT. Sepsis syndrome RCT are now being abandoned for RCT of more narrowly defined conditions.

The difficulty with ARDS is is deep rooted state in the literature. I was taught about ARDS in the 80s. Textbooks have been written pursuant to the syndrome. Indeed the same is true of sepsis syndrome.

f2harrell · July 16, 2023, 2:41pm

This is helpful. It would have been good to show the same graph where one repeatedly samples from the same population to see how much natural pure statistical variation there is.

Another angle would be to show that an ARDS dichotomization results in artefacts around the boundaries.

davidcnorrismd · July 17, 2023, 9:45am

There is a marvelous literature on construct validity in psychology that yields tools which, applied critically, could help more sharply define criticisms of ARDS and other such constructs of concern to you. Already in 1955 you see Cronbach & Meehl discussing construct validity in a way that could be helpful to you:

Construct validity would be involved in answering such questions as: To what extent is this test of intelligence culture-free? Does this test of “interpretation of data” measure reading ability, quantitative reasoning, or response sets? How does a person with A in Strong Accountant, and B in Strong CPA, differ from a person who has these scores reversed?

This spirit of constant, creatively critical probing of constructs receives more explicit emphasis in Shadish, Cook & Campbell, where (as I recall — can’t find my copy right now) construct validation is described as an ongoing, never-ending process as opposed to something that is ever achieved. (Compare present progressive tense validating vs past participle validated!) The latter even advance a taxonomy of ‘threats to validity’ that includes threats to construct validity. This could provide a vocabulary for precisely identifying what is wrong with ‘ARDS’ itself, apart from the peccadilloes of the researchers who advanced it.

I see a recent article by Matthay & Glymour attempts to cast [what they call] “The Campbell Tradition” [of discussing construct validity] into DAGs. I find that effort unhelpful at first sight, but the tables they apparently reproduce from [2] give at least a taste of the vocabulary available. See especially their Table 3.

Cronbach LJ, Meehl PE. Construct validity in psychological tests. Psychological Bulletin. 1955;52(4):281-302. doi:10.1037/h0040957 [link]
Shadish WR, Cook TD, Campbell DT. Experimental and Quasi-Experimental Designs for Generalized Causal Inference. Houghton Mifflin; 2001.
Matthay EC, Glymour MM. A Graphical Catalog of Threats to Validity: Linking Social Science with Epidemiology. Epidemiology. 2020;31(3):376-384. doi:10.1097/EDE.0000000000001161

llynn · July 17, 2023, 9:23pm

Thanks David, as usual you cite great morning coffee reading.

Although I too did not particularly find the DAGS helpful, this Matthay & Glymour article is amazing as a succinct document which can be readily consumed by physicians and trialists. The article is also timely.in its characterization of the “threats to validity” as they apply to critical care syndromes when those syndromes (Sepsis ARDS, OSA) are perceived as they actually are (ie as constructs).

Reading this article with such “syndromes” in mind provides remarkable enlightenment and is an exercise critical care physicians and trialists should consider.

In fact, I might have even found the DAGS useful if I spent more time considering them but the narrative was sufficient to make the points.

I hope this article can provide a platform for enlightened connection between critical care “syndrome science” and epidemiology. Displacing these syndromes into that perspective salvages them for their historical & cognitive value while freeing up trialists and physicians to focus on those things for which the available measurements in critical care rise to the statistical level (to borrow phrasing from @R_cubed).

I actually think this could be the breakthrough compromise needed. I hope there will be additional discussion or criticisms of this here.

Reference link to the Matthay & Glymour article cited by @davidcnorrismd

llynn · July 21, 2023, 3:01pm

Question about this study which relates to this thread. (published this month).

The study concludes that prolonged treatment with high flow nasal cannula (HFNC) oxygen and delayed intubation (and mechanical ventilation) for COVID pneumonia is associated with greater mortality.

This would be a VERY important finding, and if true would indicate that early intubation and mechanical ventilation for COVID pneumonia may be better. This would be a finding in support of the traditional ARDS protocol which called for for early intubation and which was abandoned during the pandemic due to high mortality (as discussed in this thread above).

Is this valid research? What type of subgroup division related bias is present here (of any?).

Note they divide into subgroups.

“Patients were separated into two groups based on the clinical trajectory and respiratory support modality used. One group was successfully treated with HFNC alone (HFNC success) and the other group was treated first with HFNC then transitioned to IMV (HFNC failure). Mortality was ascertained based on discharge disposition (alive or dead).”

This doesn’t seem to quite fit with survivorship bias and seems to be “recovery failure bias” ie patients with recovery failure (who may have lower survival as a function of failure of early recovery) remain on HFNC and are intubated late.

Please advise.

f2harrell · July 21, 2023, 3:34pm

It does look like they didn’t respect intent-to-treat use-true-baseline-only principles.