There is keen interest in public health in the use of incoming wastewater (i.e. pre-treatment sewage) for disease surveillance–particularly in predicting meaningful outcomes like rates of new hospital admissions. This is made technically feasible by the increasing sophistication and sensitivity of molecular methods to detect even tiny amounts of a pathogen’s RNA or DNA. I can certainly see the value in the concept: wastewater can be collected and assayed for pathogens relatively easily, and it does not depend on human behavior. For example, you can test sewage influent for SARS-CoV-2 RNA regularly, regardless of whether people in the community are getting tested.
Here is one example from my state, currently in preprint:
I’d be interested in any opinions about the methods. There’s a lot here, and I’m still working through the manuscript, but my initial, vague thoughts, possibly misguided, are:
It’s a lot of model-fitting. Over-fitting?
The researchers say they used stepwise methods for variable selection
I’d think there’d be much collinearity between some of the predictors. Effect on stability of coefficient estimates?
Caveat: Not an expert in COVID-19 or public health predictive models.Not a card-carrying statistician, so no motions to revoke my certifications will be considered. 30+ years experience in analytics & “decision support” in health care systems.
I see RNA prevalence in the wastewater … however measured, and I find the difference choices in methodology to be intriguing but waaaaay above my knowledge level … as a DIRECT MEASURE of disease prevalence in the community, and as LEADING INDICATOR of hospital admissions.
What value, in a practical sense, do the various demographic and regional “risk predictors” add to that? I would argue: very little.
The core requirements for a PREDICTIVE model of this type IMO are (1) directionally accurate (2) responsive to short-term changes and (3) available quickly enough to be useful “by decision-makers” for staffing and reponse planning.
In retrospective model-building looking back two years, all of the county and regional variables … test rates, immunization rates, etc … are readily available. However, in a production “real world use” situation, the entire suite of those variables may not be available in a timely fashion, every week. Or twice-weekly. Then what do you do?
I’d model only the relationship of RNA levels and hospital admissions. And get twice-weekly measurements from all the waste-water treatment facilities to tighten up the detection windows for changes in trends.
Big picture, I think the hope is to do exactly what you describe toward the end of your post: predict upcoming hospitalization burden using only surveillance of the RNA concentration in wastewater (WW). WW can be collected (easy) and then assayed for RNA (pretty easy) at any frequency desired (depending on budget). Other predictors, like proportion of tests positive, depend on the behavior of thousands or millions of people–getting a COVID-19 test. WW surveillance is touted as easier and cheaper; I’ll buy that.
A big operational question is my mind is using the predictions for “staffing and response planning.” Hospitals are stretched very thin. Most have oodles of open positions that they are recruiting furiously every day to fill. The idea that they could somehow quickly and temporarily ramp up staff or facilities because of a predicted uptick in COVID-19 admissions I think is mistaken. If they could do that, they’d be doing it already, regardless of the predictions of any model.
Perhaps more pertinent for this discussion board, I think there are also some interesting statistical issues. I’ve been communicating with the author about a few of them. Besides the three I alluded to in my initial post:
Three aliquots are taken from each incoming bottle of WW (one bottle per treatment plant, at intervals varying by geographic region; weekly in my neck of the woods). Each aliquot undergoes the PCR procedure. All three aliquots from a given bottle are assayed together–same plate, same technician, same pass through the machine. The result is three readings of SARS-CoV-2 RNA concentration and three readings of a fecal bacteriophage RNA concentration. This is somehow converted to a single measure of “intensity”: log(SARS-CoV-2)/log(phage) for that bottle. It remains unclear to me how exactly the three values are combined into one. They might be averaged, but perhaps with zeros excluded or handled differently. What are implications of using an average of N measurements of some quantity as a predictor to model something else? That average is itself an estimate of the “true” value, with some uncertainty around it, no? How is that uncertainty captured?
At any rate, the analyst receives the above Intensity measure. If the copy number is considered not quantifiable (> 0 but <= 5), the value of 3 is imputed. That’s troubling.
Minor technical note: any time multiple measurements are averaged and one of the components can be censored, it’s better not to sum them but to analyze the raw data, treating a subset of the raw observations at censored. The summing can be done on parameter estimates if needed.
Oh, not minor at all Frank, and I appreciate you pointing that out. There are a number of issues here that I am still trying to get my head around. In situations like this (common in bench science) would it be worth fitting a hierarchical model, with aliquot nested within sample? Or is that only if one is interested in the variability inherent in the assay?
With this particular article/study, I worry there is a lot going on, much of it loosely documented, between what the machine displays and the data that the analyst is given to model.
I think this is at least in part related to the notion of replication vs pseudoreplication? Seems like an important concept to understand, and I’m still working on that. I found this explanation interesting:
