There is a new pre-print that has caught some attention (in the German media): Ranking the effectiveness of worldwide COVID-19 government interventions
They used some ML algorithms / methods to judge NPIs from different countries, and the conclusion is that school closure is the most effective one according to pandemic spread.
I’d like to hear some opinion from someone who is more familiar with these methods. My impression is that the Artificial Intelligence from the ML algorithm isn’t that intelligent at all… E.g., I find it suspicious that banning small gathering are more effective than banning mass gatherings.
“I find it suspicious that banning small gathering are more effective than banning mass gatherings.” Literally - certainly yes. I guess what they have in mind is that permitting a few large gatherings mightn’t be as dangerous as innumerable small ones.
Firstly, are there examples of places that have banned small gatherings, but not large ones?! If not, then what this is really, is “effect of banning small gatherings on top of having already banned large ones”.
Secondly, the modeling approach has some sensible elements, but I wonder how good their models would be for predicting what would happen without intervention and identifying causal effects. One obvious choice of model (see similar modeling efforts I just noted in the programme for StanCon 2020: https://www.stancon.mc-stan.org/programm - there’s probably also a preprint for that somewhere) is a SIR model. The models they use (with the exception of the RNN) rely on the authors having engineered good features to capture the time dependent nature (their time shifted features may do this sensibly - although there’s other nasty problems that they probably struggle to capture like a lack/incomplete reporting etc.). The recurrent neural network (RNN) might learn quite complex features on its own (if this is really enough data to train it). They take a virtual twins approach to the RNN. but I’m not sure whether we truly know how well that works.
Thirdly, one probably really has to dig into this to get it. Take my impression with a grain of salt.
Thanks for your comments so far. What I find curious is that from many empirical studies I have read, also from Germany, there seems to be quite some evidence that kids are less infectious and either have more asymptomatic courses or are affected / infected less often than adults / older people. And I also read studies that claimed that school closures is, compared to other NPI, less effective. So this study is completely contradictory to previous results.
I think it’s a very sensitive topic, which NPIs should be applied at which stage of the pandemic, and my impression is that school closure is associated with one of the most severe “collateral damages” (for kids and families) of all NPIs. That’s why I think that researchers need to be careful (in general) to produce “valid” results and conclusions.