# Quality of study on non-pharmaceutical interventions (NPI) and COVID-19?

**URL:** <https://discourse.datamethods.org/t/quality-of-study-on-non-pharmaceutical-interventions-npi-and-covid-19/3621>\
**Category:** journal club\
**Created:** [August 3, 2020, 9:11am UTC](https://discourse.datamethods.org/t/quality-of-study-on-non-pharmaceutical-interventions-npi-and-covid-19/3621 "2020-08-03T09:11:11Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Daniel](https://discourse.datamethods.org/user_avatar/discourse.datamethods.org/daniel/32/525_2.png) [@Daniel](https://discourse.datamethods.org/u/Daniel)\
**Post date:** [August 3, 2020, 9:11am UTC](https://discourse.datamethods.org/t/quality-of-study-on-non-pharmaceutical-interventions-npi-and-covid-19/3621/1 "2020-08-03T09:11:11Z")

</div>

There is a new pre-print that has caught some attention (in the German media): _Ranking the effectiveness of worldwide COVID-19 government interventions_

> **[Ranking the effectiveness of worldwide COVID-19 government interventions](https://www.medrxiv.org/content/10.1101/2020.07.06.20147199v1)**
>
> Non-pharmaceutical interventions (NPIs) to mitigate the spread of SARS-CoV-2 were often implemented under considerable uncertainty and a lack of scientific evidence. Assessing the effectiveness of the individual interventions is critical to inform...

They used some ML algorithms / methods to judge NPIs from different countries, and the conclusion is that school closure is the most effective one according to pandemic spread.

I’d like to hear some opinion from someone who is more familiar with these methods. My impression is that the Artificial Intelligence from the ML algorithm isn’t that intelligent at all… E.g., I find it suspicious that banning _small_ gathering are more effective than banning _mass_ gatherings.

Any comments are highly appreciated!

---

<div class="post-metadata">

**Author:** ![RGNewcombe](https://discourse.datamethods.org/letter_avatar_proxy/v4/letter/r/85f322/32.png) [@RGNewcombe](https://discourse.datamethods.org/u/RGNewcombe)\
**Post date:** [August 4, 2020, 4:01pm UTC](https://discourse.datamethods.org/t/quality-of-study-on-non-pharmaceutical-interventions-npi-and-covid-19/3621/2 "2020-08-04T16:01:52Z")

</div>

“I find it suspicious that banning _small_ gathering are more effective than banning _mass_ gatherings.” Literally - certainly yes. I guess what they have in mind is that permitting a few large gatherings mightn’t be as dangerous as innumerable small ones.

---

<div class="post-metadata">

**Author:** ![Bjoern](https://discourse.datamethods.org/user_avatar/discourse.datamethods.org/bjoern/32/741_2.png) [@Bjoern](https://discourse.datamethods.org/u/Bjoern)\
**Post date:** [August 4, 2020, 5:52pm UTC](https://discourse.datamethods.org/t/quality-of-study-on-non-pharmaceutical-interventions-npi-and-covid-19/3621/3 "2020-08-04T17:52:57Z")

</div>

Firstly, are there examples of places that have banned small gatherings, but not large ones?! If not, then what this is really, is “effect of banning small gatherings on top of having already banned large ones”.

Secondly, the modeling approach has some sensible elements, but I wonder how good their models would be for predicting what would happen without intervention and identifying causal effects. One obvious choice of model (see similar modeling efforts I just noted in the programme for StanCon 2020: [https://www.stancon.mc-stan.org/programm](https://www.stancon.mc-stan.org/programm) - there’s probably also a preprint for that somewhere) is a SIR model. The models they use (with the exception of the RNN) rely on the authors having engineered good features to capture the time dependent nature (their time shifted features may do this sensibly - although there’s other nasty problems that they probably struggle to capture like a lack/incomplete reporting etc.). The recurrent neural network (RNN) might learn quite complex features on its own (if this is really enough data to train it). They take a virtual twins approach to the RNN. but I’m not sure whether we truly know how well that works.

Thirdly, one probably really has to dig into this to get it. Take my impression with a grain of salt.

---

<div class="post-metadata">

**Author:** ![Daniel](https://discourse.datamethods.org/user_avatar/discourse.datamethods.org/daniel/32/525_2.png) [@Daniel](https://discourse.datamethods.org/u/Daniel)\
**Post date:** [August 5, 2020, 5:36pm UTC](https://discourse.datamethods.org/t/quality-of-study-on-non-pharmaceutical-interventions-npi-and-covid-19/3621/4 "2020-08-05T17:36:27Z")

</div>

Thanks for your comments so far. What I find curious is that from many empirical studies I have read, also from Germany, there seems to be quite some evidence that kids are less infectious and either have more asymptomatic courses or are affected / infected less often than adults / older people. And I also read studies that claimed that school closures is, compared to other NPI, less effective. So this study is completely contradictory to previous results.

I think it’s a very sensitive topic, which NPIs should be applied at which stage of the pandemic, and my impression is that school closure is associated with one of the most severe “collateral damages” (for kids and families) of all NPIs. That’s why I think that researchers need to be careful (in general) to produce “valid” results and conclusions.