Confusing results from Scottish study of real-world effectiveness of Pfizer and Oxford/ Astra Zeneca vaccines

A recent publication from a well-credentialed group looks at early results of a single dose of these two vaccines on hospitalisation with CV-19 in the whole population of Scotland using controlled cohort design (Effectiveness of First Dose of COVID-19 Vaccines Against Hospital Admissions in Scotland: National Prospective Cohort Study of 5.4 Million People by Eleftheria Vasileiou, Colin R. Simpson, Chris Robertson, Ting Shi, Steven Kerr, Utkarsh Agrawal, Ashley Akbari, Stuart Bedston, Jillian Beggs, Declan Bradley, Antony Chuter, Simon de Lusignan, Annemarie Docherty, David Ford, Richard Hobbs, Mark Joy, Srinivasa Vittal Katikireddi, James Marple, Colin McCowan, Dylan McGagh, Jim McMenamin, Emily Moore, Josephine-L.K Murray, Jiafeng Pan, Lewis Ritchie, Syed Ahmar Shah, Sarah Stock, Fatemeh Torabi, Ruby S. M. Tsang, Rachael Wood, Mark Woolhouse, Aziz Sheikh :: SSRN) .

The Oxford/Astra Zeneca vaccine was given later to an older age group than the Pfizer vaccine and I am concerned that this has differentially biased the adjustment models they used to measure effectiveness of both vaccines.

The authors used Poisson regression and Cox models and present their results as age adjusted, full adjusted, and full and inverse propensity adjusted hazard ratios. The HR estimates for single doses of both Pfizer and O/AZ vaccines show a benefit at 7-13 days (VE 38 and 70% respectively), which is unexpected given the negative result before day 14 seen in the Pfizer vaccine randomized trial (

In reporting a target trial emulation of Pfizer vaccine effectiveness (NEJM Feb 24) in the Israeli population Dagan et al (Ben Gurion, Michigan, Harvard universities) have suggested that such an early apparent benefit probably indicates residual confounding.

In the Scottish study the added effect of inverse propensity weighting on the modelled estimates during this early period post vaccination is minimal for the Pfizer vaccine, but substantially increased the apparent benefit of the O/AZ vaccine, which was administered later (shorter follow-up) to a much older age group.

I am a clinician, so way out of my comfort zone. I am concerned that the Scottish study a) is showing an implausible early benefit from both vaccines and b) the adjustment model seems to perform differently in recipients of the 2 vaccines. In the elderly recipient group for the O/AZ vaccine the inclusion of inverse propensity weighting substantially increases an implausibly early benefit of a single dose of the O/AZ vaccine (non-overlapping confidence intervals for estimates with and without IPW ). I realise this is a complex set of issues spanning several publications, but the findings are very important and have practical implications. I’d love to hear expert comments about the adjustment techniques used in the Scottish study.


My recent blog takes the example of three vaccine efficacy studies and shows how misleading the results can be if you don’t use the local concurrent control. That suggests that we should be very cautious interpreting observational studies. See A Very Standard Error.
Infection rates vary enormously over space and time and this means that as soon as tight local control is abandoned one can expect trouble.