Vaccination and Transmission: Overstated Impacts and Unexplored Variables – A Critical Review of Methodological Gaps and Data Limitations

by James Lyons-Weiler, PhD, Popular Rationalism, ©2024

(Sep. 23, 2024) — A preprint is available of a study that sought to estimate the impact of SARS-CoV-2 vaccination on virus transmission in the UK during 2021 by employing a Bayesian hierarchical model. Its primary focus was assessing how one, two, and three vaccination doses influenced the time-varying reproduction number (Rt), reflecting the virus’s transmissibility at different pandemic stages.

Though the study is among the first to analyze vaccine efficacy against transmission at the population level, several factors challenge its conclusions. The context of lockdowns, easing restrictions, and the Delta variant complicates interpretation. Moreover, the authors’ methodological choices, such as assuming constant vaccine efficacy over time and excluding key demographic variations, raise concerns about the robustness and applicability of their findings.

This critique focuses on these methodological shortcomings. The goal is to provide a clearer understanding of the study’s ability to inform public health policies while highlighting where improvements could enhance the accuracy and relevance of future research on vaccination and transmission dynamics.

Methodological Critique

The methodology used in this study is central to the validity of its conclusions. While applying a Bayesian hierarchical model is appropriate for estimating transmission effects across multiple regions, several critical methodological choices undermine the reliability of the results. Lack of formal model selection, constant vaccine efficacy assumptions, and omission of sociodemographic factors weaken the analysis.

Lack of Formal Model Selection

The study did not implement a formal model selection process, such as using the Akaike Information Criterion (AIC) or Bayesian Information Criterion (BIC), to compare competing models and ensure the best fit for the data. Modeling complex systems like viral transmission, where numerous variables influence outcomes, demands formal, objective, and transparent model selection. Without this step, the chosen model may overfit or underfit the data, leading to potentially misleading conclusions. Overfitting could make the vaccine appear more effective than it is by capturing noise rather than meaningful patterns, while underfitting may miss important transmission dynamics. Additionally, without comparing alternative models, the authors cannot be confident that their choice best represents the underlying epidemiological processes at play.

Assumption of Constant Vaccine Efficacy

Another major limitation is the assumption that vaccine efficacy remains constant over time. Evidence shows that immunity conferred by COVID-19 vaccines wanes, particularly against infection and transmission, as demonstrated in multiple studies. Ignoring this dynamic can significantly distort the results by overestimating vaccine effectiveness in the later stages of the study period. Incorporating a time-varying vaccine efficacy model, which accounts for waning immunity and booster doses, would have provided a more realistic understanding of the vaccine’s role in transmission reduction. The static model used in this study likely fails to capture important changes in vaccine effectiveness over time, especially when booster doses were rolled out or when immunity began to wane.

Sociodemographic and Geographic Variability

The study also lacks consideration for sociodemographic factors that influence transmission dynamics and vaccine uptake. Age, population density, income levels, and occupation-based exposure risks are crucial in determining transmission and vaccine efficacy. By failing to account for these factors, the study risks generalizing its results across diverse populations where vaccine uptake, exposure risk, and health outcomes vary widely. Although the model incorporates Lower Tier Local Authority (LTLA)-level data to capture regional variability, this level of aggregation does not reflect the full spectrum of localized demographic differences. For example, urban and rural areas may have vastly different transmission dynamics that are not captured at the LTLA level.

Bayesian Hierarchical Model Assumptions

The choice of a Bayesian hierarchical model requires careful specification of prior distributions, yet the study does not provide sufficient transparency about these priors or how they were chosen. The results of Bayesian models can be sensitive to the choice of priors, especially in cases where data is sparse or highly variable. If the priors are not well-calibrated or are too rigid, they can skew the results and lead to biased estimates of vaccine efficacy. More detail on how the priors were selected and whether sensitivity analyses were conducted to test their robustness would strengthen the study’s credibility.

---

Read the rest here.