This study evaluates the efficacy of the new BNT162b2 mRNA vaccine1 against Covid-19 in a nationwide mass vaccination environment. Estimated vaccine efficacy during the follow-up period starting 7 days after the second dose was 92% for documented infection, 94% for symptomatic Covid-19, 87% for hospitalization, and 92% for severe Covid-19. Estimated efficacy during days 14 to 20 (after one dose) and days 21 to 27 (gradual shift between the first and second vaccine doses) was 46% and 60% for documented infection, 57% and 66% for symptomatic Covid-19, 74% and 78% for hospitalization, 62% and 80% for severe Covid-19, and 72% and 84% for Covid-19-related deaths, respectively.
The first primary endpoint evaluated in the randomized trial of the BNT162b2 vaccine was symptomatically Covid-19. In both the randomized trial and in our study, the cumulative incidence of symptomatic Covid-19 in the vaccinated and non-vaccinated groups began to deviate around day 12 after the first dose.1 The estimated vaccine efficacy for symptomatic Covid-19 from day 7 after the second dose was 95% in the randomized trial, compared with 94% in our study. The estimated efficacy between the first dose and the second dose was 52% in the trial, compared to 29% in our study. This difference may reflect the high transfer rate in Israel during the study period,14 which affected both the vaccinated persons and the controls equally during the first 12 days after administration of the first dose. To eliminate this distortion, we estimated the first dose efficacy of the vaccine against Covid-19 for the period from days 14 to 20; the estimated effectiveness was 57%.
The estimated efficacy for documented infection during days 14 to 20 was 46% in our study. A relatively similar efficacy of 51% was reported by Chodick et al.,15 who evaluated a group from another health care organization in Israel and used a different study design that compared infection among vaccinated persons on days 13 to 24 after the first dose with infection during days 0 to 12.
In the randomized trial, the estimated vaccination of the vaccine for severe Covid-19 (89% during the entire study period) was based on only 10 cases. Our study recorded 229 cases of severe Covid-19 – 55 in the vaccinated group and 174 in the non-vaccinated group – resulting in an estimated efficacy of 62% for days 14 to 20 after the first dose, 80% for days 21 to 27, and 92% for 7 or more days after the second dose.
The large sample size in our study also enabled us to estimate the vaccine efficacy for specific subpopulations, which did not drive the random sample sufficiently to evaluate. In the trial, the estimated efficacy for Covid-19 among persons up to 55 years, older than 55 years and 65 years or older 7 days after the second dose was 94 to 96%. We were able to study more granular age groups, and we estimated that the vaccine efficacy was similar for adults 70 years or older and for younger age groups for the same period.
The randomized trial estimated the effectiveness of the vaccine for patients with one or more concomitant conditions according to the Charlson comorbidity index.16 and specifically for patients with obesity or hypertension. These measures do not provide clarity regarding efficacy in patients with multiple concomitant conditions. We estimated the efficacy of the vaccine in relation to the different numbers of related conditions and found indications that the efficacy may be slightly lower among persons with a greater number of concomitant conditions.
Two factors make the present study unique in evaluating the efficacy of the BNT162b2 vaccine in a practical application: first, a rare combination of rich medical background data, Covid-19 PCR test results (for the documented outcome of the infection), and the patient follows – data in both communities (for the symptomatic Covid-19 outcome) and inpatients (for all other outcomes) institutions – CHS maintained such an integrated database for more than half of the Israeli population, and maintained it daily for updated more than two decades; and second, the rapid rate and high uptake of Covid-19 vaccine in Israel and the high disease rates during the vaccination campaign. On the other hand, the rapid pace of the vaccination campaign contributed to the frequent censorship of data for corresponding unvaccinated controls, especially among those older than 60 years (often only a few days after the agreement) and the corresponding decrease in the mean-period of the study.
Concerns have been raised about the possible resistance of SARS-CoV-2 variants to Covid-19 vaccines17.18 and neutralizing antibodies.19.20 During the study period, an increasing proportion of the SARS-CoV-2 isolates in Israel – up to 80% in the days before data extraction – were of the B.1.1.7 variant.21 This study therefore estimates the average efficacy of the vaccine across multiple strains. Although we can not give a specific efficacy estimate for the B.1.1.7 variant, the plateau observed during later periods in the cumulative vaccination incidence curve indicates that the BNT162b2 vaccine is also effective for this variant, an observation consistent with previous reports showing conservative neutralizing antibody piters.22 The B.1.351 variant is estimated to be rare in Israel during data mining.23
As with any observational study, our study may be affected by residual confusion due to differences between vaccinated and unvaccinated controls, especially in terms of behavior seeking health. We therefore performed a strict match on a wide range of factors that can be expected to confuse the causal effect of the vaccine on the different outcomes. After the corresponding process, we found a constant agreement between the groups in the days just before day 12 after the first dose (the expected onset of the vaccine effect), thus serving as a ‘negative control’24 period (Figure 2S6 and Table S7). This similarity occurred despite a temporary increase in events under unvaccinated controls during the first days after the first dose of vaccine, probably due to the fact that persons who choose to be vaccinated on a particular day feel well during the vaccination. . The similarity between the study groups in concomitant conditions and known risk factors for severe Covid-19 (Table 1 and FIG. S2) provides further evidence of the interchangeability (ie absence of confusion). However, this strict matching process has the cost of not including about 34% of the vaccinated people who otherwise meet the admission requirements of the study in the final group. Limited correlation with age and gender would have been insufficient to eliminate the early confusion (Fig. S6).
We also excluded population groups with a large internal variability in the likelihood of vaccination or outcomes, such as health workers, people confined to the home for medical reasons, and residents of nursing homes, to avoid confusion. Although the chance of persons not being healthy enough to comply with the scheduled visits and vaccination plan was less likely to include the subjects, this did not exclude health workers.
To evaluate a possible bias of the selection that may result from informative censorship, whereby vaccinated vaccines feel good around the vaccination period, we conducted a sensitivity analysis in which they were kept in the unvaccinated group for a period, each outcome (Fig. S7 and Table S5). This analysis showed results similar to those of the main analysis, indicating that such bias was small in our analysis.
Finally, the onset date of symptoms was not available for analysis. Instead, for infection outcomes, the date is set to the date of swab collection for the first positive PCR test. Since there would probably be a time gap between the onset of symptoms and the accumulation of blemishes, the observed deviation of the cumulative prevalence rates for the infection outcomes between the vaccinated persons and unvaccinated controls could be slightly delayed. In parallel, the efficacy of the vaccine can be underestimated at each timing, as the estimate is actually a narrower window on which the vaccine may be active. Because SARS-CoV-2 PCR testing is very accessible in Israel and can be done within hours without reference, we estimate the potential time gap and the underestimation of vaccine effectiveness is therefore small. When interpreting the efficacy estimates for more severe outcomes, longer median gaps should be kept in mind (Fig. S3): 1 day for hospitalization, 5 days for severe Covid-19, and 11 days for Covid-19 death.
This study estimates the high efficacy of the BNT162b2 vaccine for the prevention of symptomatic Covid-19 in an uncontrolled environment, similar to the efficacy of the vaccine reported in the randomized trial. Our study also suggests that effectiveness is high for the more serious outcomes: hospitalization, serious illness, and death. Furthermore, the estimated benefit increases as time passes. These results reinforce the expectation that newly approved vaccines could mitigate the profound global consequences of the Covid-19 pandemic.