Authors: Noam Barda, M.D., Noa Dagan, M.D. https://orcid.org/0000-0001-8811-7825, Yatir Ben-Shlomo, B.Sc., Eldad Kepten, Ph.D., Jacob Waxman, M.D., Reut Ohana, M.Sc., Miguel A. Hernán, M.D. https://orcid.org/0000-0003-1619-8456, Marc Lipsitch, D.Phil. https://orcid.org/0000-0003-1504-9213, Isaac Kohane, M.D., Doron Netzer, M.D., Ben Y. Reis, Ph.D., and Ran D. Balicer, M.D.
Published August 25, 2021
N Engl J Med 2021;385:1078-1090
DOI: 10.1056/NEJMoa2110475
Abstract
Background
Preapproval trials showed that messenger RNA (mRNA)–based vaccines against severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) had a good safety profile, yet these trials were subject to size and patient-mix limitations. An evaluation of the safety of the BNT162b2 mRNA vaccine with respect to a broad range of potential adverse events is needed.
Methods
We used data from the largest health care organization in Israel to evaluate the safety of the BNT162b2 mRNA vaccine. For each potential adverse event, in a population of persons with no previous diagnosis of that event, we individually matched vaccinated persons to unvaccinated persons according to sociodemographic and clinical variables. Risk ratios and risk differences at 42 days after vaccination were derived with the use of the Kaplan–Meier estimator. To place these results in context, we performed a similar analysis involving SARS-CoV-2–infected persons matched to uninfected persons. The same adverse events were studied in the vaccination and SARS-CoV-2 infection analyses.
Results
In the vaccination analysis, the vaccinated and control groups each included a mean of 884,828 persons. Vaccination was most strongly associated with an elevated risk of myocarditis (risk ratio, 3.24; 95% confidence interval [CI], 1.55 to 12.44; risk difference, 2.7 events per 100,000 persons; 95% CI, 1.0 to 4.6), lymphadenopathy (risk ratio, 2.43; 95% CI, 2.05 to 2.78; risk difference, 78.4 events per 100,000 persons; 95% CI, 64.1 to 89.3), appendicitis (risk ratio, 1.40; 95% CI, 1.02 to 2.01; risk difference, 5.0 events per 100,000 persons; 95% CI, 0.3 to 9.9), and herpes zoster infection (risk ratio, 1.43; 95% CI, 1.20 to 1.73; risk difference, 15.8 events per 100,000 persons; 95% CI, 8.2 to 24.2). SARS-CoV-2 infection was associated with a substantially increased risk of myocarditis (risk ratio, 18.28; 95% CI, 3.95 to 25.12; risk difference, 11.0 events per 100,000 persons; 95% CI, 5.6 to 15.8) and of additional serious adverse events, including pericarditis, arrhythmia, deep-vein thrombosis, pulmonary embolism, myocardial infarction, intracranial hemorrhage, and thrombocytopenia.
Conclusions
In this study in a nationwide mass vaccination setting, the BNT162b2 vaccine was not associated with an elevated risk of most of the adverse events examined. The vaccine was associated with an excess risk of myocarditis (1 to 5 events per 100,000 persons). The risk of this potentially serious adverse event and of many other serious adverse events was substantially increased after SARS-CoV-2 infection. (Funded by the Ivan and Francesca Berkowitz Family Living Laboratory Collaboration at Harvard Medical School and Clalit Research Institute.)
More than 1 year into the pandemic of coronavirus disease 2019 (Covid-19), the disease caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), an unprecedented number of mass vaccination efforts are under way worldwide. Globally, nearly 3.4 billion doses of vaccine have been administered over the 6-month period since the first vaccines were approved.1
Phase 3 clinical trials showed that several Covid-19 vaccines were efficacious and had an acceptable safety profile.2-4 A number of potential adverse events were identified during these trials, including lymphadenopathy and idiopathic facial-nerve (Bell’s) palsy.2,3 Trials of the BNT162b2 vaccine (Pfizer–BioNTech) also showed a mild imbalance between the vaccinated and placebo groups with respect to the number of cases of appendicitis, hypersensitivity reactions, acute myocardial infarction, and cerebrovascular accidents.5 However, phase 3 trials may have inherent limitations in assessing vaccine safety because of a small number of participants and a healthier-than-average sample population. Hence, they are often underpowered to identify less common adverse events. Postmarketing surveillance is required to monitor the safety of new vaccines in real-world settings.
Much effort is currently focused on characterizing the safety profiles of the recently approved Covid-19 vaccines. Passive surveillance systems such as the Vaccine Adverse Event Reporting System (VAERS)6 collect information about adverse events that are potentially related to vaccination. This information is voluntarily reported by health care providers and the public. These systems are useful for quickly identifying potential safety signals, which, along with the findings of phase 3 trials, can be translated to lists of adverse events of interest for further exploration (such as that provided by the Safety Platform for Emergency Vaccines [SPEAC]).7,8 Active surveillance systems such as the Biologics Effectiveness and Safety (BEST) system (part of the Sentinel Initiative)9 aim to compare the incidence of adverse events of interest in large electronic health record databases with the background historical incidence. Although active surveillance can help highlight suspicious trends, the lack of a rigorously constructed comparable control group limits the ability of such surveillance to identify causal effects of vaccination.
The effectiveness of vaccines against SARS-CoV-2 has been confirmed in real-world studies,10,11 but high-quality real-world safety data on the messenger RNA (mRNA)–based Covid-19 vaccines remain relatively sparse in the literature. The results of a study based on data reported by more than 600,000 vaccinated persons were recently published12; that study mainly assessed common and mild side effects. Two additional studies, which were based on surveys of vaccinated participants, involved small cohorts,13,14 and another study analyzed adverse events reported in the VAERS database.15 All these studies lacked controls. One study that did incorporate a control group included 8533 long-term care facility residents who had received the first dose of vaccine.16 The authors of this study concluded that the mRNA-based vaccines had an acceptable safety profile, and no notable adverse events were reported.
As of May 24, 2021, nearly 5 million people in Israel, comprising more than 55% of the population, had received two doses of the BNT162b2 vaccine.1 In this study, we used the integrated data repositories of the largest health care organization in Israel to evaluate the safety profile of the BNT162b2 vaccine. We compared the incidence of a broad set of potential short- and medium-term adverse events among vaccinated persons with the incidence among matched unvaccinated persons. Potential adverse events related to medical interventions are best understood in the context of the risks associated with the disease that these interventions aim to prevent or treat, so we also estimated the effects of SARS-CoV-2 infection on this same set of adverse events.
Methods
Study Setting
We analyzed observational data from Clalit Health Services (CHS) in order to emulate a target trial of the effects of the BNT162b2 vaccine on a broad range of potential adverse events in a population without SARS-CoV-2 infection. CHS is the largest of four integrated payer–provider health care organizations that offer mandatory health care coverage in Israel. CHS insures approximately 52% of the population of Israel (>4.7 million of 9.0 million persons), and the CHS-insured population is approximately representative of the Israeli population at large.17 CHS directly provides outpatient care, and inpatient care is divided between CHS and out-of-network hospitals. CHS information systems are fully digitized and feed into a central data warehouse. Data regarding Covid-19, including the results of all SARS-CoV-2 polymerase-chain-reaction (PCR) tests, Covid-19 diagnoses and severity, and vaccinations, are collected centrally by the Israeli Ministry of Health and shared with each of the four national health care organizations daily.
This study was approved by the CHS institutional review board. The study was exempt from the requirement for informed consent.
Eligibility Criteria
Eligibility criteria included an age of 16 years or older, continuous membership in the health care organization for a full year, no previous SARS-CoV-2 infection, and no contact with the health care system in the previous 7 days (the latter criterion was included as an indicator of a health event not related to subsequent vaccination that could reduce the probability of receiving the vaccine). Because of difficulties in distinguishing the recoding of previous events from true new events, for each adverse event, persons with a previous diagnosis of that event were excluded.
As in our previous study of the effectiveness of the BNT162b2 vaccine,10 we also excluded persons from populations in which confounding could not be adequately addressed — long-term care facility residents, persons confined to their homes for medical reasons, health care workers, and persons for whom data on body-mass index or residential area were missing (missing data for these variables are rare in the CHS data). A complete definition of the study variables is included in Table S1 in the Supplementary Appendix, available with the full text of this article at NEJM.org.
Study Design and Oversight
The target trial for this study would assign eligible persons to either vaccination or no vaccination. To emulate this trial, on each day from the beginning of the vaccination campaign in Israel (December 20, 2020) until the end of the study period (May 24, 2021), eligible persons who were vaccinated on that day were matched to eligible controls who had not been previously vaccinated. Since the matching process each day considered only information available on or before that day (and was thus unaffected by later vaccinations or SARS-CoV-2 infections), unvaccinated persons matched on a given day could be vaccinated on a future date, and on that future date they could become newly eligible for inclusion in the study as a vaccinated person.
In an attempt to emulate randomized assignment, vaccinated persons and unvaccinated controls were exactly matched on a set of baseline variables that were deemed to be potential confounders according to domain expertise — namely, variables that were potentially related to vaccination and to a tendency toward the development of a broad set of adverse clinical conditions. These matching criteria included the sociodemographic variables of age (categorized into 2-year age groups), sex (male or female), place of residence (at city- or town-level granularity), socioeconomic status (divided into seven categories), and population sector (general Jewish, Arab, or ultra-Orthodox Jewish). In addition, the matching criteria included clinical factors to account for general clinical condition and disease load, including the number of preexisting chronic conditions (those considered to be risk factors for severe Covid-19 by the Centers for Disease Control and Prevention [CDC] as of December 20, 2020,18 divided into four categories), the number of diagnoses documented in outpatient visits in the year before the index date (categorized into deciles within each age group), and pregnancy status.
All the authors designed the study and critically reviewed the manuscript. The first three authors collected and analyzed the data. A subgroup of the authors wrote the manuscript. The last author vouches for the accuracy and completeness of the data and for the fidelity of the study to the protocol. There was no commercial funding for this study, and no confidentiality agreements were in place.
Adverse Events of Interest
The set of potential adverse events for the target trial was drawn from several relevant sources, including the VAERS, BEST, and SPEAC frameworks, information provided by the vaccine manufacturer, and relevant scientific publications. We cast a wide net to capture a broad range of clinically meaningful short- and medium-term potential adverse events that would be likely to be documented in the electronic health record. Accordingly, mild adverse events such as fever, malaise, and local injection-site reactions were not included in this study. The study included 42 days of follow-up, which provided 21 days of follow-up after each of the first and second vaccine doses. A total of 42 days was deemed to be sufficient for identifying medium-term adverse events, without being so long as to dilute the incidence of short-term adverse events. Similarly, adverse events that could not plausibly be diagnosed within 42 days (e.g., chronic autoimmune disease) were not included.
Adverse events were defined according to diagnostic codes and short free-text phrases that accompany diagnoses in the CHS database. A complete list of the study outcomes (adverse events) and their definitions is provided in Table S2.
For each adverse event, persons were followed from the day of matching (time zero of follow-up) until the earliest of one of the following: documentation of the adverse event, 42 days, the end of the study calendar period, or death. We also ended the follow-up of a matched pair when the unvaccinated control received the first dose of vaccine or when either member of the matched pair received a diagnosis of SARS-CoV-2 infection.
Risks of SARS-CoV-2 Infection
To place the magnitude of the adverse effects of the vaccine in context, we also estimated the effects of SARS-CoV-2 infection on these same adverse events during the 42 days after diagnosis. We used the same design as the one that we used to study the adverse effects of vaccination, except that the analysis period started at the beginning of the Covid-19 pandemic in Israel (March 1, 2020) and persons who had had recent contact with the health care system were not excluded (because such contact may be expected in the days before diagnosis).
Each day in this SARS-CoV-2 analysis, persons with a new diagnosis of SARS-CoV-2 infection were matched to controls who were not previously infected. As in the vaccine safety analysis, persons could become infected with SARS-CoV-2 after they were already matched as controls on a previous day, in which case their data would be censored from the control group (along with their matched SARS-CoV-2–infected person) and they could then be included in the group of SARS-CoV-2–infected persons with a newly matched control. Follow-up of each matched pair started from the date of the positive PCR test result of the infected member and ended in an analogous manner to the main vaccination analysis, this time ending when the control member was infected or when either of the persons in the matched pair was vaccinated.
The effects of vaccination and of SARS-CoV-2 infection were estimated with different cohorts. Thus, they should be treated as separate sets of results rather than directly compared.
Statistical Analysis
Because a large proportion of the unvaccinated controls were vaccinated during the follow-up period, we opted to estimate the observational analogue of the per-protocol effect if all unvaccinated persons had remained unvaccinated during the follow-up. To do so, we censored data on the matched pair if and when the control member was vaccinated. Persons who were first matched as unvaccinated controls and then became vaccinated during the study period could be included again as vaccinated persons with a new matched control. The same procedure was followed in the SARS-CoV-2 infection analysis (i.e., persons who were first matched as uninfected controls and then became infected during the study period could be included again as infected persons with a new matched control).
We used the Kaplan–Meier estimator19 to construct cumulative incidence curves and to estimate the risk of each adverse event after 42 days in each group. The risks were compared with ratios and differences (per 100,000 persons).
In the vaccination analysis, so as not to attribute complications arising from SARS-CoV-2 infection to the vaccination (or lack thereof), we also censored data on the matched pair if and when either member received a diagnosis of SARS-CoV-2 infection. Similarly, in the SARS-CoV-2 infection analysis, we censored data on the matched pair if and when either member was vaccinated. Additional details are provided in the Supplementary Methods 1 section in the Supplementary Appendix.
We calculated confidence intervals using the nonparametric percentile bootstrap method with 500 repetitions. As is standard practice for studies of safety outcomes, no adjustment for multiple comparisons was performed. Analyses were performed with the use of R software, version 4.0.4.
Results
Vaccination Analysis
A total of 1,736,832 persons were eligible for inclusion in the vaccination cohort (Figure 1). The median age in the eligible cohort was 43 years (Table S3). The final size of the study population differed for each studied adverse event because of adverse event–specific exclusion of persons with a history of that event. On average, across the adverse event–specific cohorts, 72.4% of the eligible persons were successfully matched. Table 1 shows the baseline characteristics of the total study population, with the mean distribution of characteristics across the various adverse event–specific cohorts. The characteristics of each adverse event–specific cohort are provided in Table S4. The vaccination cohorts included a mean of 884,828 vaccinated persons, with a median age of 38 years (5 years younger than the median age of the eligible cohort). A total of 48% of the population was female.
Figure 1
Study Population for the Vaccination Analysis.
Absolute numbers and percentage changes are shown for each inclusion and exclusion criterion. The chart focuses on the vaccinated population. The derivation group includes the entire population, including unvaccinated persons. The shaded boxes indicate the two study groups. The same exclusion criteria were applied to the unvaccinated persons for each index date on which they were considered for matching. BMI denotes body-mass index, CHS Clalit Health Services, and PCR polymerase chain reaction.
Table 1
| Characteristic | Vaccination Analysis | SARS-CoV-2 Analysis | ||
|---|---|---|---|---|
| Vaccinated Group (N=884,828) | Control Group (N=884,828) | SARS-CoV-2–Infected Group (N=173,106) | Control Group (N=173,106) | |
| Median age (IQR) — yr | 38 (27–53) | 38 (27–53) | 34 (24–47) | 34 (24–47) |
| Age group — no. (%) | ||||
| 16–39 yr | 472,095 (53) | 472,095 (53) | 107,046 (62) | 107,046 (62) |
| 40–49 yr | 160,413 (18) | 160,413 (18) | 28,738 (17) | 28,738 (17) |
| 50–59 yr | 93,110 (11) | 93,110 (11) | 17,851 (10) | 17,851 (10) |
| 60–69 yr | 87,236 (10) | 87,236 (10) | 12,100 (7) | 12,100 (7) |
| 70–79 yr | 51,924 (6) | 51,924 (6) | 5,371 (3) | 5,371 (3) |
| ≥80 yr | 20,050 (2) | 20,050 (2) | 1,999 (1) | 1,999 (1) |
| Sex — no. (%) | ||||
| Female | 423,238 (48) | 423,238 (48) | 93,263 (54) | 93,263 (54) |
| Male | 461,590 (52) | 461,590 (52) | 79,843 (46) | 79,843 (46) |
| Population sector — no. (%) | ||||
| General Jewish | 595,897 (67) | 595,897 (67) | 90,903 (53) | 90,903 (53) |
| Ultra-Orthodox Jewish | 24,343 (3) | 24,343 (3) | 20,864 (12) | 20,864 (12) |
| Arab | 264,588 (30) | 264,588 (30) | 61,339 (35) | 61,339 (35) |
| No. of risk factors according to CDC criteria — no. (%) | ||||
| 0 | 571,604 (65) | 571,604 (65) | 108,980 (63) | 108,980 (63) |
| 1 | 200,789 (23) | 200,789 (23) | 41,502 (24) | 41,502 (24) |
| 2 | 61,924 (7) | 61,924 (7) | 11,976 (7) | 11,976 (7) |
| 3 | 27,175 (3) | 27,175 (3) | 5,181 (3) | 5,181 (3) |
| ≥4 | 23,335 (3) | 23,335 (3) | 5,467 (3) | 5,467 (3) |
| CDC “certain” risk criteria — no. (%) | ||||
| Cancer | 9,957 (1) | 10,300 (1) | 2,037 (1) | 2,308 (1) |
| Chronic kidney disease | 39,837 (4) | 40,339 (5) | 8,269 (5) | 8,141 (5) |
| Chronic obstructive pulmonary disease | 10,121 (1) | 11,498 (1) | 1,791 (1) | 2,212 (1) |
| Heart disease | 31,836 (4) | 31,596 (4) | 5,653 (3) | 5,880 (3) |
| Solid-organ transplantation | 351 (<1) | 370 (<1) | 148 (<1) | 136 (<1) |
| Obesity: BMI, 30 to 40 | 129,148 (15) | 125,120 (14) | 30,558 (18) | 28,580 (17) |
| Severe obesity: BMI, ≥40 | 11,861 (1) | 12,568 (1) | 3,478 (2) | 3,107 (2) |
| Pregnancy | 6,082 (1) | 6,082 (1) | 4,959 (3) | 4,959 (3) |
| Sickle cell disease | 140 (<1) | 182 (<1) | 50 (<1) | 55 (<1) |
| Smoking | 157,803 (18) | 187,822 (21) | 18,899 (11) | 30,376 (18) |
| Type 2 diabetes mellitus | 61,865 (7) | 61,093 (7) | 12,448 (7) | 12,396 (7) |
| CDC “possible” risk criteria — no. (%) | ||||
| Asthma | 46,836 (5) | 47,151 (5) | 10,079 (6) | 10,413 (6) |
| Cerebrovascular disease | 14,296 (2) | 14,919 (2) | 2,661 (2) | 2,738 (2) |
| Other respiratory disease | 1,884 (<1) | 1,961 (<1) | 322 (<1) | 362 (<1) |
| Hypertension | 94,819 (11) | 93,357 (11) | 15,514 (9) | 15,682 (9) |
| Immunosuppression | 15,430 (2) | 15,433 (2) | 4,346 (2) | 4,457 (3) |
| Neurologic disease | 26,340 (3) | 28,421 (3) | 5,194 (3) | 5,455 (3) |
| Liver disease | 10,491 (1) | 12,558 (1) | 2,391 (1) | 2,600 (2) |
| Overweight: BMI, 25 to 30 | 284,904 (32) | 271,335 (31) | 53,374 (31) | 50,038 (29) |
| Thalassemia | 5,884 (1) | 5,644 (1) | 1,599 (1) | 1,595 (1) |
| Type 1 diabetes mellitus | 2,797 (<1) | 2,648 (<1) | 694 (<1) | 763 (<1) |
Baseline Characteristics of the Study Populations According to Vaccination Status and SARS-CoV-2 Infection Status.*
*
Statistics are based on means and distributions from a pool of all the adverse event–specific cohorts. Characteristics of the various study populations after application of all eligibility criteria and the matching process are listed. BMI denotes body-mass index (the weight in kilograms divided by the square of the height in meters), CDC Centers for Disease Control and Prevention, IQR interquartile range, RT-PCR reverse-transcriptase polymerase chain reaction, and SARS-CoV-2 severe acute respiratory syndrome coronavirus 2.
The effect of vaccination on the various potential adverse events included in this study is presented in Table 2. The risk was substantially higher on either the multiplicative (risk ratio) or additive (risk difference) scales in the vaccinated group than in the unvaccinated group for myocarditis (risk ratio, 3.24; 95% confidence interval [CI], 1.55 to 12.44; risk difference, 2.7 events per 100,000 persons; 95% CI, 1.0 to 4.6), lymphadenopathy (risk ratio, 2.43; 95% CI, 2.05 to 2.78; risk difference, 78.4 events per 100,000 persons; 95% CI, 64.1 to 89.3), appendicitis (risk ratio, 1.40; 95% CI, 1.02 to 2.01; risk difference, 5.0 events per 100,000 persons; 95% CI, 0.3 to 9.9), and herpes zoster infection (risk ratio, 1.43; 95% CI, 1.20 to 1.73; risk difference, 15.8 events per 100,000 persons; 95% CI, 8.2 to 24.2). Vaccination was substantially protective against adverse events such as anemia, acute kidney injury, intracranial hemorrhage, and lymphopenia.
Table 2
| Event | Adverse-Event Cohort in Each Group | Vaccinated Group | Control Group | Risk Ratio (95% CI) | Risk Difference (95% CI) |
|---|---|---|---|---|---|
| no. of persons | no. of events | no. of events/100,000 persons | |||
| Acute kidney injury | 912,019 | 20 | 45 | 0.44 (0.23 to 0.73) | −4.6 (−7.8 to −1.8) |
| Anemia | 709,267 | 298 | 378 | 0.79 (0.67 to 0.93) | −18.7 (−32.1 to −6.1) |
| Appendicitis | 900,289 | 95 | 66 | 1.40 (1.02 to 2.01) | 5.0 (0.3 to 9.9) |
| Arrhythmia | 856,152 | 254 | 284 | 0.89 (0.74 to 1.04) | −6.1 (−14.7 to 1.8) |
| Arthritis or arthropathy | 731,340 | 64 | 70 | 0.95 (0.65 to 1.34) | −0.8 (−6.3 to 4.2) |
| Bell’s palsy | 923,692 | 81 | 59 | 1.32 (0.92 to 1.86) | 3.5 (−1.1 to 7.8) |
| Cerebrovascular accident | 917,598 | 45 | 55 | 0.84 (0.54 to 1.27) | −1.6 (−5.3 to 2.0) |
| Deep-vein thrombosis | 925,380 | 39 | 47 | 0.87 (0.55 to 1.40) | −1.1 (−4.5 to 2.7) |
| Herpes simplex infection | 876,328 | 219 | 205 | 1.13 (0.95 to 1.38) | 4.8 (−1.9 to 12.4) |
| Herpes zoster infection | 888,647 | 283 | 204 | 1.43 (1.20 to 1.73) | 15.8 (8.2 to 24.2) |
| Intracranial hemorrhage | 933,130 | 13 | 30 | 0.48 (0.20 to 0.89) | −2.9 (−5.6 to −0.5) |
| Lymphadenopathy | 823,006 | 660 | 279 | 2.43 (2.05 to 2.78) | 78.4 (64.1 to 89.3) |
| Lymphopenia | 938,939 | 2 | 7 | 0.26 (0.00 to 1.03) | −0.9 (−2.0 to <0.1) |
| Myocardial infarction | 892,785 | 59 | 60 | 1.07 (0.74 to 1.60) | 0.8 (−3.3 to 5.2) |
| Myocarditis | 938,812 | 21 | 6 | 3.24 (1.55 to 12.44) | 2.7 (1.0 to 4.6) |
| Neutropenia | 919,291 | 20 | 22 | 0.87 (0.46 to 1.66) | −0.5 (−2.8 to 1.8) |
| Other thrombosis† | 932,469 | 12 | 22 | 0.46 (0.19 to 0.91) | −2.2 (−4.6 to −0.3) |
| Paresthesia | 827,478 | 552 | 496 | 1.12 (0.98 to 1.24) | 10.8 (−1.8 to 21.4) |
| Pericarditis | 936,197 | 27 | 18 | 1.27 (0.68 to 2.31) | 1.0 (−1.6 to 3.4) |
| Pulmonary embolism | 937,116 | 10 | 17 | 0.56 (0.21 to 1.15) | −1.5 (−3.6 to 0.4) |
| Seizure | 913,091 | 36 | 35 | 0.99 (0.62 to 1.64) | −0.4 (−3.0 to 3.1) |
| Syncope | 858,068 | 326 | 267 | 1.12 (0.94 to 1.34) | 6.2 (−3.2 to 15.4) |
| Thrombocytopenia | 923,123 | 56 | 60 | 0.94 (0.63 to 1.27) | −0.6 (−4.6 to 2.3) |
| Uveitis | 933,217 | 26 | 20 | 1.27 (0.68 to 2.67) | 1.0 (−1.5 to 3.8) |
| Vertigo | 773,263 | 433 | 395 | 1.12 (0.97 to 1.28) | 9.3 (−2.5 to 20.0) |
Adverse Events Associated with SARS-CoV-2 Vaccination.*
*
Estimates were calculated with the use of the Kaplan–Meier estimator 42 days after vaccination or SARS-CoV-2 infection. Confidence intervals (CIs) were estimated with the use of the percentile bootstrap method with 500 repetitions.
†
The “other thrombosis” category is a composite diagnosis that includes arterial embolism and thrombosis, venous embolism and thrombosis, vascular insufficiency of the intestine, portal-vein thrombosis, or cranial venous sinus thrombosis.
Figure S1 shows the cumulative incidence (risk) curves for each specific adverse event. Spikes in the incidence of lymphadenopathy were seen after both the first and second doses of vaccine, whereas the incidence of myocarditis spiked mainly after the second dose of vaccine.
SARS-CoV-2 Infection Analysis
A total of 233,392 persons (median age, 36 years) were eligible to be included in the SARS-CoV-2 infection cohort (Figure 2). On average, across the adverse event–specific cohorts, 75.8% of the eligible persons were successfully matched. Table 1 shows the average distribution of characteristics in these cohorts, across the two study groups (infected and noninfected). The characteristics of each adverse event–specific cohort are provided in Table S5. The cohorts for the analysis of SARS-CoV-2 infection comprised a mean of 173,106 SARS-CoV-2–infected persons (median age, 34 years). A total of 54% of these persons were female.
Figure 2
Study Population for the SARS-CoV-2 Analysis.
Absolute numbers and percentage changes are shown for each inclusion and exclusion criterion. The chart focuses on the SARS-CoV-2–infected population. The derivation group includes the entire population, including uninfected persons. The shaded boxes indicate the two study groups. The same exclusion criteria were applied to the uninfected persons for each index date on which they were considered for matching. Covid-19 denotes coronavirus disease 2019.
Table S6 shows the effect of SARS-CoV-2 infection on the incidence of various adverse events. Infection substantially increased the risk of many different adverse events, including myocarditis (risk ratio, 18.28; 95% CI, 3.95 to 25.12; risk difference, 11.0 events per 100,000 persons; 95% CI, 5.6 to 15.8), acute kidney injury (risk ratio, 14.83; 95% CI, 9.24 to 28.75; risk difference, 125.4 events per 100,000 persons; 95% CI, 107.0 to 142.6), pulmonary embolism (risk ratio, 12.14; 95% CI, 6.89 to 29.20; risk difference, 61.7 events per 100,000 persons; 95% CI, 48.5 to 75.4), intracranial hemorrhage (risk ratio, 6.89; 95% CI, 1.90 to 19.16; risk difference, 7.6 events per 100,000 persons; 95% CI, 2.7 to 12.6), pericarditis (risk ratio, 5.39; 95% CI, 2.22 to 23.58; risk difference, 10.9 events per 100,000 persons; 95% CI, 4.9 to 16.9), myocardial infarction (risk ratio, 4.47; 95% CI, 2.47 to 9.95; risk difference, 25.1 events per 100,000 persons; 95% CI, 16.2 to 33.9), deep-vein thrombosis (risk ratio, 3.78; 95% CI, 2.50 to 6.59; risk difference, 43.0 events per 100,000 persons; 95% CI, 29.9 to 56.6), and arrhythmia (risk ratio, 3.83; 95% CI, 3.07 to 4.95; risk difference, 166.1 events per 100,000 persons; 95% CI, 139.6 to 193.2).
Both Analyses
Figure 3 shows estimated risk ratios in both the vaccination and SARS-CoV-2 infection analyses for adverse events in which vaccination or infection substantially increased the risk. Figure 4 shows the absolute risk associated with vaccination, alongside the absolute risk associated with SARS-CoV-2 infection, for the same adverse events.
Figure 3
Risk Ratios for Adverse Events after Vaccination or SARS-CoV-2 Infection.
Estimated risk ratios for adverse events after vaccination or SARS-CoV-2 infection are shown. The risk ratio on the y axis is presented on a logarithmic scale to facilitate comparison of both increased and decreased risk. 𝙸 bars indicate 95% confidence intervals.
Figure 4
Absolute Excess Risk of Various Adverse Events after Vaccination or SARS-CoV-2 Infection.
Point estimates of the risk differences for selected adverse events are shown. Estimates were derived 42 days after vaccination or SARS-CoV-2 infection with the use of the Kaplan–Meier estimator. Risk differences are shown per 100,000 persons and rounded to the nearest integer. Negative differences (decreased risk) are represented as negative values on the y axis, and positive differences (increased risk) are represented as positive values on the y axis. The abbreviation mRNA denotes messenger RNA.
Discussion
We used a data set involving more than 2.4 million vaccinated persons from an integrated health care organization to evaluate the safety profile of the BNT162b2 mRNA Covid-19 vaccine. The main potential adverse events identified included an excess risk of lymphadenopathy (78.4 events per 100,000 persons), herpes zoster infection (15.8 events), appendicitis (5.0 events), and myocarditis (2.7 events).
To place these risks in context, we also examined data on more than 240,000 infected persons to estimate the effects of a documented SARS-CoV-2 infection on the incidence of the same adverse events. SARS-CoV-2 infection was not estimated to have a meaningful effect on the incidence of lymphadenopathy, herpes zoster infection, or appendicitis, but it was estimated to result in a substantial excess risk of myocarditis (11.0 events per 100,000 persons). SARS-CoV-2 infection was also estimated to substantially increase the risk of several adverse events for which vaccination was not found to increase the risk, including an estimated excess risk of arrhythmia (166.1 events per 100,000 persons), acute kidney injury (125.4 events), pulmonary embolism (61.7 events), deep-vein thrombosis (43.0 events), myocardial infarction (25.1 events), pericarditis (10.9 events), and intracranial hemorrhage (7.6 events).
An association between Covid-19 vaccination and myocarditis has been previously reported.20 Although no cases of myocarditis were reported in the BNT162b2 (Pfizer–BioNTech),2 mRNA-1273 (Moderna),3 or ChAdOx1 nCoV-19 (AstraZeneca)4 phase 3 clinical trials, multiple cases of myocarditis after Covid-19 vaccination have recently been reported in the literature,21-25 and both the Israeli Ministry of Health26 and the CDC have investigated this association.27 The risk appears to be highest among young men.26,27 We found that the risk of myocarditis increased by a factor of three after vaccination, which translated to approximately 3 excess events per 100,000 persons; the 95% confidence interval indicated that values between 1 and 5 excess events per 100,000 persons were compatible with our data. Among the 21 persons with myocarditis in the vaccinated group, the median age was 25 years (interquartile range, 20 to 34), and 90.9% were male.
Another vaccine-related adverse event that has recently received attention in the medical literature is Bell’s palsy. In a recent article based on publicly available data from the BNT162b2 and mRNA-1273 vaccine trials, Ozonoff et al.28 suggested a possible association between these vaccines and Bell’s palsy and estimated a rate ratio of approximately 7.0. This conclusion differed from the Food and Drug Administration briefing on these vaccines in December 2020; that briefing considered the incidence of Bell’s palsy to be similar to the background incidence.5 A small number of cases of Bell’s palsy after Covid-19 vaccination have also been reported in the literature.29,30 In the current study, the effect estimate was consistent with a potentially mild increase in the risk of Bell’s palsy after vaccination, with a risk ratio of 1.32 (95% CI, 0.92 to 1.86). The absolute effect was small, with up to 8 excess events per 100,000 persons being highly compatible with our data according to the 95% confidence interval. Herpes zoster infection, the incidence of which we found to be increased after vaccination, is one of the potential causes of facial-nerve palsy.31
Another particularly notable class of adverse events that has been reported in the context of Covid-19 vaccines is thromboembolic events. These adverse events, which primarily affect young women, have been linked with the ChAdOx1 nCoV-1932 and Ad26.COV2.S (Johnson & Johnson–Janssen) Covid-19 vaccines,33 both of which are adenoviral vector vaccines. However, we did not find an association between the BNT162b2 vaccine and various thromboembolic events in this study.
Some initially unexpected effects were seen in the results of the current study. The BNT162b2 vaccine appears to be protective against certain conditions such as anemia and intracranial hemorrhage. These same adverse events are also identified in this study as complications of SARS-CoV-2 infection, so it appears likely that the protective effect of the vaccine is mediated through its protection against undiagnosed SARS-CoV-2 infection, which may be undiagnosed either because of a lack of testing or because of false negative PCR results.
This study has several limitations. First, persons in the study were not randomly assigned according to exposures (vaccinations and SARS-CoV-2 infections); this may have introduced confounding at baseline and selection bias at censoring, especially since a single set of confounders was used for adjustment in the assessment of many disparate adverse events. Second, the matching process that was necessary to attain exchangeability between the study groups resulted in a study population with a different composition than the eligible population (e.g., median age, 38 years rather than 43 years). Because this different composition changes the population over which the causal effect is being estimated, different estimates might be found for adverse events for which the incidence may differ substantially between subgroups (e.g., myocarditis). Also, we excluded certain populations (such as health care workers and persons residing in long-term care facilities) that could be at particularly high risk for certain adverse events. Both of these issues should be taken into account when considering the generalizability of the findings.
Third, some diagnoses that were recorded in out-of-network hospitals, which were delayed in being reported to the insurer and were not entered by the person’s general practitioner from the hospital discharge notes into the outpatient medical record, could have been missed. Fourth, it is possible that persons are more likely to increase their levels of clinical awareness, concern, or both after vaccination or SARS-CoV-2 infection, and thus they may be more likely to report or seek medical care for their symptoms, resulting in a spuriously increased incidence of the various adverse events in the vaccinated or infected groups. Similarly, among persons with SARS-CoV-2 infection, the spike in the incidence of certain adverse events in the first day of follow-up could indicate the initial clinical manifestation of the infection, but it could also be related to active testing for SARS-CoV-2. Fifth, all the effect measures that we presented are based only on a new incidence of the specific adverse event under study; thus, less light was shed on the potential additional risk among persons with a medical history of each of these adverse events. However, this choice was necessary to distinguish between true new diagnoses of a given adverse event and recoding of past diagnoses and to ensure the accuracy of the adverse-event labels.
In this study, we sought to place the increased risk of adverse events caused by the BNT162b2 vaccine in context by contrasting this risk with that of the same adverse events after documented infection with SARS-CoV-2. We thought that this was necessary because vaccination and its potential risks do not occur in a void but rather in the context of an ongoing pandemic. Although the general risks of hospitalization, severe disease, and death from Covid-19 are widely recognized, secondary complications of infection are less well known. Therefore, in this analysis, we sought to estimate the effects of SARS-CoV-2 infection on the incidence of the same list of adverse events examined in the vaccination analysis. Because the cohorts that we used to study the vaccine and infection effects were different in composition, care should be taken when comparing the resulting risk estimates. In addition, knowledge of these risks alone is insufficient for a complete decision-theoretic analysis. When a person decides to become vaccinated, this choice results in a probability of 100% for the vaccination, whereas the alternative of contracting SARS-CoV-2 infection is an event with uncertain probability that depends on the person, place, and time. Moreover, infection with SARS-CoV-2 has many other adverse effects beyond those considered here, including the risk of transmission to family members and others.
We estimated that the BNT162b2 vaccine resulted in an increased incidence of a few adverse events over a 42-day follow-up period. Although most of these events were mild, some of them, such as myocarditis, could be potentially serious. However, our results indicate that SARS-CoV-2 infection is itself a very strong risk factor for myocarditis, and it also substantially increases the risk of multiple other serious adverse events. These findings help to shed light on the short- and medium-term risks of the vaccine and place them in clinical context. Further studies will be needed to estimate the potential of long-term adverse events.
Notes
Because of data privacy regulations, the raw data for this study cannot be shared.
This article was published on August 25, 2021, at NEJM.org.
Supported by the Ivan and Francesca Berkowitz Family Living Laboratory Collaboration at Harvard Medical School and Clalit Research Institute. Dr. Lipsitch receives support from the Morris–Singer Foundation.
Disclosure forms provided by the authors are available with the full text of this article at NEJM.org.
We thank Oren Miron for his valuable advice.
Supplementary Material
References
2.
Polack FP, Thomas SJ, Kitchin N, et al. Safety and efficacy of the BNT162b2 mRNA Covid-19 vaccine. N Engl J Med 2020;383:2603-2615.
3.
Baden LR, El Sahly HM, Essink B, et al. Efficacy and safety of the mRNA-1273 SARS-CoV-2 vaccine. N Engl J Med 2021;384:403-416.
4.
Voysey M, Clemens SAC, Madhi SA, et al. Safety and efficacy of the ChAdOx1 nCoV-19 vaccine (AZD1222) against SARS-CoV-2: an interim analysis of four randomised controlled trials in Brazil, South Africa, and the UK. Lancet 2021;397:99-111.
7.
World Health Organization. Background paper on Covid-19 disease and vaccines: prepared by the Strategic Advisory Group of Experts (SAGE) on immunization working group on COVID-19 vaccines. December 22, 2020 (https://apps.who.int/iris/handle/10665/338095).
10.
Dagan N, Barda N, Kepten E, et al. BNT162b2 mRNA Covid-19 vaccine in a nationwide mass vaccination setting. N Engl J Med 2021;384:1412-1423.
11.
Hall VJ, Foulkes S, Saei A, et al. COVID-19 vaccine coverage in health-care workers in England and effectiveness of BNT162b2 mRNA vaccine against infection (SIREN): a prospective, multicentre, cohort study. Lancet 2021;397:1725-1735.
12.
Menni C, Klaser K, May A, et al. Vaccine side-effects and SARS-CoV-2 infection after vaccination in users of the COVID Symptom Study app in the UK: a prospective observational study. Lancet Infect Dis 2021;21:939-949.
13.
Mathioudakis AG, Ghrew M, Ustianowski A, et al. Self-reported real-world safety and reactogenicity of COVID-19 vaccines: a vaccine recipient survey. Life (Basel) 2021;11:249-249.
14.
Song JY, Cheong HJ, Kim SR, et al. Early safety monitoring of COVID-19 vaccines in healthcare workers. J Korean Med Sci 2021;36(15):e110-e110.
15.
Chen G, Li X, Sun M, et al. COVID-19 mRNA vaccines are generally safe in the short term: a vaccine vigilance real-world study says. Front Immunol 2021;12:669010-669010.
16.
Bardenheier BH, Gravenstein S, Blackman C, et al. Adverse events following mRNA SARS-CoV-2 vaccination among U.S. nursing home residents. Vaccine 2021;39:3844-3851.
19.
Kaplan EL, Meier P. Nonparametric estimation from incomplete observations. J Am Stat Assoc 1958;53:457-481.
20.
Castiello T, Georgiopoulos G, Finocchiaro G, et al. COVID-19 and myocarditis: a systematic review and overview of current challenges. Heart Fail Rev 2021 March 24 (Epub ahead of print).
21.
Ammirati E, Cavalotti C, Milazzo A, et al. Temporal relation between second dose BNT162b2 mRNA Covid-19 vaccine and cardiac involvement in a patient with previous SARS-COV-2 infection. Int J Cardiol Heart Vasc 2021;34:100774-100774.
22.
Abu Mouch S, Roguin A, Hellou E, et al. Myocarditis following COVID-19 mRNA vaccination. Vaccine 2021;39:3790-3793.
23.
Marshall M, Ferguson ID, Lewis P, et al. Symptomatic acute myocarditis in seven adolescents following Pfizer-BioNTech COVID-19 vaccination. Pediatrics 2021 June 4 (Epub ahead of print).
24.
Albert E, Aurigemma G, Saucedo J, Gerson DS. Myocarditis following COVID-19 vaccination. Radiol Case Rep 2021;16:2142-2145.
25.
Bautista García J, Peña Ortega P, Bonilla Fernández JA, Cárdenes León A, Ramírez Burgos L, Caballero Dorta E. Acute myocarditis after administration of the BNT162b2 vaccine against COVID-19. Rev Esp Cardiol (Engl Ed) 2021 April 27 (Epub ahead of print).
28.
Ozonoff A, Nanishi E, Levy O. Bell’s palsy and SARS-CoV-2 vaccines. Lancet Infect Dis 2021;21:450-452.
29.
Repajic M, Lai XL, Xu P, Liu A. Bell’s palsy after second dose of Pfizer COVID-19 vaccination in a patient with history of recurrent Bell’s palsy. Brain Behav Immun Health 2021;13:100217-100217.
30.
Colella G, Orlandi M, Cirillo N. Bell’s palsy following COVID-19 vaccination. J Neurol 2021 February 21 (Epub ahead of print).
31.
Peitersen E. Bell’s palsy: the spontaneous course of 2,500 peripheral facial nerve palsies of different etiologies. Acta Otolaryngol Suppl 2002;549:4-30.
32.
Greinacher A, Thiele T, Warkentin TE, Weisser K, Kyrle PA, Eichinger S. Thrombotic thrombocytopenia after ChAdOx1 nCov-19 vaccination. N Engl J Med 2021;384:2092-2101.
33.
Centers for Disease Control and Prevention, Health Alert Network. Emergency preparedness and response: cases of cerebral venous sinus thrombosis with thrombocytopenia after receipt of the Johnson & Johnson COVID-19 vaccine. April 13, 2021 (https://emergency.cdc.gov/han/2021/han00442.asp).
Information & Authors
Information
Published In
New England Journal of Medicine
Copyright
Copyright © 2021 Massachusetts Medical Society. All rights reserved.
Translation
History
Published online: August 25, 2021
Published in issue: September 16, 2021
Topics
Authors
Authors
Noam Barda, M.D., Noa Dagan, M.D. https://orcid.org/0000-0001-8811-7825, Yatir Ben-Shlomo, B.Sc., Eldad Kepten, Ph.D., Jacob Waxman, M.D., Reut Ohana, M.Sc., Miguel A. Hernán, M.D. https://orcid.org/0000-0003-1619-8456, Marc Lipsitch, D.Phil. https://orcid.org/0000-0003-1504-9213, Isaac Kohane, M.D., Doron Netzer, M.D., Ben Y. Reis, Ph.D., and Ran D. Balicer, M.D.
Affiliations
From the Clalit Research Institute, Innovation Division (N.B., N.D., Y.B.-S., E.K., J.W., R.O., R.D.B.), and the Community Medical Services Division (D.N.), Clalit Health Services, Tel Aviv, and Software and Information Systems Engineering (N.B., N.D.) and the School of Public Health, Faculty of Health Sciences (R.D.B.), Ben-Gurion University of the Negev, Be’er Sheva — both in Israel; the Department of Biomedical Informatics (N.B., N.D., I.K.), and the Ivan and Francesca Berkowitz Family Living Laboratory Collaboration at Harvard Medical School and Clalit Research Institute (N.B., N.D., I.K., B.Y.R., R.D.B.), Harvard Medical School (B.Y.R.), the Departments of Epidemiology and Biostatistics (M.A.H.), CAUSALab (M.A.H.), and the Center for Communicable Disease Dynamics, Departments of Epidemiology and Immunology and Infectious Diseases (M.L.), Harvard T.H. Chan School of Public Health, and the Predictive Medicine Group, Computational Health Informatics Program, Boston Children’s Hospital (B.Y.R.) — all in Boston.
Notes
Address reprint requests to Dr. Balicer at the Clalit Research Institute, Innovation Division, Clalit Health Services, 101 Arlozorov St., Tel Aviv, Israel, or at [email protected].
Drs. Barda and Dagan and Drs. Reis and Balicer contributed equally to this article.
Metrics & Citations
Metrics
Altmetrics
Citations
Export citation
Select the format you want to export the citation of this publication.
Cited by
- Nelson Luis Cahuapaza-Gutierrez,
- Renzo Pajuelo-Vasquez,
- Cristina Quiroz-Narvaez,
- Flavia Rioja-Torres,
- María Quispe-Andahua,
- Fernando M. Runzer-Colmenares,
- Qiong Wu,
- Jiayi Tong,
- Bingyu Zhang,
- Dazheng Zhang,
- Jiajie Chen,
- Yuqing Lei,
- Yiwen Lu,
- Yudong Wang,
- Lu Li,
- Yishan Shen,
- Jie Xu,
- L. Charles Bailey,
- Jiang Bian,
- Dimitri A. Christakis,
- Megan L. Fitzgerald,
- Kathryn Hirabayashi,
- Ravi Jhaveri,
- Alka Khaitan,
- Tianchen Lyu,
- Suchitra Rao,
- Hanieh Razzaghi,
- Hayden T. Schwenk,
- Fei Wang,
- Margot I. Gage Witvliet,
- Eric J. Tchetgen Tchetgen,
- Jeffrey S. Morris,
- Christopher B. Forrest,
- Yong Chen,
- Henry Krasner,
- Nicolette Harmon,
- Jeffrey Martin,
- Crysty-Ann Olaco,
- Dale M. Netski,
- Kavita Batra,
- Tal Gazitt,
- Noa Hayat,
- Nili Stein,
- Amir Haddad,
- Ilan Feldhamer,
- Arnon Dov Cohen,
- Walid Saliba,
- Devy Zisman,
- Jung Yoon Choi,
- Yongjoon Lee,
- Nam Gi Park,
- Mi Sung Kim,
- Sandy Jeong Rhie,
- Martin Heil,
- Shir Azrielant,
- Yair Levin,
- Alon Peled,
- Liat Samuelov,
- Eli Sprecher,
- Mor Pavlovsky,
- Jin Kyun Park,
- Eun Bong Lee,
- Kevin L Winthrop,
- Fabrizio Martora,
- Matteo Megna,
- Teresa Battista,
- Massimiliano Scalvenzi,
- Alessia Villani,
- Sara Cacciapuoti,
- Luca Potestio,
- Chunhua Zhou,
- Yuanzheng Qiu,
- Jianxin Wang,
- Xiang Zhong,
- Xiufang Zhu,
- Xiaojing Huang,
- Lan Yang,
- Qiaolei Ji,
- Feifei Zhou,
- Shunquan Wu,
- Mengjie Yang,
- Jing Zhang,
- Kaili Liu,
- Li Ji,
- Hanyu Yang,
- Chunlei Li,
- Yuanyuan Zhao,
- See more
Media
Figures
Other
Tables