RCTs and RWE Considerations
24-25 Appaneal Study Design
24-25 Appaneal VE Analysis
Additional RWE Information
Tab Number 5
RCTs: Gold Standard
- Gold standard of clinical trial design with the highest level of evidence in clinical studies1
- Designed to reduce biases when comparing drug treatments2
- Due to highly specific inclusion and exclusion criteria, RCT results may not be generalizable to the broader patient population in clinical practice1
- May not be able to address questions that require large patient populations or when ethical or time constraints exist1
RWE: May Complement RCT Findings
- May use data from routine clinical practice3
- RWE is clinical evidence about the usage and potential benefits or risks of a medical product derived from analysis of real-world data. Real-world data are data related to patient health status and/or delivery of health care3,4
- May include a broader, more heterogeneous patient population and, therefore, may be more generalizable2,3,5,6
- Unable to determine causality3,6
- Biases related to treatment selection and unobserved variables cannot be fully addressed3
- Other limitations include limited internal validity, potential data quality, and/or methodology issues3,5,6
Summary of Pivotal Trial (Study 2)
Study 2 was a Phase 1/2/3, multicenter, randomized, observer-blind, placebo-controlled trial evaluating the efficacy and safety of COMIRNATY for the prevention of COVID-19.7
Approximately 46,000 participants 12 years of age and older were enrolled, of which ~23,000 received COMIRNATY, including ~20,000 participants ≥16 years of age who were part of the updated vaccine efficacy analysis.7
Participants were randomized equally to receive 2 doses of COMIRNATY or placebo 21 days apart and were followed for the development of COVID-19 beginning 7 days after Dose 2. The study excluded individuals who were immunocompromised or had prior SARS-CoV-2 infection.7
Select Important Safety Information
Do not administer COMIRNATY to individuals with known history of a severe allergic reaction (e.g., anaphylaxis) to any component of COMIRNATY or to individuals who had a severe allergic reaction (e.g., anaphylaxis) following a previous dose of a Pfizer-BioNTech COVID-19 vaccine.
Adverse reactions
Serious adverse events reported in participants who had received at least 1 dose of vaccine or placebo in Study 2:
- 12 years of age and older (≥10%) were pain at the injection site (up to 90.5%), fatigue (up to 77.5%), headache (up to 75.5%), chills (up to 49.2%), muscle pain (up to 45.5%), joint pain (up to 27.5%), fever (up to 24.3%), injection site swelling (up to 11.8%), and injection site redness (up to 10.4%).
Efficacy analyses supported the approval of COMIRNATY for preventing COVID-19 caused by SARS-CoV-2 in individuals 16 years of age and older, with the primary endpoint assessing prevention of COVID-19 and secondary endpoints assessing prevention of severe disease.7 Descriptive efficacy analysis for 12-15 years of age assessed prevention of COVID-19.
Click to see the pivotal trial (Study 2) results for COMIRNATY
VE definition and limitations of RWE studies
VE definition and limitations of RWE studies
This study has several limitations that healthcare professionals should consider in evaluating results to potentially complement RCT data:
- This test-negative, case-control study is considered a reliable design for evaluation of real-world VE but is still susceptible to selection bias
- This study could not adjust for potential residual confounding due to unmeasured or unknown factors
- This study presented early VE estimates with a median time since KP.2 vaccination of only 33 days; therefore, future studies to evaluate longer-term durability are needed
- Some ARI episodes may have involved seeking care with COVID-19 rather than for COVID-19, potentially leading to an underestimation of VE
- The Veterans Affairs population is generally older, predominantly male, and with a high prevalence of multiple comorbid conditions, and may not represent the general US population
- Prior-season vaccination may influence VE; however, stratified analyses based on vaccination history was not carried out due to limited sample size
Other analyses of COVID-19–associated outcomes are available; however, they are not shown here.
Up next >
Find links for ordering, customer service, and information on COVID-19.
ARI=acute respiratory infection; SARS-CoV-2=severe acute respiratory syndrome coronavirus 2.
- “Early effectiveness of the BNT162b2 KP.2 vaccine against COVID-19 in the US Veterans Affairs Healthcare System" by Appaneal et al was a test-negative, case-control study within the US Veterans Affairs Healthcare System that aimed to estimate early vaccine effectiveness of the BNT162b2 KP.2 vaccine (2024-2025 formulation) compared to not receiving the KP.2 vaccine against COVID-19 outcomes9
- VE of BNT162b2 KP.2 vaccine was assessed among adult patients (≥18 years of age) diagnosed with an acute respiratory infection (ARI) in the hospital, ED/UC, or outpatient setting (in-person or virtual) between September 5, 2024, and November 30, 2024. As in prior work, to be included patients had to be tested for SARS-CoV-2 via nucleic acid amplification test (NAAT) or rapid antigen test (RAT) within 14 days prior through 3 days after the ARI encounter. Patients were excluded if they (1) did not have at least one visit to the VA Healthcare System in the previous 12 months, (2) had another prior positive SARS-CoV-2 test in the 90 days prior to their ARI episode, (3) received a KP.2 vaccine other than BNT162b2, (4) received BNT162b2 KP.2 vaccine within 8 weeks of a prior COVID-19 vaccine dose, (5) received BNT162b2 KP.2 vaccine within 14 days prior to their ARI episode, (6) received BNT162b2 KP.2 vaccine but the date of administration was unknown, or (7) received a COVID-19 antiviral (nirmatrelvir/ritonavir, remdesivir, or molnupiravir) within 30 days prior to their ARI episode. Patients could contribute more than one ARI episode to the study if the episodes were more than 30 days apart9
- Hospitalization is a widely recognized measure of severe COVID-19. The limitations of this measure as a marker of severe disease include the challenge in distinguishing hospitalizations due to COVID-19 from hospitalizations in which SARS-CoV-2 infection is incidentally identified, which can lead to differences in vaccine effectiveness estimates generated across various platforms8
Up next >
Find links for ordering, customer service, and information on COVID-19.
PCR=polymerase chain reaction; RCT=randomized controlled trial; RWE=real-world evidence; SARS-CoV-2=severe acute respiratory syndrome coronavirus 2.
Observed estimated early vaccine effectiveness (VE)
Estimated adjusted VE against COVID-19–related outcomes9*
 |
- VE was adjusted for: age, sex, race/ethnicity, BMI, Charlson Comorbidity Index, hospital admission, nursing home admission, ED/UC visit, primary care visit, prior documented SARS-CoV-2 infection, receipt of pneumococcal vaccine in the past 5 years, immunocompromised status, smoking status, and Census region9
- VE findings should be interpreted as the added benefit of 2024-2025 COVID-19 vaccination in a population with high levels of infection-induced immunity, vaccine-induced immunity, or both9,13
- This study presents early VE estimates after a median of 33 days (IQR: 22-46 days) since receipt of the 2024-2025 Formula of COMIRNATY9
- Further evaluation of vaccine durability of protection is needed9
- Lower VE results were reported in 2 other studies in immunocompetent adults utilizing different database populations who received 2024-2025 COVID-19 vaccines9,14,15†
Do not administer COMIRNATY to individuals with known history of a severe allergic reaction (e.g., anaphylaxis) to any component of COMIRNATY or to individuals who had a severe allergic reaction (e.g., anaphylaxis) following a previous dose of a Pfizer-BioNTech COVID-19 vaccine.
Note: Vaccine effectiveness will naturally wane over time and may also be impacted by the emergence of new variants.16
VE should be interpreted as the added benefit of 2023-2024 COVID-19 vaccination in a population with high levels of infection-induced immunity, vaccine-induced immunity, or both.19,20
 |
- “Estimated effectiveness of the BNT162b2 XBB vaccine against COVID-19” by Tartof et al, was a test-negative, case-control study that evaluated estimates of the COMIRNATY 2023-2024 Formula against COVID-19–associated hospital admissions and ED or UC encounters17
- This test-negative, case-control vaccine effectiveness (VE) analysis assessed the VE against XBB and JN.1 sublineages of COVID-19 during the 2023-2024 season17,18
- Cases included those presenting with an acute respiratory illness who had a positive SARS-CoV-2 PCR test; while controls had an acute respiratory illness but tested negative for SARS-CoV-2 (and had no encounters, in any setting, with a positive SARS-CoV-2 test in the prior 90 days)17,18
- Hospitalization is a widely recognized measure of severe COVID-19. The limitations of this measure as a marker of severe disease include the challenge in distinguishing hospitalizations due to COVID-19 from hospitalizations in which SARS-CoV-2 infection is incidentally identified, which can lead to differences in vaccine effectiveness estimates generated across various platforms8
2023-2024 | Potential Limitations of RWE Studies in Vaccine Effectiveness17,18
Overall, limitations of an RWE analysis provide important contextual information and highlight the need for cautious interpretation of findings, as limitations may affect the reliability and generalizability of vaccine effectiveness estimates. VE findings should be interpreted as the added benefit provided by COVID-19 vaccination in a population with a high prevalence of vaccine- and infection-induced immunity at the start of the 2023-2024 respiratory virus season.
This study has several limitations that healthcare professionals should consider in evaluating results to potentially complement RCT data:
- The primary analysis included the use of healthcare encounter data from a single healthcare system. Reliance on data from one healthcare system may limit external validity due to potential biases or lack of heterogeneity of study populations
- Residual Confounding from Test Negative Design: Differences in exposure risk and disease severity between vaccinated and unvaccinated individuals may not have been fully accounted for, potentially impacting the study's conclusions on vaccine effectiveness
- Short Follow-Up Period: The median follow-up after receiving a 2023-2024 Formula COMIRNATY vaccination was 58 days, which may not capture long-term vaccine effectiveness or associated durability of protection
- Potential misclassification of a patient hospitalized with a positive SARS-CoV-2 test as being hospitalized for severe COVID-19 when SARS-CoV-2 infection was incidental to but not the reason for hospitalization. Inclusion of patients hospitalized for other reasons can result in underestimation of VE
- Missing genotype information and potential for misclassification of SARS-CoV-2 sublineages: Lack of complete genotype data or misclassification can limit the ability to differentiate and assess the vaccine's effectiveness against specific SARS-CoV-2 sublineages
- Information on Prior Infection: Misclassification of previous infections, particularly unreported or undocumented cases due to at-home testing or decreases in testing rates, potentially affecting the reliability of study results. Limited data on prior infections and high rates of prior SARS-CoV-2 infection may confound vaccine effectiveness estimates
- COMIRNATY Uptake: Low vaccination rates with COMIRNATY among participants, or vaccination rates with significant differences between groups (eg, elderly or individuals with comorbidities), may impact results, potentially impacting generalizability and resulting in underestimation of vaccine effectiveness
- Misclassification of Vaccination Status: There is a risk of errors in documenting previous COVID-19 vaccination status, which could affect study results
*Those who did not receive an XBB.1.5–adapted vaccine included participants who received wild-type boosters, variant-adapted vaccines (eg, BA.1 or BA.4/5 bivalent vaccines), or unvaccinated individuals.17
2023-2024 | Note: Due to high rates of prior infection, VE estimates are evaluated as an incremental or relative benefit beyond what individuals may have from prior infection(s), vaccination(s), or both (hybrid immunity).21
[Tartof SY, Slezak JM, Frankland TB, et al. Estimated effectiveness of the BNT162b2 XBB vaccine against COVID-19. JAMA Intern Med. 2024;184(8):932-940. doi:10.1001/jamainternmed.2024.1640]
[Tartof SY, Slezak JM, Puzniak L, et al. Effectiveness of BNT162b2 XBB vaccine against XBB and JN.1 sublineages. Open Forum Infect Dis. 2024;11(7):ofae370. doi:10.1093/ofid/ofae370]
2023-2024 | Note: Numerous studies and analyses have been conducted to evaluate the effectiveness of COVID-19 vaccines, yielding varying results across different populations, settings, and study designs. These variations may be attributed to differences in factors such as the timing of vaccine administration, the emergence of new viral variants, and demographic characteristics of study participants. Healthcare professionals should consider the context and specific conditions of individual studies when interpreting vaccine effectiveness data.8,10,11,17
Up next >
Dosing & Administration
Find links for ordering, customer service, and information on COVID-19.
CI=confidence interval; RCT=randomized controlled trial; RWE=real-world evidence.
Andersen KM, Ahi T, Mateus JS, et al. 2024-2025 BNT162b2 COVID-19 vaccine effectiveness in non-immunocompromised adults: mid-season estimates from vaccine registries in two states linked to administrative claims. Vaccine. 2025;62:127534. doi:10.1016/j.vaccine.2025.127534