Early Efficacy from Covid-19 Phase 3 Studies: Ethical, Operational, & Scientific Considerations

With time to address follow-up questions from the prior COVAX workshop on Sep 24

Clinical Development & Operations SWAT Team | Wednesday October 28, 2020 Meeting Norms and Recording Disclaimer

• Throughout the workshop, please ask any questions in the “Q&A” function. If you see that your question is already asked, you can “like” the question in the “Q&A” function.

• During the discussion sessions, please “Raise Your Hand” if you want to say something. If called on by the moderator, you will have the ability to unmute yourself.

• Please use the “Chat” function for any technology or logistical issues.

• This workshop will be recorded. Please be mindful of the diverse audience attending the meeting when participating in open discussions.

Privileged and confidential 2 Workshop Agenda

Time (CET) Topic Lead Speaker(s) 14:00 – 14:05 Welcome & Meeting Objectives Jakob Cramer Part 1: Follow-Up from Previous Workshop on Vaccine Efficacy (Sep 24, 2020) 14:05 – 14:20 Primary Efficacy Endpoints Edde Loeliger 14:20 – 14:30 Set of Symptoms that Triggers Diagnostic Work-Up of Suspected COVID-19 Cases Joan Capdevila Pujol Part 2: Group in Phase 3 Trials 14:30 – 14:35 Introduction – Global Phase 3 Studies and Emergency Use Overviews Peter Dull 14:35 – 14:50 Recent Experience with Novel in Placebo-Controlled Trials Peter Smith 14:50 – 15:00 Introduce Early Efficacy Scenarios for Discussion Peter Dull 15:00 – 15:10 Perspectives from an Indian Context Gagandeep Kang 15:10 – 15:20 Perspectives from a Brazilian Context Gustavo Santos 15:20 – 15:55 Panel Discussion: Scenario-based Discussion of Efficacy Results and Implications in Moderated by Peter Dull Different Trial Settings 15:55 – 16:05 Impact of an Efficacious Vaccine on COVID-19 Vaccine Trials Dean Follmann 16:05 – 16:15 Non-Inferiority Trial Design Considerations Martha Nason 16:15 – 16:25 Evaluating VAED in Phase III Beyond: Setting Realistic Expectations Steven Black 16:25 – 16:55 Panel Discussion: Operational, Regulatory, and Scientific Considerations Moderated by Jakob Cramer 16:55 – 17:00 Wrap Up and Next Steps Jakob Cramer Privileged and confidential 3 Welcome & Meeting Objectives Jakob Cramer, MD Head of Clinical Development (CEPI)

4 Context for today’s workshop

• Numerous developers are currently conducting Phase 3 trials for COVID-19 vaccines, with more developers preparing to start Phase 3 trials soon

• After the September 24 COVAX workshop on vaccine efficacy, there are a few follow-up questions to address on de- risking primary endpoints and clinical case workup

• Also, the first EUA for a COVID-19 vaccine could be issued in the United States by the end of the year

• After early efficacy results are released, developers may experience pressure to offer the active vaccine to the placebo group in ongoing Phase 3 trials

• It’s also unclear what approach developers should take for Phase 3 studies that have not started yet, in particular for countries where there are no national guidelines

• This topic is already being discussed in the media and developers have asked for guidance

Privileged and confidential 5 Primary Efficacy Endpoints Edde Loeliger, MD, MSc Clinical Development (CEPI)

6 Primary efficacy endpoints

COVID-19 Burden of Disease Endpoint & Vaccine-Attenuated Disease A. Edde Loeliger MD, MSc Measuring Vaccine Efficacy

• Vaccine efficacy can be assessed in individually-randomized efficacy trials by measuring 1. The efficacy to prevent disease 2. The efficacy to prevent 3. The efficacy to reduce disease severity in individuals with breakthrough disease 4. The efficacy to reduce infectiousness in individuals with breakthrough infection

• Endpoints in COVID-19 efficacy trials the above objectives include 1. The incidence of COVID-19; can be measured as COVID-19 irrespective of disease severity; as moderate to severe COVID-19; as severe COVID-19; as hospitalizations; ICU admissions; death etc. 2. Symptomatic and asymptomatic SARS-CoV-2 infection as measured indirectly by seroconversion of against antigens not included in the vaccine; directly by PCR or other NAAT 3. Burden of Disease of COVID-19 4. Viral shedding by measuring SARS-CoV-2 viral load in bodily fluids

8 The COVID-19 Vaccine Efficacy Conundrum

• “Ideally COVID-19 vaccines should prevent infection and therefore interrupt disease transmission; should that fail, they should reduce the likelihood of severe disease and hospital admission” (adapted from BMJ Editorial: "Will COVID-19 vaccines save lives? Current trials aren’t designed to tell us” 1) • COVID-19 vaccines, like other respiratory and mucosal virus vaccines, may not prevent infection per se • NHP challenge data for most vaccines in Phase 3 trials show only partial protection against infection2

• Primary efficacy endpoints in ongoing COVID-19 Phase 3 efficacy trials • COVID-19 (all symptomatic cases irrespective of disease severity ): ModeRNA, Pfizer/BioNTech, AstraZeneca/Oxford, CanSino, Sinovac. • Moderate-or-severe COVID-19: Janssen Vaccines/J&J (protocol-defined moderate and severe disease) • Both symptomatic COVID-19 irrespective-of-disease-severity, and moderate-or-severe COVID-19: Novavax

• The balancing act behind COVID-19 primary efficacy is larger trials versus the need for more COVID-19 cases • The low incidence of severe COVID-19 → need larger trials • Lower vaccine efficacy in preventing mild disease than severe disease → need more cases to reject H0

1 Doshi P. Will covid-19 vaccines save lives? Current trials aren’t designed to tell us BMJ 2020;371:m4037 doi: https://doi.org/10.1136/bmj.m4037 (Published 21 October 2020) 2 Krammer, F. et al. SARS-CoV-2 vaccines in development. Nature https://doi.org/10.1038/s41586-020-2798-3 (2020) 9 Vaccine-Attenuated Disease & Vaccine Efficacy

• STATEMENT: “Vaccines are often more successful in preventing severe disease than mild disease”

• MISCONCEPTION: “vaccines have a greater biological activity against severe disease than mild disease”

• FACT: Vaccine efficacy is almost always greater against severe disease than mild disease for vaccines associated with attenuated disease severity, in people with breakthrough disease → Vaccines almost always have lower efficacy in preventing mild disease than severe disease

• When vaccines cannot prevent infection per se, typically, VE against critical disease > severe disease > moderate disease > mild disease.

• The gradual lowering of VE estimated is caused by the accumulation of clinically-symptomatic breakthrough disease in vaccinated persons → lower efficacy against mild disease is driven by vaccine-attenuated disease (VAD) cases

10 Distribution of COVID-19 by disease severity

Critical

Severe

Critical

Severe Moderate Moderate

Mild C-19 Mild C-19

PLACEBO ARM VACCINE ARM

11 Hypothetical ‘black-box’ visualization of impact of VAD on C-19 vaccine efficacy

C-19 cases Vaccine arm ‘’black-box” outcome C-19 disease severity Placebo arm Vaccine arm VE averted Critical Critical 1 n=10 Severe (=VAD) 1 Endpoint BOD score Cases (n) BOD score Cases (n) BOD score 100*(1-RR) n/n*100 Moderate (=VAD) 2 Critical C-19 4 10 40 1 4 90% 10% Mild (=VAD) 5

Severe C-19 3 20 60 3 9 85% 20% Asymptomatic 1

Moderate C-19 2 40 80 10 20 75% 70% Severe Severe 2

Mild C-19 1 60 60 30 30 50% 88% n=20 Moderate (=VAD) 4

Asymptomatic SARS-CoV-2 + 0 70 0 156 0 -123% NA Mild (=VAD) 10 Asymptomatic 4 Burden of Disease (BOD) 240 63 74% Moderate Moderate 4 Severe/critical 30 4 87% n=40 Mild (=VAD) 8 Moderate/severe/critical 70 14 80% Asymptomatic 28 Mild (n=60) Mild/moderate/severe/critical 130 44 66% Mild 7

All SARS-CoV-2 200 200 0% Asymptomatic 53 Asymptomatic Asymptomatic 70

12 Efficacy estimates – hypothetical example

30% 50%

13 Burden of Disease (BOD) Endpoint

• Developed in 1994 by Chang, Guess and Heyse as a new efficacy measure for prophylactic interventions that may affect both disease incidence and disease severity1 → for vaccines associated with VAD • Proposed by Devan Mehrotra et.al. as a composite endpoint incorporating COVID-19 incidence and severity for use in efficacy trails2 • Accepted as a primary endpoint in the prevention study4 • Used to support the label claim that Zostavax is indicated “for prevention of zoster and zoster-related post- herpetic ” to regulatory agencies5 • Could be used as an outcome measure for the prevention of COVID-19 and COVID-19-related pneumonia

1 Chang MN, Guess HA, Heyse JF. Reduction in burden of illness: a new efficacy measure for prevention trials. Stat Med 1994;13:1807-14 2 Devan V. Mehrotra et.al. Clinical Endpoints for Evaluating Efficacy in COVID-19 Vaccine Trials. Annals of Internal Medicine 0;0 [Epub ahead of print 22 October 2020]. Doi: https://doi.org/10.7326/M20-6169 3 Van Doremaelen et.al. ChAdOx1 nCoV-19 vaccine prevents SARS-CoV-2 pneumonia in rhesus macaques. Nature. 2020. [PMID: 32731258] doi:10.1038/s41586-020-2608-y 4 Oxman MN et.al A vaccine to prevent Herpes Zoster and in older adults. N. Engl. J Med 2005;352:2271-84 5 EMA 2006: Zostavax EPAR scientific discussion; available from https://www.ema.europa.eu/en/documents/scientific-discussion/zostavax-eparscientific-discussion_en.pdf (accessed 23 October 2020)

14 COVID-19 BOD Endpoint & Pneumonia

• COVID-19 disease severity categorization (WHO)1 • Mild COVID-19: symptomatic disease meeting the case definition for COVID-19 without evidence of viral pneumonia or hypoxia. • Moderate COVID-19: clinical signs of pneumonia (fever, cough, dyspnoea, fast breathing) but no signs of severe pneumonia • Why include moderate disease in a BOD endpoint score ? • The low incidence of severe COVID-19 limits the number of VAD cases that are of essence for BOD, making it too similar to an all-severity COVID-19 • Moderate disease if WHO-defined as clinical signs of pneumonia (fever, cough, dyspnoea, fast breathing) but no signs of severe pneumonia and occurs in up to 40% of symptomatic individuals • Allows for a more granular assessment of severity reduction • Prevention of COVID-19 related pneumonia is a clinically relevant outcome

1 WHO 2020: Clinical management of COVID-19; Interim guidance 27 May 2020; available from https://www.who.int/publications/i/item/clinicalmanagement-of-covid-19 (accessed 26 October 2020) 15 COVID-19 BOD Endpoint – Wrap Up

• The BOD endpoint to de-risk the primary efficacy endpoint • Insufficient cases of severe COVID-19 • Insufficient cases of COVID-19 cases if as a result of accumulation of VAD cases • Consider the BOD endpoint for a multiple primary endpoint approach • Differs from both a composite primary endpoint approach and a co-primary endpoint approach1 • Useful when demonstration of a effect on at least one of several primary endpoints is sufficient • As a dual primary endpoint, with the COVID-19 ‘irrespective-of-disease-severity’ endpoint • As a triple primary endpoint that includes, in addition, a ‘moderate-to-severe/critical’ COVID-19 endpoint • The statistical penalty for closely related multiple endpoints for COVID-19 is relatively small • Including moderate disease may be critical to its added value when used as a dual or triple primary endpoint • Moderate disease as defined by WHO by of pneumonia is present in up to 40% of individuals2 • Prevention of COVID-19 and COVID-19 related pneumonia is a clinically-relevant primary objective

1 FDA 2017: Multiple endpoints for clinical trials; draft guidance for industry; available from https://www.fda.gov/media/102657/download (accessed 26 October 2020) 16 2 WHO 2020: Clinical management of COVID-19; Interim guidance 27 May 2020; available from https://www.who.int/publications/i/item/clinicalmanagement-of-covid-19 (accessed 26 October 2020) Set of Symptoms that Triggers Diagnostic Work-Up of Suspected COVID-19 Cases Joan Capdevila Pujol, PhD Data Scientist (ZOE)

17 COVID Symptom Study

Set of symptoms that triggers diagnostic work- up of suspected COVID-19 cases

Interim analysis by a joint team from CEPI & ZOE & KCL

24 Sep 2020 Rationale

● In subjects enrolled in vaccine efficacy trials, ideally, all COVID-19 (C-19) symptoms should trigger case work-up, including PCR testing for C-19.

○ Indiscriminate PCR testing may overwhelm laboratory capacity.

● This study aims:

○ to quantify how individual and combinations of C-19 symptoms contribute to case finding in a community-based, prospective, observational cohort study.

○ to obtain sets of symptoms through a multi-objective optimisation on two conflicting objectives: recall and number of tests needed.

● Laboratory capacity can be taken into account when deciding which symptoms should trigger a case work-up. Recap - methods

Data selection from the COVID Symptom Study App:

● 105,123 newly symptomatic users have entered valid PCR results - positive or negative

○ 55% aged 18 - 49, 34% aged 50 - 65 and 11% aged 65+

○ 75% female and 25% male Terminology:

● Recall or Sensitivity: % of C-19 positive users who are correctly identified by a symptom or a combination of symptoms.

● Precision or PPV: % of users identified by a symptom or a combination of symptoms who are C-19 positive. Recap - previous results

3-day analysis 7-day analysis

Symptom Tests per Tests per Recall Precision Recall Precision combinations C-19 case5 C-19 case5

Respiratory 516/22390 693/26011 516/1154 (44.7%) 43 693/1202 (57.7%) 37 symptoms1 (2.3%) (2.7%) WHO defined 679/34726 845/38979 679/1154 (58.8%) 51 845/1202 (70.3%) 46 pneumonia2 (1.9%) (2.2%) C-19-specific 778/37259 974/41658 778/1154 (67.4%) 47 974/1202 (81.0%) 42 symptoms3 (2.1%) (2.3%) Extended 1047/1154 1047/90547 1148/1202 1148/94447 86 83 symptoms4 (90.7%) (1.2%) (95.5%) (1.2%)

1Cough, dyspnoea; 2Cough, dyspnoea, fever; 3Fever, cough, dyspnoea, and anosmia/ageusia; 4Fever, cough, dyspnoea, anosmia/ageusia, fatigue, and ; 5The numbers of PCR tests needed to identify one PCR+ C-19 case Outline

- Rationale

- Recap previous results

- Result consistency across

- Age groups

- UK and US cohorts

- Multi-objective optimisation: learn triggering symptoms from the data

- Generalisation

- Most frequently selected symptoms

- Conclusions Results by age group (18-54, 55+)

3-day analysis 7-day analysis Tests per Symptom Age Tests per Recall Precision Recall Precision C-19 combinations group C-19 case5 case5 614/902 614/25873 772/942 772/29135 18 - 54 42 37 C-19-specific (68.1%) (2.4%) (82.0%) (2.6%) symptoms3 164/252 164/11386 202/260 202/12523 55+ 69 62 (65.1%) (1.4%) (77.7%) (1.6%) 823/902 823/61972 900/942 900/64881 18 - 54 75 72 Extended (91.2%) (1.3%) (95.5%) (1.4%) symptoms4 224/252 224/28575 248/260 248/39439 55+ 128 119 (88.9%) (0.8%) (95.4%) (0.8%)

3Fever, cough, dyspnoea, and anosmia/ageusia; 4Fever, cough, dyspnoea, anosmia/ageusia, fatigue, and headache; 5The numbers of PCR tests needed to identify one PCR+ C-19 case Result by cohort (UK/US)

The UK cohort has been regularly invited for testing thanks to the DHSC testing programme, while the US cohort has not. 3-day analysis 7-day analysis Tests per Symptom Tests per Cohort Recall Precision Recall Precision C-19 combinations C-19 case5 case5 778/1154 778/37259 974/1202 974/41658 UK 47 42 C-19-specific (67.4%) (2.1%) (81.0%) (2.3%) symptoms3 63/79 63/1142 73/79 73/1614 US 23 22 (79.7%) (4.4%) (92.4%) (4.5%) 1047/1154 1047/90547 1148/1202 1148/94447 UK 86 83 Extended (90.7%) (1.2%) (95.5%) (1.2%) symptoms4 76/79 76/2640 78/79 78/2745 US 35 35 (96.2%) (2.9%) (98.7%) (2.8%)

3Fever, cough, dyspnoea, and anosmia/ageusia; 4Fever, cough, dyspnoea, anosmia/ageusia, fatigue, and headache; 5The numbers of PCR tests needed to identify one PCR+ C-19 case Multi-objective optimization

Recall ~ 97.5% # test ~ 89.0%

Recall ~ 94.6% 3 days # test ~ 77.5% symptoms

Each point has a different trade-off between recall and % tested population 7 days symptoms Most frequently selected symptoms for solutions >90% recall

● 3 days scenario ○ Fatigue, loss of smell, persistent cough, diarrhea, headache and sore throat

● 7 days scenario ○ Fatigue, loss of smell, persistent cough, sore throat, fever and unusual muscle Overall results

3-day analysis 7-day analysis Tests per Symptom Tests per Recall Precision C-19 Recall Precision combinations C-19 case5 case5 Respiratory 516/1154 516/22390 693/1202 693/26011 43 37 symptoms1 (44.7%) (2.3%) (57.7%) (2.7%) WHO defined 679/1154 679/34726 845/1202 845/38979 51 46 pneumonia2 (58.8%) (1.9%) (70.3%) (2.2%) C-19-specific 778/1154 778/37259 974/1202 974/41658 47 42 symptoms3 (67.4%) (2.1%) (81.0%) (2.3%) Extended 1047/1154 1047/90547 1148/1202 1148/94447 86 83 symptoms4 (90.7%) (1.2%) (95.5%) (1.2%) Most frequently 1097/1154 1097/107748 1161/1202 1161/109784 98 83 selected symptoms (95.1%) (1.0%) (96.6%) (1.2%)

1Cough, dyspnoea; 2Cough, dyspnoea, fever; 3Fever, cough, dyspnoea, and anosmia/ageusia; 4Fever, cough, dyspnoea, anosmia/ageusia, fatigue, and headache; 5The numbers of PCR tests needed to identify one PCR+ C-19 case 6 (3-day) fatigue, loss of smell, persistent cough, diarrhea, headache and sore throat, (7-day) fatigue, loss of smell, persistent cough, sore throat, fever and unusual muscle pains Conclusions (I)

● Pneumonia symptoms are overall not very predictive of positive PCR result by themself and they do better when combined with anosmia/ageusia - classic symptoms.

● 32.6% of the positive cases do not show classic symptoms during the first three days of symptoms and 19%, not even during the first week.

● An extended list of symptoms, which includes the classic symptoms plus fatigue and headache, increases recall above 90% despite doubling the number of test required.

● These findings were shown to be consistent across two age groups (18 - 54, 55+) and across the UK and US cohorts. Conclusions (II)

● We also presented an optimisation method to learn sets of triggering symptoms from the data which takes into account the trade-off between recall and number of required tests.

● We showed that solutions with a high recall tend to include most of the extended symptoms, but we noticed the following differences:

○ Headache is more likely to be selected during the first 3 days but Fever tends to be more selected during the 7 days scenario

○ Shortness of breath tends to never be selected, suggesting it might occur later in the disease or co-occur with other selected symptoms Introduction: Global Phase 3 Studies and Emergency Use Overviews Peter Dull, MD Deputy Director, Integrated Clinical Vaccine Development (BMGF)

30 Key objectives for Part 2 of the workshop

• Review key historical examples of placebo-controlled vaccine studies and relevance for COVID-19

• Discuss country- and vaccine-specific scenarios with early efficacy results o Hear perspectives from countries facing important decisions about managing early efficacy results in the regional context (India and Brazil) o Hear perspectives on scientific, regulatory, and operational considerations if the placebo groups can no longer be enrolled or require vaccination

Privileged and confidential 31 Additional context – Why does this matter?

• Multiple phase 3 efficacy placebo-controlled studies for "Wave 1" Covid-19 vaccines have initiated, many of which are multi-country studies

• Further vaccines are in development and these "Wave 2" vaccines may have product characteristics which are critical for global impact (e.g., higher efficacy, better tolerability profiles, scalability, impact on shedding, population sub-groups)

• Immune correlates recognized as clearly critical but information very limited at present o If correlate identifiable, likely will occur only after initial efficacy studies completed and may be platform-specific

• On October 22 at the VRBPAC meeting, US FDA stated that “availability of a licensed vaccine does not automatically preclude continuation of blinded, placebo-controlled trials” o Also stated that “continuation of placebo-controlled follow-up after EUA will be critical to ensure that additional safety and effectiveness data are accrued to support submission of a licensure application as soon as possible following an EUA”

Privileged and confidential 32 Recent Experience with Novel Vaccines in Placebo-Controlled Trials Peter Smith Professor of Tropical (London School of Hygiene & Tropical Medicine)

33 Recent experience with novel vaccines in placebo-controlled trials

Peter Smith Participation is a should be driven by altruism

• Vaccine trials are not conducted for the benefit of those in the trial. • Participants may benefit from better care, and have a chance of receiving the vaccine, which might be beneficial, ineffective or cause harm. • Participants should not be disadvantaged compared to those in the same community not included in the trial. • Nor should they be unduly advantaged. Key questions in a pivotal vaccine trial

• Is the vaccine safe? • Is the vaccine efficacious? • How long does protection last (and, if necessary, can it be prolonged by booster doses)?

Addressing these questions generally requires a comparator group who have not been vaccinated.

• Do the results justify “licensure”? • Will the results convince public health authorities to deploy the vaccine (and potential recipients to be take it up) Early examples of different approaches to unvaccinated group MRC trial of BCG vaccine (1950-52) • Participants randomised at age 14-15y • Unvaccinated group never vaccinated • Enabled efficacy to be evaluated over 20+ years • Short-term results (VE >80%) provided basis for policy for BCG vaccination in schools at age ~13y.

MRC trial of live vaccine (1964) • Participants aged 10mo-2y • Efficacy shown to be 85% in first 9 months • All unvaccinated offered vaccine at 9 months, as originally promised • About 20% declined vaccination (had similar measles incidence as those ineligible for the trial) • Follow-up for further 2 years showed continuing high efficacy Recent examples of placebo group preserved beyond demonstration of efficacy Rotavirus vaccines • Protection against severe rotavirus gastroenteritis demonstrated in first year of follow-up • Placebo group maintained to measure protection in second year.

Dengvaxia • Primary analysis showed protection against dengue in 1st year after last dose • Follow-up, with placebo group, continued for 5 years • Evidence of vaccine enhanced disease in a sub-group found only from 2nd year after last dose.

RTS,S/AS01 • Vaccine efficacy against primary endpoint after one-year of follow-up • Placebo groups maintained to access decline of protection with time and impact of a booster dose. Apparent rebound effect of severe malaria found with extended follow-up, indicating importance of booster dose. Situations in which use of a placebo in vaccine trials may be justifiable when there is already an efficacious vaccine

1. Developing a locally affordable vaccine (e.g. Rotavac in India) 2. Evaluating the local safety and efficacy of an existing vaccine (e.g. Rotarix and Rotateq in Africa) 3. Testing a new vaccine when an existing vaccine is not yet in local use (e.g. Rotasil in Niger) – likely to be relevant for COVID-19 vaccines 4. Determining the local burden of disease (e.g. vaccine-probe studies – Hib) Head-to-head vaccine comparisons with clinical outcome

Such comparison are rarely done, mainly because of the large study sizes required to establish non-inferiority or superiority. However, some exceptions:

Relative efficacy of 1 or 2 dose AS03-Adjuvanted A(H1N1) reduced dose vaccine vs. 2-dose Non- Adjuvanted A(H1N1) (2010-11) • 6000 children (6mo-10y) randomised between 3-arms followed for 1y for H1N1 • Fewer cases observed (total 23) than expected in 3rd pandemic wave • Relative efficacy 2 dose adjuvanted vs non-adjuvanted = 77% (95%CI 18%-93%) Early Efficacy Scenarios for Discussion Peter Dull, MD Deputy Director, Integrated Clinical Vaccine Development (BMGF)

41 What is the impact on i) ongoing or ii) planned new Ph3 placebo-controlled vaccine efficacy trials in case …

Trial site country: levels evidence / action

… a vaccine’s efficacy has been demonstrated per interim analysis?

… a vaccine has been approved*, supply not yet available?

… a vaccine has been approved* and some supply available (however, no formal recommendations for selected risk populations in place yet)?

… a vaccine has been approved*, some supply available only to cover official recommendation for high-risk populations?

… a vaccine has been approved*, recommended for high-risk populations, and sufficient supply available for routine use in the full adult population?

42 *Approved according to national requirements via emergency use procedure or full licensure Privileged and confidential Specific efficacy trial scenarios for discussion during today’s workshop

Vaccine A Scenarios Vaccine B Scenarios

Vaccine A demonstrates efficacy at interim analysis and sponsor initiates application for emergency use, all within the same country*

1 NRA has not approved Vaccine A for emergency use or 3a full licensure Vaccine B has ongoing Phase 3 trial in the same country as approved vaccine A 2 NRA has approved Vaccine A for emergency use or full licensure but supply only sufficient for high-risk populations 3b Vaccine B has planned Phase 3 trial in the same country as approved vaccine A

*For simplicity, presume efficacy supported in all age groups (adults and elderly); At any time, subjects may exit study without penalty and receive standard-of-care interventions. Sponsor is required to update informed consent form as vaccine risk-benefit is modified Privileged and confidential 43 Vaccine A Scenarios Vaccine B Scenarios

Vaccine A demonstrates efficacy at interim analysis and sponsor initiates application for Perspectives emergency use, all within the same country* from an Indian 1 Context NRA has not approved Vaccine A for emergency 3a use or full licensure Gagandeep Kang, MD, Vaccine B has ongoing Phase 3 trial in the same country as PhD approved vaccine A Professor of 2 NRA has approved Vaccine A for emergency use Microbiology or full licensure but supply only sufficient for high-risk populations (CMC Vellore) 3b Vaccine B has planned Phase 3 trial in the same country as approved vaccine A

44 Vaccine A Scenarios Vaccine B Scenarios

Vaccine A demonstrates efficacy at interim analysis and sponsor initiates application for Perspectives emergency use, all within the same country* from a Brazilian 1 Context NRA has not approved Vaccine A for emergency 3a use or full licensure Gustavo Mendes Lima Vaccine B has ongoing Phase 3 trial in the same country as Santos approved vaccine A General Manager of 2 NRA has approved Vaccine A for emergency use Medicines and Biological or full licensure but supply only sufficient for high-risk populations Products 3b (ANVISA) Vaccine B has planned Phase 3 trial in the same country as approved vaccine A

45 Panel Discussion Scenario-based Discussion of Efficacy Results and Implications in Different Trial Settings

46 Discussion Panel Members and Example Questions

Panel Members Potential Discussion Questions

• Peter Smith, Professor of Tropical 1. Does it change the approach if approval is for emergency use, accelerated / Epidemiology, LSHTM conditional, or full licensure?

• Gagandeep Kang, Professor of 2. What is the effect of an approval by an NRA outside the country where the trial is being Microbiology (CMC Vellore) conducted of a multi-regional study? Of a separate study using the same vaccine?

• Gustavo Mendes Lima Santos, 3. What if the trial vaccine is approved but supply is not yet available – should the trial General Manager of Medicines and vaccine be offered to the placebo group? Biological Products, ANVISA 4. What if a different (non-trial) vaccine is approved and supply is available with • Ross Upshur, Professor (University of formal recommendation in high-risk populations – should vaccination be offered to Toronto) and Co-Chair (WHO COVID-19 respective participants in both the intervention and placebo groups? Ethics Working Group)

Privileged and confidential 47 Impact of an Efficacious Vaccine on COVID-19 Vaccine Trials Dean Follmann, PhD CoVPN Statistics Group Chief, Biostatistics Research Branch (NIH / NIAID)

48 Setup

• Trial of Vaccine A shows efficacy

• Try to maintain blinded follow-up to assess vaccine durability & vaccine associated enhanced disease (VAED) o US FDA guidance strongly encourages continued follow-up past EUA to support BLA

• At some point, vaccine may become widely available o Placebo volunteers may cross-over to receive vaccine A o Ideal to continue follow-up to assess immune correlates of protection and vaccine durability

Privileged and confidential 49 Immune Correlates

• Assess whether after last dose predicts disease acquisition Dengvaxia

• Goal: Estimate an antibody level that gives very high VE

• Correlate allows licensure or bridging of same/similar platforms using small immunogenicity studies

Illustration: Ab ≥ 103 supportive of licensure

Gilbert et al., 2018. Neutralizing Antibody Correlates Analysis of Tetravalent Dengue Vaccine Efficacy Trials in Asia and Latin America.Privileged and confidential 50 Measure Immune Response in the Placebo Crossovers

• When vaccine shows efficacy, may be relatively few disease cases on vaccine arm

• Correlates analysis at time of efficacy readout may be unconvincing/underpowered

• Can double the sample size for correlates by measuring antibody in the placebo crossovers

Placebo Crossover

Vaccine

Blood draws to measure vaccine induced antibodies

Privileged and confidential 51 Randomized Trial of Vaccine vs Placebo . . . Becomes Rebranded

Randomized Trial of Vaccine vs Placebo Randomized Trial of Immediate vs. Delayed Vaccination

Placebos Crossover to Vaccine

# cases # cases # cases Randomized to August-November Randomized to August-November December-March Placebo Placebo 125a 125a 25b Delayed Vaccine Vaccine Vaccine 25b 25b 25b Immediate Vaccine

Vaccine Efficacy 80% Vaccine Efficacy 80%

(a) # cases on placebo (b) # cases on vaccine Privileged and confidential 52 Following placebo crossover, can collect data on durability

Placebos Crossover to Vaccine

Highly durable Poorly durable # cases vaccine vaccine Randomized to August-November # cases # cases December-March December-March

Placebo 125a 25b 25b 0-4 months post Delayed Vaccine vaccination

Vaccine Immediate 25b 25b 50b 4-8 months post Vaccine vaccination

Vaccine Efficacy 80%

If more cases in original vaccine arm post crossover, vaccine efficacy is waning

(a) # cases on placebo (b) # cases on vaccine Privileged and confidential 53 Summary

• If a vaccine trial shows efficacy, continue blinded follow-up as long as possible to assess durability and VAED

• If placebo crossover to vaccine does occur maintain follow-up o Still have a randomized trial o Doubles sample size for immune correlates analysis o Can still assess vaccine durability, though less reliably than if no crossover

Privileged and confidential 54 Non-Inferiority Trial Design Considerations Martha Nason, PhD Mathematical Statistician, Biostatistics Research Branch (NIH / NIAID)

55 When do we need a non-inferiority trial?

▪ Non-inferiority trial necessary when it is no longer considered ethical to randomize participants to Placebo

▪ Once another vaccine has demonstrated efficacy and is available to the population being studied. This decision may be different in different locations, different subpopulations (priority groups)

▪ Strong scientific arguments to continue Placebo group as long as possible to collect data on safety, durability (recent discussions with WHO, FDA’s VRBPAC)

Privileged and confidential 56 Non-inferiority trials: setup

Sometimes referred to as “equivalence trials”

Can’t ever prove two vaccines are the same. Instead show that: • New vaccine is better than established vaccine (superiority) Or • New vaccine is at least “not much worse” than established vaccine (non-inferiority)

Need to define how much worse is too much, and how much worse might be acceptable • This is the non-inferiority margin • "Typically" this is 10% and requires justification by the sponsor

Privileged and confidential 57 Choice of margin

FDA initial guidance:

For non-inferiority comparison to a COVID-19 vaccine already proven to be effective, the statistical success criterion should be that the lower bound of the appropriately alpha-adjusted confidence interval around the primary relative efficacy point estimate is > -10 %.

Assume vaccine A is established, vaccine B is new candidate:

if VE(A)=50% then must show VE(B)> 45% if VE(A)=60% then must show VE(B)> 56% 푟푎푡푒(푉푎푐푐 퐵) < 1.10 = 푀푎푟푔푖푛 if VE(A)=70% then must show VE(B)> 67% 푟푎푡푒(푉푎푐푐 퐴) if VE(A)=80% then must show VE(B)> 78% if VE(A)=90% then must show VE(B)> 89%

Privileged and confidential 58 Sample Size calculations – Vaccine B comparable or better than Vaccine A

Approx Person-years VE in VE to rule NI margin True VE in # years # infections assuming 1% “successful” out for new (RR new vaccine Assuming needed incidence in vaccine (A) vaccine (B) (B) n=30,000 scale) unvaccinated enrolled

60% 4,660 1,160,000 39 60% 56% 10% 70% 293 82,000 2.8

60% 857 214,000 7.1 60% 50% 25% 70% 164 45,900 1.5

Privileged and confidential 59 Scenario Discussion – Late arriving vaccine required to demonstrate non-inferiority:

Vaccine A Observed Efficacy: VEA = 50%

Want to show RRB:A < 1.2 (i.e. VEB > 40%)

Vaccine B Hypothesized Efficacy: VEB = 60%

Need 256 events under these assumptions, 90% power

If 1% incidence without vaccine, n=30,000 people enrolled → 2 years to 256 events Scenario Discussion – Late arriving vaccine required to demonstrate non-inferiority:

Suppose:  Vaccine A shows efficacy with 50% Vaccine Efficacy, rules out VE<30%  Vaccine B looks promising, high neutralizing titers and high scalability potential.

Potential Course of Action: • Enroll n=30,000 people into a non-inferiority study of vaccine A vs B • At 1 month post 2nd dose, compare Neut titers between vaccines A and B in a small predefined subset. • If Neut titers to Vaccine B are shown to be non-inferior (or superior) to Neut titers to vaccine B, share data with regulators as confidence builds around correlates of protection • Request “accelerated/conditional" approval, depending on setting • Continue to follow n=30,000 people for symptomatic disease, clinical non-inferiority for ~2 years

Privileged and confidential 61 Conclusions

• Defining the margin is central to planning any non-inferiority trial • Suggest discussion among stakeholders now about the appropriate margin • Must think about margin on both VE and relative risk scale

• Non-inferiority takes very large sample size (or many years) if vaccines are truly equivalent

• If VE(A)=60% and VE(B)=70%, demonstration of non-inferiority would take 2-3 years*

*assuming n=30,000 and 1% incidencePrivileged in unvaccinated and confidential 62 Acknowledgements

• Dean Follmann • Michael Fay • Yunda Huang • David Benkasser • And the rest of the OWS stats group

Privileged and confidential 63 Backup: Sample Size calculations – Vaccine B comparable or better than Vaccine A Additional scenarios ---- Fixed non-inferiority margin at 10%

Approx VE to Non- Person-years # years VE in rule out True VE in # inferiority assuming 1% follow-up “successful” for new new infections Margin incidence in on vaccine (A) vaccine vaccine (B) needed On RR unvaccinated n=30,000 (B)

50% 4660 931,000 31 50% 45% 10% 60% 426 93,500 3.1 70% 119 28,000 1 60% 4660 1,160,000 39 60% 56% 10% 70% 293 82,000 2.8 80% 75 22,500 0.8 70% 4660 1,550,000 52 70% 67% 10% 80% 173 66,500 2.2 90% 33 13,200 0.5

Privileged and confidential 64 Backup: Sample Size calculations – Vaccine B comparable or better than Vaccine A Additional scenarios ---- Increasingly wide non-inferiority margin

Approx VE to Non- Person-years # years VE in rule out True VE in inferiority # infections assuming 1% follow-up “successful” for new new vaccine Margin needed incidence in on vaccine (A) vaccine (B) On RR unvaccinated n=30,000 (B)

50% 1270 255,000 8.5 50% 40% 20% 60% 257 56,400 1.9 70% 92 21,600 0.9 60% 857 214,000 7.1 60% 50% 25% 70% 164 45,900 1.5 80% 54 16,200 0.5 70% 512 171,000 5.7 70% 60% 33% 80% 91 35,000 1.2 90% 26 10,400 0.4

Privileged and confidential 65 Evaluating VAED in Phase III Beyond: Setting Realistic Expectations Steven Black, MD Executive Board SPEAC Project (Safety Platform for Emergency vACcines)

66 Overview

• What is meant by enhanced disease and why would be worry about it?

• Vaccine failures versus VAED

• Outline what data should be collected on all vaccine failures

• A possible approach to evaluation What is VAED and Why are We Talking about it

• Simply, Vaccine Associated Enhanced Disease occurs when a vaccine recipient is exposed to the wild type virus and experiences more severe disease than they would have if they had not been vaccinated.

• Brighton Collaboration has developed a case definition with much more detail. https://brightoncollaboration.us/vaed/

• In some studies in animal models of other corona virus vaccines, vaccinated animals experienced more severe disease when exposed to the wild type virus

• This phenomenon has been observed for other vaccines including vaccines developed for measles, RSV and dengue. Vaccine Failure versus Enhanced Disease

• No vaccine is 100% effective so that vaccine failures will occur.

• By definition, all cases of enhanced disease would occur in vaccinees and hence be a vaccine failure.

• Without a specific biomarker for enhanced disease, it is not currently possible to differentiate between • A vaccine failure with severe disease • A vaccine failure with more severe disease than they otherwise would have had without vaccine --- enhanced disease

• So what to do? Evaluating VAED in phase 2-3 clinical trials

• This will require evaluating all vaccine failures and COVID disease patients in the control group in a blinded manner using a standard protocol which includes a standardized severity assessment score in all vaccine failures in the trial.

• One would then compare the average severity score in per protocol cases of COVID in the vaccine and placebo groups.

• Limiting factors include • VAED, if it exists for COVID vaccines, is likely to be uncommon. • Importantly, the risk of VAED may increase as the immune response wanes with time --- follow up may be too short in phase III trials. Cross over designs and safety outcomes

Placebo Cross Over VACCINE

Safety outcomes here have a Safety outcomes here have NO comparison group comparison group except initial control time period

If events occur here assessing causality is problematic

Duration of comparative Safety follow up is short With blinded cross over, assessment of cases is blinded but one no longer has an unvaccinated group for comparison

71 Practical Considerations for Assessing VAED in Large Phase III trials

• If a trial of 30,000 participants randomized 1:1 has accrued 100 cases of COVID and the vaccine’s true efficacy is 75%, one would expect 80 COVID cases in controls and 20 in the vaccine group.

• If VAED occurs in 10% of vaccine failures, one would only expect to see two such cases in vaccine recipients

• If VAED occurs in 5% of cases, one would only expect one such case

• Thus even in a large phase III trial, there is not sufficient power to detect VAED even it is relatively common.

• This situation is made worse if VAED risk occurs only after a long lag period in which case it could be occurring after the trial follow up is complete or in a cross over trial after the comparison group has been vaccinated. Evaluating VAED after vaccine introduction.

• This will be very challenging unless • The rate of VAED is high • The severity and disease characteristics in VAED cases are different than routine COVID cases • A biomarker is identified

• Similar to the case in phase III trials, one would want to establish a registry of all vaccine failures with all the cases evaluated using a common protocol and standardized severity score.

• Since there would be no control group in this case, one would have to compare vaccinated and unvaccinated COVID cases adjusting for co-morbidities, therapies given, age, etc. Summary

• There is a possibility that VAED may occur after some COVID vaccines

• Risk might be higher for unadjuvanted inactivated vaccines that develop a Th2 oriented response.

• There is no biomarker for VAED so that it is not currently possible to differentiate a vaccine failure from a vaccine failure with enhanced disease on an individual level.

• Assessment of all vaccine failures using a standardized protocol and severity score is critical. Panel Discussion Operational, Regulatory, and Scientific Considerations

75 Discussion Panel Members and Potential Questions

Panel Members Potential Discussion Questions

• Dean Follmann, Chief, Biostatistics Research 1. How do we help additional vaccines move through licensure if Branch (NIH/NIAID) we've lost the placebo arm and non-inferiority studies are not feasible? • Martha Nason, Mathematical Statistician, Biostatistics Research Branch (NIH/NIAID) 2. What kind of arguments are required to define the non-inferiority margin for vaccine efficacy studies? • Steven Black, Executive Board Member (SPEAC) 3. What have we learned so far about risk of vaccine-associated enhanced disease from either more recent animal studies or • Marco Cavaleri, Head of Biological Health ongoing clinical trials? Threats and Vaccines Strategy (EMA) 4. From review of the publicly available study protocols, how are we • Philip Krause, Deputy Director (US FDA / doing at characterizing Covid-19 disease cases in ongoing and CBER / OVRR) planned studies?

Privileged and confidential 76 Wrap Up & Next Steps Jakob Cramer Head of Clinical Development (CEPI)

77 Closing remarks

• Thank you all for your participation and engagement today

• The COVAX Clinical SWAT Team plans to continue sharing learnings across developers as we pursue our common goal – a global supply of safe and effective vaccines

• We will be holding a next workshop on advances toward identifying Immune Correlates of Protection (CoP), targeted for late November / early December

• We will continue to share resources at the website here: https://epi.tghn.org/covax-overview/clinical/

• We will distribute a workshop report to summarize today’s conversation and a post-workshop survey to collect feedback

Privileged and confidential 78 Clinical Development & Operations SWAT Team

79