Testimony in SUPPORT of HB-7199: An Act Concerning Immunizations Against the Meningococcal Virus and Human Papillomavirus Hannah G. Rosenblum, MD March 13, 2019

Dear Co-Chairs Representative Steinberg, Senator Abrams and Distinguished Members of the Public Health Committee:

As an internist and a pediatrician at Yale-New Haven Hospital and St. Mary's Hospital, I care for both adults and children in the hospital and in the office. My patients are all ages, from newborns to the elderly, and I also care for patients affected by cancer in my regular practice. I am a resident of New Haven, Connecticut and I am writing to testify about why I strongly SUPPORT HB-7199, an act that concerns prevention of cancer and prevention of meningitis.

My testimony focuses on the critical public health issue of disease prevention, and addressing misinformation about vaccinations. Please see the attached addendum for several rigorous scientific studies in reputable high-impact journals that support the safety, efficacy and impact of the life-saving immunization against human papilloma virus (HPV).

We Connecticut pediatricians are familiar with the routine schedule of vaccinations and with counseling parents about protecting their kids against illnesses that used to harm children early in life (to name a few: diseases like diphtheria, polio, haemophilus influenza, and measles). Because these critical immunizations given in early childhood are required to enter Connecticut public schools, I have not seen children harmed, maimed or killed by those infectious diseases of the past.

However, human papilloma virus (HPV) is a widespread infectious disease of our time and I am all too familiar with HPV’s effect on my patient’s health. Transmitted through skin-to-skin contact, nearly all men and women are exposed to HPV at some point in their lives. High risk types cause cervical, other genital cancers and head and neck cancer.

One young primary care patient of mine was diagnosed with oropharyngeal cancer; were the available in her youth, her life would have been drastically different. Admitted to Smilow Cancer Center for debilitating jaw and mouth pain, she was on a strictly liquid diet. The HPV-positive tumor that had started in her sinus and nose had eroded into her sensitive facial bones. Multiple rounds of chemotherapy, radiation and surgeries left her with profound difficulty chewing and swallowing, not to mention constant pain and social anxiety due to disfigurement. Her speech was barely intelligible to her daughter, who remained faithful at her bedside all day and night. She is one of the lucky ones- her malignancy has not taken her life!

As an internist, I have cared for countless adult patients who have died from cancers that were not preventable. But for patients with cervical cancer, some head and neck, and many genital cancers, prevention is now the reality!

The essential vaccine that was explicitly developed to prevent cancer: the vaccine against human papilloma virus (HPV) has been greatly underutilized in our state of Connecticut. Though recommended by the ACIP and CDC, this is because the HPV vaccine has not made it to the list of immunizations required for public school and therefore, the burden is currently on parents to make sure their children receive it. In my practice, parents frequently interpret this vaccine as “not important” because it is not expected for school entry. This bill will not Testimony in SUPPORT of HB-7199: An Act Concerning Immunizations Against the Meningococcal Virus and Human Papillomavirus Hannah G. Rosenblum, MD March 13, 2019 remove parental consent in decision-making about their children- it will eradicate HPV, just as we’ve eradicated polio and other deadly diseases.

This bill is so critical to the health of our state’s children and adults because it makes concordant the medical recommendations for nationwide vaccination, pediatrician recommendations and school requirements.

We internists recommend regular mammograms to detect early breast cancer, regular colonoscopies to detect colon cancer, regular Pap smears to detect early abnormalities in the cervix. Rarely do we have the opportunity to actually prevent cancer. And just as we recommend patients use seatbelts to protect against injury or death in the case of a car accident, the HPV vaccine protects a patient in the case of exposure to HPV virus. Let’s make the next generation of physicians and patients unfamiliar with HPV-related cancers. As data from Australia shows, this can be even be done in one-generation!

As a doctor that counsels patients, and an advocate for protecting children, I know that there is a lot of misinformation circulating about vaccinations. In fact, the WHO (World Health Organization) has named “” a top ten threat to global health in 2019, joining such public health crises such as the Ebola epidemic and climate change. There is also an ongoing national conversation about the role of social media in promoting anti-science lies, prompting Google, Facebook and Pinterest to take action.

Key FACTs about the HPV vaccine are addressed in the attached articles:

1. It is safe & effective! 2. It is for BOTH boys and girls, and protects both against cancers of the cervix, vulva, vagina, penis, anus and oropharynx (back of the throat including base of the tongue and tonsils). 3. It does not promote earlier onset of sexual behavior in adolescents. 4. School-entry requirements are impactful in decreasing burden of HPV and cervical dysplasia.

Legislators: Thank you for using your position on the public health committee to address cancer prevention. I urge you to protect Connecticut's children and adults against cancer with this immunization against human papilloma virus. SUPPORT HB-7199.

Sincerely,

Hannah G. Rosenblum, MD

Testimony in SUPPORT of HB-7199: An Act Concerning Immunizations Against the Meningococcal Virus and Human Papillomavirus Hannah G. Rosenblum, MD March 13, 2019

Table of Contents for References Attached Below

1. Petrosky E, Bocchini Jr JA, Hariri S, Chesson H, Curtis CR, Saraiya M, Unger E, Markowitz L. Use of 9-Valent Human Papillomavirus (HPV) Vaccine: Updated HPV Vaccination Recommendations of the Advisory Committee on Immunization Practices. Morbidity and Mortality Weekly Report 2015; 64(11): 300-304. 2. Centers for Disease Control and Prevention: https://www.cdc.gov/vaccines/vpd/hpv/hcp/recommendations.html

3. D’Addario M, Redmond S, Scott P, Egli-Gany D, Riveros-Balta AX, Henao Restrepo AM, Low N. Two-dose schedules for human papillomavirus vaccine: Systematic review and meta-analysis. Vaccine 2017; 35, 2892-2901. 4. Luna J, Plata M, Gonzalez M, Correa A, Maldonado I, Nossa C, Radley D, Vuocolo S, Haupt RM, Saah A. Long-term Follow-up Observation of the Safety, Immunogenicity, and Effectiveness of Gardasil in Adult Women. PLoS ONE 2013; 8(12):e83431. 5. Markowitz LE, Liu G, Hariri S, Steinau M, Dunne EF, Unger ER. Prevalence of HPV After Introduction of the Vaccination Program in the United States. Pediatrics 2016; 137(2): e20151968. 6. Spinner C, Ding L, Bernstein DI, Brown DR, Franco EL, Covert C, Kahn JA. Human Papillomavirus Vaccine Effectiveness and Herd Protection in Young Women. Pediatrics. 2019; 143(2):e20181902. 7. Niccolai L, Meek JI, Brackney M, Hadler JL, Sosa LE, Weinberger DM. Declines in Human Papillomavirus (HPV)-Associated High-Grade Cervical Lesions After Introduction of HPV in Connecticut, United States, 2008-2015. Clinical Infectious Diseases 2017; 65(6):884-9. 8. Brisson M, Benard E, et al. Population-level impact, herd immunity, and elimination after human papillomavirus vaccination: a systematic review and meta-analysis of predictions from transmission-dynamic models. Lancet Public Health 2016; 1:e8-17.

9. Arnheim-Dahlstrom L, Pasternak B, Svanstrom H, Sparen P, Hviid A. Autoimmune, neurological, and venous thromboembolic adverse events after immunization of adolescent girls with quadrivalent human papillomavirus vaccine in Denmark and Sweden: cohort study. BMJ 2013; 347:f5906. 10. Brotherton JML, Fridman M, May C, Chappell G, Saville AM, Gertig DM. Early effect of the HPV vaccination programme on cervical abnormalities in Victoria, Australia: an ecological study. Lancet. 2011; 377: 2085-92.

Testimony in SUPPORT of HB-7199: An Act Concerning Immunizations Against the Meningococcal Virus and Human Papillomavirus Hannah G. Rosenblum, MD March 13, 2019

11. Machalek DM, Garland SM, et al. Very Low Prevalence of Vaccine Human Papillomavirus Types Among 18-35 Year Old Australian Women 9 Years Following Implementation of Vaccination. The Journal of Infectious Diseases. 2018; 217: 1590-600. 12. Arana JE, Harrington T et al. Post-licensure safety monitoring of quadrivalent human papillomavirus vaccine in the Reporting System (VAERS), 2009- 2015. Vaccine. 2018; 36: 1781-1788. 13. Naleway AL, Mittendorf KF, Irving SA, et al. Primary Ovarian Insufficiency and Adolescent Vaccination. Pediatrics. 2018; 142(3):e20180943.

14. Bednarczyk RA, Davis R, Ault K, Orenstein W, Omer S. Sexual Activity-Related Outcomes After Human Papillomavirus Vaccination of 11-to 12-Year Olds. Pediatrics. 2012; 130;798.

15. Barraza L, Weidenaar K, Campos-Outcalt D, Yang YT. Human Papillomavirus and Mandatory Immunization Laws: What Can We Learn From Early Mandates? Public Health Reports. 2016;131(5): 728-731. 16. Daley E, Thompson E, Zimet G. Human Papillomavirus Vaccination and School Entry Requirements. JAMA Pediatrics. 2019; 173(1);3327. 17. Thompson EL, Livingston MD, Daley EM, Zimet GD. Human Papillomavirus Vaccine Initiation for Adolescents Following Rhode Island’s School-Entry Requirement, 2010- 206. Am J Public Health. 2018; 108:1421-1423.

18. Broniatowski DA, Jamison AM, Qi S. Weaponized Health Communication: Twitter Bots and Russian Trolls Amplify the Vaccine Debate. Am J Public Health. 2018:108-1378- 1384. 19. Omer SB, Amin AB, Limaye RL. Communicating About Vaccines in a Fact-Resistant World. JAMA Pediatrics. 2017; 171 (10):929-930. Morbidity and Mortality Weekly Report

Use of 9-Valent Human Papillomavirus (HPV) Vaccine: Updated HPV Vaccination Recommendations of the Advisory Committee on Immunization Practices

Emiko Petrosky, MD1,2, Joseph A. Bocchini Jr, MD3, Susan Hariri, PhD2, Harrell Chesson, PhD2, C. Robinette Curtis, MD4, Mona Saraiya, MD5, Elizabeth R. Unger, PhD, MD6, Lauri E. Markowitz, MD2 (Author affiliations at end of text)

During its February 2015 meeting, the Advisory Committee (2vHPV), which contains HPV 16, 18 VLPs, is licensed for use on Immunization Practices (ACIP) recommended 9-valent in females (1). This report summarizes evidence considered by human papillomavirus (HPV) vaccine (9vHPV) (Gardasil 9, ACIP in recommending 9vHPV as one of three HPV vaccines Merck and Co., Inc.) as one of three HPV vaccines that can be that can be used for vaccination and provides recommenda- used for routine vaccination (Table 1). HPV vaccine is recom- tions for vaccine use. mended for routine vaccination at age 11 or 12 years (1). ACIP also recommends vaccination for females aged 13 through 26 Methods years and males aged 13 through 21 years not vaccinated previ- From October 2013 to February 2015, the ACIP HPV ously. Vaccination is also recommended through age 26 years Vaccine Work Group reviewed clinical trial data assessing the for men who have sex with men and for immunocompromised efficacy, immunogenicity, and safety of 9vHPV, modeling persons (including those with HIV infection) if not vaccinated data on cost-effectiveness of 9vHPV, and data on burden of previously (1). 9vHPV is a noninfectious, virus-like particle type-specific HPV-associated disease in the United States. (VLP) vaccine. Similar to quadrivalent HPV vaccine (4vHPV), Summaries of reviewed evidence and Work Group discussions 9vHPV contains HPV 6, 11, 16, and 18 VLPs. In addition, were presented to ACIP before recommendations were pro- 9vHPV contains HPV 31, 33, 45, 52, and 58 VLPs (2). 9vHPV posed. Recommendations were approved by ACIP in February was approved by the Food and Drug Administration (FDA) 2015. Evidence supporting 9vHPV use was evaluated using on December 10, 2014, for use in females aged 9 through the Grading of Recommendations, Assessment, Development, 26 years and males aged 9 through 15 years (3). For these rec- and Evaluation (GRADE) framework (5) and determined to ommendations, ACIP reviewed additional data on 9vHPV in be type 2 (moderate level of evidence) among females and 3 males aged 16 through 26 years (4). 9vHPV and 4vHPV are (low level of evidence) among males; the recommendation was licensed for use in females and males. Bivalent HPV vaccine categorized as a Category A recommendation (for all persons in an age- or risk-factor–based group) (6). Recommendations for routine use of vaccines in children, ado- lescents and adults are developed by the Advisory Committee on HPV-Associated Disease Immunization Practices (ACIP). ACIP is chartered as a federal HPV is associated with cervical, vulvar, and vaginal cancer in advisory committee to provide expert external advice and guidance females, penile cancer in males, and anal cancer and oropha- to the Director of the Centers for Disease Control and Prevention ryngeal cancer in both females and males (7–10). The burden (CDC) on use of vaccines and related agents for the control of of HPV infection also includes cervical precancers, including vaccine-preventable diseases in the civilian population of the United cervical intraepithelial neoplasia grade 2 or 3 and adenocar- States. Recommendations for routine use of vaccines in children cinoma in situ (≥CIN2). The majority of all HPV-associated and adolescents are harmonized to the greatest extent possible with cancers are caused by HPV 16 or 18, types targeted by 2vHPV, recommendations made by the American Academy of Pediatrics 2 11 12 (AAP), the American Academy of Family Physicians (AAFP), 4vHPV and 9vHPV ( , , ). In the United States, approxi- and the American College of Obstetricians and Gynecologists mately 64% of invasive HPV-associated cancers are attributable (ACOG). Recommendations for routine use of vaccines in adults to HPV 16 or 18 (65% for females; 63% for males; approxi- are harmonized with recommendations of AAFP, ACOG, and the mately 21,300 cases annually) and 10% are attributable to the American College of Physicians (ACP). ACIP recommendations five additional types in 9vHPV: HPV 31, 33, 45, 52, and 58 approved by the CDC Director become agency guidelines on the (14% for females; 4% for males; approximately 3,400 cases date published in the Morbidity and Mortality Weekly Report annually) (1,12,13). HPV 16 or 18 account for 66% and the (MMWR). Additional information about ACIP is available at five additional types for about 15% of cervical cancers (12). http://www.cdc.gov/vaccines/acip/. Approximately 50% of ≥CIN2 are caused by HPV 16 or 18

300 MMWR / March 27, 2015 / Vol. 64 / No. 11 Morbidity and Mortality Weekly Report

TABLE 1. Characteristics of the three human papillomavirus (HPV) vaccines licensed for use in the United States Characteristic Bivalent (2vHPV)* Quadrivalent (4vHPV)† 9-valent (9vHPV)§ Brand name Cervarix Gardasil Gardasil 9 VLPs 16, 18 6, 11, 16, 18 6, 11, 16, 18, 31, 33, 45, 52, 58 Manufacturer GlaxoSmithKline Merck and Co., Inc. Merck and Co., Inc. Manufacturing Trichoplusia ni insect cell line infected with L1 Saccharomyces cerevisiae (Baker’s yeast), Saccharomyces cerevisiae (Baker’s yeast), encoding recombinant baculovirus expressing L1 expressing L1 Adjuvant 500 µg aluminum hydroxide, 225 µg amorphous aluminum 500 µg amorphous aluminum 50 µg 3-O-desacyl-4’ monophosphoryl lipid A hydroxyphosphate sulfate hydroxyphosphate sulfate Volume per dose 0.5 ml 0.5 ml 0.5 ml Administration Intramuscular Intramuscular Intramuscular Abbreviation: L1 = the HPV major capsid protein; VLPs = virus-like particles. * Only licensed for use in females in the United States. Package insert available at http://www.fda.gov/downloads/BiologicsBloodVaccines/Vaccines/ApprovedProducts/ UCM186981.pdf. † Package insert available at http://www.fda.gov/downloads/BiologicsBloodVaccines/Vaccines/ApprovedProducts/UCM111263.pdf. § Package insert available at http://www.fda.gov/downloads/BiologicsBloodVaccines/Vaccines/ApprovedProducts/UCM426457.pdf.

and 25% by HPV 31, 33, 45, 52, or 58 (14). HPV 6 or 11 groups, and GMTs were noninferior in the 9vHPV group cause 90% of anogenital warts (condylomata) and most cases compared with the 4vHPV group. of recurrent respiratory papillomatosis (15). Immunogenicity in males aged 16 through 26 years was compared with females of the same age group in a separate 9vHPV Efficacy, Immunogenicity, and Safety study. In both females and males, >99% seroconverted to all In a phase III efficacy trial comparing 9vHPV with 4vHPV nine HPV vaccine types, and GMTs in males were noninferior among approximately 14,000 females aged 16 through 26 to those in females (4). years, 9vHPV efficacy for prevention of ≥CIN2, vulvar The immunogenicity of concomitant and nonconcomitant intraepithelial neoplasia grade 2 or 3, and vaginal intraepithelial administration of 9vHPV with quadrivalent meningococcal neoplasia grade 2 or 3 caused by HPV 31, 33, 45, 52, or 58 conjugate vaccine (Menactra, MenACWY-D) and tetanus, was 96.7% in the per protocol population* (Table 2) (2,16). diphtheria, acellular pertussis vaccine (Adacel, Tdap) was Efficacy for prevention of ≥CIN2 caused by HPV 31, 33, 45, evaluated. The GMTs were noninferior for all nine HPV 52, or 58 was 96.3% and for 6-month persistent infection was vaccine types in the co-administered group (all p<0.001). For 96.0% (16). Few cases were caused by HPV 6, 11, 16, or 18 in Menactra, the noninferiority criterion was met for all four either vaccine group. Noninferior immunogenicity of 9vHPV serogroups, and for Adacel, for diphtheria, tetanus, and all compared with 4vHPV was used to infer efficacy for HPV four pertussis antigens. 6, 11, 16, and 18. Geometric mean antibody titers (GMTs) Safety has been evaluated in approximately 15,000 subjects 1 month after the third dose were noninferior for HPV 6, 11, in the 9vHPV clinical development program; approximately 16, and 18; in the 9vHPV group, >99% seroconverted to all 13,000 subjects in six studies were included in the initial appli- nine HPV vaccine types (Table 3). cation submitted to FDA (2). The vaccine was well-tolerated, Two immunobridging trials were conducted. One compared and most adverse events were injection site-related pain, swell- 9vHPV in approximately 2,400 females and males aged 9 ing, and erythema that were mild to moderate in intensity. The through 15 years with approximately 400 females aged 16 safety profiles were similar in 4vHPV and 9vHPV vaccinees. through 26 years. Over 99% seroconverted to all nine HPV Among females aged 9 through 26 years, 9vHPV recipients had vaccine types; GMTs were significantly higher in adolescents more injection-site adverse events, including swelling (40.3% aged 9 through 15 years compared with females aged 16 in the 9vHPV group compared with 29.1% in the 4vHPV through 26 years. In a comparison of 4vHPV with 9vHPV group) and erythema (34.0% in the 9vHPV group compared in approximately 600 adolescent females aged 9 through 15 with 25.8% in the 4vHPV group). Males had fewer injection years, 100% seroconverted to HPV 6, 11, 16, and 18 in both site adverse events. In males aged 9 through 15 years, injection site swelling and erythema in 9vHPV recipients occurred in * Females who received all 3 vaccinations within 1 year of enrollment, did not 26.9% and 24.9%, respectively. Rates of injection-site swelling have major deviations from the study protocol, were naïve (polymerase chain reaction [PCR] negative and seronegative) to the relevant HPV type(s) before and erythema both increased following each successive dose dose 1, and who remained PCR negative to the relevant HPV type(s) through of 9vHPV. 1 month after dose 3 (month 7).

MMWR / March 27, 2015 / Vol. 64 / No. 11 301 Morbidity and Mortality Weekly Report

TABLE 2. Results of a Phase III efficacy trial comparing 9-valent human papillomavirus (HPV) vaccine (9vHPV) with quadrivalent HPV vaccine (4vHPV), per protocol population* in females aged 16 through 26 years† 9vHPV 4vHPV Vaccine efficacy Endpoint-related types Endpoint No. participants Cases No. participants Cases % (95% CI) HPV 31, 33, 45, 52, 58 ≥CIN2, VIN2/3, VaIN2/3 6,016 1 6,017 30 96.7 (80.9–99.8) ≥CIN2 5,948 1 5,943 27 96.3 (79.5–99.8) 6-month persistent infection 5,939 35 5,953 810 96.0 (94.4–97.2) HPV 6, 11, 16, 18 ≥CIN2§ 5,823 1 5,832 1 — — Anogenital warts 5,876 5 5,893 1 — — Abbreviations: CI = confidence interval; ≥CIN2 = cervical intraepithelial neoplasia grade 2 or 3 or adenocarcinoma in situ; VaIN2/3 = vaginal intraepithelial neoplasia grade 2 or 3; VIN2/3 = vulvar intraepithelial neoplasia grade 2 or 3. Sources: Package insert available at http://www.fda.gov/downloads/BiologicsBloodVaccines/Vaccines/ApprovedProducts/UCM426457.pdf. Joura EA, Giuliano AR, Iversen OE, et al. A 9-valent HPV vaccine against infection and intraepithelial neoplasia in women. N Engl J Med 2015;372:711–23. * Females who received all 3 vaccinations within 1 year of enrollment, did not have major deviations from the study protocol, were naïve (polymerase chain reaction [PCR] negative and seronegative) to the relevant HPV type(s) before dose 1, and who remained PCR negative to the relevant HPV type(s) through 1 month after dose 3 (month 7). † Participants were enrolled from sites in 18 countries; median duration of follow-up was 40 months.

TABLE 3. Human papillomavirus (HPV) 6, 11, 16, and 18 seroconversion and geometric mean titers (GMTs*) after 3 doses of 9-valent HPV vaccine (9vHPV) compared with quadrivalent HPV vaccine (4vHPV), per protocol population† in females aged 16 through 26 years§ 9vHPV 4vHPV Seropositivity Assay (cLIA) No. participants (%) GMT (mMU/mL) No. participants Seropositivity (%) GMT (mMU/mL) Anti-HPV 6 3,993 (99.8) 893 3,975 (99.8) 875 Anti-HPV 11 3,995 (100) 666 3,982 (99.9) 830 Anti-HPV 16 4,032 (100) 3,131 4,062 (100) 3,157 Anti-HPV 18 4,539 (99.8) 805 4,541 (99.7) 679 Abbreviations: cLIA = competitive Luminex immunoassay; mMU = milli-Merck units. Source: Joura EA, Giuliano AR, Iversen OE, et al. A 9-valent HPV vaccine against infection and intraepithelial neoplasia in women, and supplementary appendix. N Engl J Med 2015;372:711–23. * The noninferiority criterion for GMTs was met for all four HPV types (p<0.001). † Females who received all 3 vaccinations within 1 year of enrollment, did not have major deviations from the study protocol, were naïve (polymerase chain reaction [PCR] negative and seronegative) to the relevant HPV type(s) before dose 1, and who remained PCR–negative to the relevant HPV type(s) through 1 month after dose 3 (month 7). § Participants were enrolled from sites in 18 countries; median duration of follow-up was 40 months.

Health Impact and Cost Effectiveness beginning at age 9 years. Vaccination is also recommended Introduction of 9vHPV in both males and females was for females aged 13 through 26 years and for males aged 13 cost-saving when compared with 4vHPV for both sexes in a through 21 years who have not been vaccinated previously or cost-effectiveness model that assumed 9vHPV cost $13 more who have not completed the 3-dose series (1). Males aged 22 per dose than 4vHPV. Cost-effectiveness ratios for 9vHPV through 26 years may be vaccinated.† Vaccination of females remained favorable compared with 4vHPV (9vHPV was cost- is recommended with 2vHPV, 4vHPV (as long as this for- saving in most scenarios, and the cost per quality-adjusted life mulation is available), or 9vHPV. Vaccination of males is year gained did not exceed $25,000 in any scenario) when recommended with 4vHPV (as long as this formulation is varying assumptions about HPV natural history, cervical available) or 9vHPV. cancer screening, vaccine coverage, vaccine duration of protec- 2vHPV, 4vHPV, and 9vHPV all protect against HPV 16 tion, and health care costs, but were sensitive to 9vHPV cost and 18, types that cause about 66% of cervical cancers and assumptions (17). Because the additional five types in 9vHPV the majority of other HPV-attributable cancers in the United account for a higher proportion of HPV-associated cancers in States (1,12). 9vHPV targets five additional cancer causing females compared with males and cause cervical precancers, the types, which account for about 15% of cervical cancers (12). additional protection from 9vHPV will mostly benefit females. 4vHPV and 9vHPV also protect against HPV 6 and 11, types that cause anogenital warts. Recommendations for Use of HPV Vaccines † Vaccination is also recommended through age 26 years for men who have sex ACIP recommends that routine HPV vaccination be initi- with men and for immunocompromised persons (including those with HIV ated at age 11 or 12 years. The vaccination series can be started infection) if not vaccinated previously.

302 MMWR / March 27, 2015 / Vol. 64 / No. 11 Morbidity and Mortality Weekly Report

are in settings transitioning to 9vHPV, any available HPV What is currently recommended? vaccine product may be used to continue or complete the The Advisory Committee on Immunization Practices (ACIP) series for females for protection against HPV 16 and 18; recommends routine HPV vaccination at age 11 or 12 years. The vaccination series can be started beginning at age 9 years. 9vHPV or 4vHPV may be used to continue or complete the Vaccination is also recommended for females aged 13 through series for males. There are no data on efficacy of fewer than 26 years and for males aged 13 through 21 years who have not 3 doses of 9vHPV. been vaccinated previously or who have not completed the Special Populations. HPV vaccination is recommended 3-dose series. Males aged 22 through 26 years may be vacci- through age 26 years for men who have sex with men and for nated. ACIP recommends vaccination of men who have sex with men and immunocompromised persons through age 26 years if immunocompromised persons (including those with HIV not vaccinated previously. infection) who have not been vaccinated previously or have Why are the recommendations being updated now? not completed the 3-dose series. Precautions and Contraindications. HPV vaccines are 9-valent HPV vaccine (9vHPV) was approved by the Food and Drug Administration on December 10, 2014. This vaccine contraindicated for persons with a history of immediate hyper- targets HPV types 6, 11, 16, and 18, the types targeted by the sensitivity to any vaccine component. 4vHPV and 9vHPV quadrivalent HPV vaccine (4vHPV), as well as five additional are contraindicated for persons with a history of immediate types, HPV types 31, 33, 45, 52, and 58. ACIP reviewed results of hypersensitivity to yeast. 2vHPV should not be used in persons a randomized trial among approximately 14,000 females aged with anaphylactic latex allergy. 16 through 26 years that showed noninferior immunogenicity for the types shared by 4vHPV and 9vHPV and high efficacy for HPV vaccines are not recommended for use in pregnant the five additional types. Other trials in the 9vHPV clinical women (1). If a woman is found to be pregnant after initiating development program included studies that compared the vaccination series, the remainder of the 3-dose series should antibody responses across age groups and females and males be delayed until completion of pregnancy. Pregnancy testing and concomitant vaccination studies. The evidence supporting is not needed before vaccination. If a vaccine dose has been 9vHPV vaccination was evaluated using the Grading of administered during pregnancy, no intervention is needed. Recommendations, Assessment, Development, and Evaluation (GRADE) framework and determined to be type 2 (moderate A new pregnancy registry has been established for 9vHPV level of evidence) among females and 3 (low level of evidence) (2). Pregnancy registries for 4vHPV and 2vHPV have been among males; the recommendation was designated as a closed with concurrence from FDA (1,18). Exposure during Category A recommendation (recommendation for all persons pregnancy can be reported to the respective manufacturer.¶ in an age- or risk-factor–based group). Patients and health care providers can report an exposure to What are the new recommendations? HPV vaccine during pregnancy to the Vaccine Adverse Event 9vHPV, 4vHPV or 2vHPV can be used for routine vaccination of Reporting System (VAERS). females aged 11 or 12 years and females through age 26 years Adverse events occurring after administration of any vaccine who have not been vaccinated previously or who have not should be reported to VAERS. Additional information about completed the 3-dose series. 9vHPV or 4vHPV can be used for routine vaccination of males aged 11 or 12 years and males VAERS is available by telephone (1–800–822–7967) or online through age 21 years who have not been vaccinated previously at http://vaers.hhs.gov. or who have not completed the 3-dose series. ACIP recommends Cervical Cancer Screening. Cervical cancer screening is rec- either 9vHPV or 4vHPV vaccination for men who have sex with ommended beginning at age 21 years and continuing through men and immunocompromised persons (including those with age 65 years for both vaccinated and unvaccinated women HIV infection) through age 26 years if not vaccinated previously. (19,20). Recommendations will continue to be evaluated as further postlicensure monitoring data become available. Administration. 2vHPV, 4vHPV, and 9vHPV are each administered in a 3-dose schedule. The second dose is admin- Future Policy Issues istered at least 1 to 2 months after the first dose, and the third A clinical trial is ongoing to assess alternative dosing sched- § dose at least 6 months after the first dose (1). If the vaccine ules of 9vHPV. ACIP will formally review the results as data schedule is interrupted, the vaccination series does not need become available. HPV vaccination should not be delayed to be restarted. pending availability of 9vHPV or of future clinical trial data. If vaccination providers do not know or do not have avail- able the HPV vaccine product previously administered, or ¶ 9vHPV exposure during pregnancy should be reported to the Merck Pregnancy Registry at telephone 1-800-986-8999; 4vHPV exposure during pregnancy can be reported to Merck at telephone 1-877-888-4231. 2vHPV exposure during § Minimum intervals are 1 month between the first and second dose, 3 months pregnancy can be reported to GlaxoSmithKline at telephone 1-888-825-5249. between the second and third dose, and 6 months between the first and third dose.

MMWR / March 27, 2015 / Vol. 64 / No. 11 303 Morbidity and Mortality Weekly Report

Acknowledgments . 8 IARC Working Group on the Evaluation of Carcinogenic Risks to Humans. Biological agents. Volume 100 B: a review of human carcinogens. ACIP members (membership roster for July 2014–June 2015 IARC Monographs Eval Carcinog Risks Hum 2012;100(Pt B):1–441. available at http://www.cdc.gov/vaccines/acip/committee/members- . 9 Walboomers JM, Jacobs MV, Manos MM, et al. Human papillomavirus archive.html). ACIP HPV Work Group. is a necessary cause of invasive cervical cancer worldwide. J Pathol 1999;189:12–9. 1Epidemic Intelligence Service, CDC; 2National Center for HIV/AIDS, Viral 10. Forman D, de Martel C, Lacey CJ, et al. Global burden of human Hepatitis, STD and TB Prevention, CDC; 3Louisiana State University Health papillomavirus and related diseases. Vaccine 2012;30(Suppl 5):F12–23. Sciences Center, Shreveport, Louisiana; 4National Center for Immunization 11. Serrano B, Alemany L, Tous S, et al. Potential impact of a nine-valent and Respiratory Diseases, CDC; 5National Center for Chronic Disease vaccine in human papillomavirus related cervical disease. Infect Agent Prevention and Health Promotion, CDC; 6National Center for Emerging and Cancer 2012;7:38. Zoonotic Infectious Diseases, CDC (Corresponding author: Emiko Petrosky, 12. Saraiya M. Population-based HPV genotype attribution in HPV- [email protected], 404-639-1817) associated cancers. Presented at Anal Intraepithelial Neoplasia Society Conference, March 13–15, 2015, Atlanta, GA. References 13. CDC. Human papillomavirus (HPV)-associated cancers. Atlanta, GA: . 1 Markowitz LE, Dunne EF, Saraiya M, et al.; Centers for Disease Control US Department of Health and Human Services, CDC; 2015. Available and Prevention (CDC). Human papillomavirus vaccination: at http://www.cdc.gov/cancer/hpv/statistics/cases.htm. recommendations of the Advisory Committee on Immunization 14. Hariri S, Unger ER, Schafer S, et al.; HPV-IMPACT Working Group. Practices (ACIP). MMWR Recomm Rep 2014;63(No. RR-05):1–30. HPV type attribution in high-grade cervical lesions: assessing the 2. Food and Drug Administration. Highlights of prescribing information. potential benefits of vaccines in a population-based evaluation in the Gardasil 9 (human papillomavirus 9-valent vaccine, recombinant). Silver United States. Cancer Epidemiol Biomarkers Prev 2015;24:393–9. Spring, MD: US Department of Health and Human Services, Food and 15. Lacey CJ, Lowndes CM, Shah KV. Chapter 4: Burden and management Drug Administration; 2014. Available at http://www.fda.gov/downloads/ of non-cancerous HPV-related conditions: HPV-6/11 disease. Vaccine BiologicsBloodVaccines/Vaccines/ApprovedProducts/UCM426457.pdf. 2006;24(Suppl 3):35–41. 3. Food and Drug Administration. December 10, 2014 Approval letter— 16. Joura EA, Giuliano AR, Iversen OE, et al.; Broad Spectrum HPV Vaccine GARDASIL 9. Silver Spring, MD: US Department of Health and Study. A 9-valent HPV vaccine against infection and intraepithelial Human Services, Food and Drug Administration; 2014. Available at neoplasia in women. N Engl J Med 2015;372:711–23. http://www.fda.gov/BiologicsBloodVaccines/Vaccines/ 17. Brisson M. Cost-effectiveness of 9-valent HPV vaccination. Presentation ApprovedProducts/ucm426520.htm. before the Advisory Committee on Immunization Practices (ACIP), 4. Luxembourg A. Program summary and new 9-valent HPV October 30, 2014. Atlanta, GA: US Department of Health and Human data. Presentation before the Advisory Committee on Immunization Services, CDC; 2014. Available at http://www.cdc.gov/vaccines/acip/ Practices (ACIP), October 30, 2014. Atlanta, GA: US Department of meetings/downloads/min-archive/min-2014-10.pdf. Health and Human Services, CDC; 2014. Available at http://www.cdc. 18. Food and Drug Administration. Highlights of prescribing information. gov/vaccines/acip/meetings/downloads/min-archive/min-2014-10.pdf. Cervarix [human papillomavirus bivalent (types 16, 18) vaccine, . 5 Ahmed F, Temte JL, Campos-Outcalt D, Schünemann HJ; ACIP recombinant]. Silver Spring, MD: US Department of Health and Human Evidence Based Recommendations Work Group (EBRWG). Methods Services; Food and Drug Administration; 2009. Available at http://www. for developing evidence-based recommendations by the Advisory fda.gov/downloads/BiologicsBloodVaccines/Vaccines/ Committee on Immunization Practices (ACIP) of the U.S. Centers for ApprovedProducts/UCM186981.pdf. Disease Control and Prevention (CDC). Vaccine 2011;29:9171–6. 19. Saslow D, Solomon D, Lawson HW, et al. American Cancer Society, . 6 CDC. GRADE evidence tables—recommendations in MMWR. Atlanta, American Society for Colposcopy and Cervical Pathology, and American GA: US Department of Health and Human Services, CDC; 2015. Society for Clinical Pathology screening guidelines for the prevention Available at http://www.cdc.gov/vaccines/acip/recs/grade/table-refs.html. and early detection of cervical cancer. J Low Genit Tract Dis . 7 Muñoz N, Bosch FX, de Sanjosé S, et al.; International Agency for 2012;16:175–204. Research on Cancer Multicenter Cervical Cancer Study Group. 20. Moyer VA; US Preventive Services Task Force. Screening for cervical Epidemiologic classification of human papillomavirus types associated cancer: US Preventive Services Task Force recommendation statement. with cervical cancer. N Engl J Med 2003;348:518–27. Ann Intern Med 2012;156:880–91, W312.

304 MMWR / March 27, 2015 / Vol. 64 / No. 11

Vaccine 35 (2017) 2892–2901

Contents lists available at ScienceDirect

Vaccine

journal homepage: www.elsevier.com/locate/vaccine

Review Two-dose schedules for human papillomavirus vaccine: Systematic review and meta-analysis

Maddalena D’Addario a, Shelagh Redmond a, Pippa Scott a,b, Dianne Egli-Gany a, A. Ximena Riveros-Balta c, ⇑ Ana Maria Henao Restrepo c, Nicola Low a, a Institute of Social and Preventive Medicine (ISPM), University of Bern, Finkenhubelweg 11, 3012 Bern, Switzerland b Department of Pathology, University of Otago, 2 Riccarton Ave., Christchurch 8011, New Zealand c Initiative for Vaccine Research, Vaccines and Biologicals, World Health Organization, 20 Avenue Appia, 1211 Geneva 27, Switzerland article info abstract

Article history: Simpler schedules for human papillomavirus (HPV) vaccine delivery could improve vaccine coverage and Received 15 November 2016 the effectiveness of cervical cancer prevention. The objective of this study was to systematically review Received in revised form 6 March 2017 evidence about the effects of two-dose compared with three-dose schedules for human papillomavirus Accepted 31 March 2017 (HPV) vaccine and to describe the uptake of two-dose HPV vaccination schedules globally. We searched Available online 25 April 2017 PubMed, the Cochrane Central Registry of Controlled Trials, trials registers, and manufacturers’ databases from their earliest date to February 2016. We selected randomised controlled trials and controlled clinical Keywords: trials that directly compared HPV vaccine schedules with two or three doses. We extracted data on Human papillomavirus vaccines immunological and clinical outcomes and used meta-analysis where appropriate. We also described Vaccine schedules Vaccination the use of two-dose HPV vaccine schedules globally. We screened 1464 items and included seven eligible Systematic review noninferiority trials in 11 countries. In randomised comparisons amongst adolescent girls (three trials), Meta-analysis geometric mean concentrations (GMC) of antibodies against HPV16 and HPV18 were non-inferior or inconclusive, up to 24 months after a two-dose compared with a three-dose schedule. One trial with a clinical outcome found no persistent HPV infections occurred after either two or three doses. In non- randomised comparisons, GMC were non-inferior or superior in adolescent girls receiving the two- dose schedule compared with women receiving the three-dose schedule for at least 21 months after vac- cination. By February 2017, 23 low and middle income and 25 high income countries had adopted a two- dose HPV vaccination schedule. A two-dose HPV vaccine schedule provides satisfactory immunological outcomes in adolescent girls, but uptake globally is limited, particularly in countries with the highest burden of cervical cancer. Ó 2017 The Authors. Published by Elsevier Ltd. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).

Contents

1. Introduction ...... 2893 2. Methods ...... 2893 2.1. Inclusion and exclusion criteria ...... 2893 2.2. Search strategy and study selection...... 2893 2.3. Data extraction ...... 2894 2.4. Data synthesis and statistical analysis ...... 2894 3. Results...... 2894 3.1. Two-dose vs. three-dose HPV vaccine schedules in adolescent girls ...... 2894 3.2. Two-dose schedule in girls vs. three-dose schedule in women ...... 2896 3.3. Two-dose schedules comparing different intervals between doses ...... 2898 3.4. Introduction of two-dose HPV vaccination schedules...... 2898 4. Discussion...... 2898

⇑ Corresponding author at: Institute of Social and Preventive Medicine (ISPM), University of Bern, Finkenhubelweg 11, CH-3012 Bern, Switzerland. E-mail address: [email protected] (N. Low). http://dx.doi.org/10.1016/j.vaccine.2017.03.096 0264-410X/Ó 2017 The Authors. Published by Elsevier Ltd. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/). M. D’Addario et al. / Vaccine 35 (2017) 2892–2901 2893

4.1. Summary of main findings ...... 2898 4.2. Strengths and limitations of the review ...... 2898 4.3. Interpretation of the review findings...... 2899 4.4. Implications for policy and research ...... 2899 Conflict of interest ...... 2900 Acknowledgements ...... 2900 Appendix A. Supplementary material...... 2900 References ...... 2900

1. Introduction voiced [16]. The objectives of this study were to: describe the sys- tematic review evidence that informed the WHO recommendation; Vaccination against human papillomavirus (HPV) with sched- review trial evidence published since the recommendation; and ules that are efficacious, simple and cheap could improve the effec- describe adoption of two-dose HPV vaccination schedules. tiveness of cervical cancer prevention [1]. Persistent infection with HPV types 16 and 18 causes about 70% of cervical cancer [2].An 2. Methods estimated 485,000 new cases of cervical cancer in 2013 caused 6.9 million disability adjusted life years, 85% in developing coun- The systematic review followed a study protocol (Supplemen- tries [3]. tary File 1). We follow the Preferred Reporting Items for Systematic Simple immunisation schedules are essential to the optimisa- reviews and Meta-Analyses (PRISMA) guidelines for reporting [17]. tion of HPV vaccine effectiveness because the vaccine is given to adolescents, for whom health services are not well-developed [4]. 2.1. Inclusion and exclusion criteria In contrast, maternal and child health services are largely struc- tured to facilitate delivery of infant and childhood vaccines. Population and study design: We defined primary study popu- Reduced dose schedules for HPV vaccine have been suggested for lations in (a) RCTs comparing eligible vaccine schedules including several reasons. First, a subgroup analysis of a randomised con- girls in the age range nine to 14 years; and (b) controlled trials with trolled trial (RCT) in Costa Rica found that women who did not non-randomised comparisons of a two-dose schedule in girls ver- complete the planned three-dose schedule had similar rates of sus a three-dose schedule in women aged 15 years and over, with incident HPV 16/18 infection as those who received all three doses the groups enrolled concurrently in the same geographic locations. [5]. Second, two-dose schedules for hepatitis B vaccine in adoles- Secondary study populations were adult women or men aged cents are as effective as three-dose schedules when the two doses 15 years and over. are given at least six months apart [6,7]. The first vaccine dose Intervention: bivalent vaccine against HPV types 16 and 18 stimulates a B cell response that primes the immune system (CervarixÒ, GlaxoSmithKline) or quadrivalent vaccine against HPV (‘prime’ dose). Affinity-matured B cells then produce an anamnes- types 6, 11, 16 and 18 (GardasilÒ, Merck). We define HPV vaccina- tic response after a booster (‘boost’); affinity maturation takes at tion schedules, as follows: the first dose is a ‘prime’ dose, any sec- least four months to develop after the first prime dose(s) [7]. Third, ond dose given <4 months later is a ‘prime’ dose, and any second or it is hypothesised that ongoing exposure to HPV through sexual subsequent dose given 4 months after the prime dose is a ‘boos- intercourse should sustain vaccine-induced antibody levels, ter’ [7]. Results of trials of two-dose schedules of a 9-valent HPV through ‘natural boosting’ [7]. vaccine [18,19] were not available when this review was done. HPV vaccine presents challenges for evaluation and policy mak- Comparisons: two doses versus three doses of the same vaccine ing for two main reasons. First, adolescent girls who have not yet and the same dosage (three-dose arm using a schedule recom- had sexual intercourse are the target group for vaccination to pre- mended by WHO); and two doses versus two doses (prime-boost vent cervical cancer [8], but the pivotal RCT results enrolled or prime-prime) with different intervals between doses (same vac- women aged 15–26 years, because the outcome assessment cine and same dosage). We hypothesised that a prime-boost sched- required repeated endocervical sampling [9,10]. Second, the pri- ule would result in higher antibody levels than a prime-prime mary efficacy endpoint was a surrogate for invasive cancer, which schedule [7]. takes decades to develop after initial HPV infection. RCTs showed Outcomes: immunological (antibodies in serum presented as high levels of protection against cervical intraepithelial neoplasia geometric mean concentration, GMC, or percentage seropositive); (CIN) grade two or above caused by HPV16 or HPV18 amongst or clinical (incident HPV infection, CIN2, CIN3, adenocarcinoma women without detectable HPV antibodies when they were vacci- in situ, squamous cell carcinoma or adenocarcinoma, or genital nated [9,10]. HPV vaccine licensure for adolescents was based on warts). Neutralising antibody concentrations were measured using ‘immunological bridging’ studies showing antibody responses in either a competitive Luminex immunoassay for quadrivalent vac- adolescents that were as good as or better than those in women cine (cLIA seropositive defined for HPV16 as 20 milliMerck units, after the standard three-dose schedule [11,12]. mMU per mL, HPV18 24 mMU per mL) or enzyme-linked Two-dose schedules for the licensed HPV vaccines were first immunosorbent assay for bivalent vaccine (seropositive defined approved in 2014 [13]. In 2014, the World Health Organization for HPV16 as 8 ELISA units, EU per mL, HPV18 7 EU per mL). (WHO) strategic advisory group of experts on vaccination con- We did not assess vaccine safety in this review. cluded that evidence from immunological studies was sufficient to recommend a two-dose HPV vaccine schedule, with an interval 2.2. Search strategy and study selection of at least six months between doses, for girls aged under 15 years [14]. Although the International Agency for Research on Cancer We searched PubMed, the Cochrane Central Registry of Con- recommended immunogenicity as a surrogate marker for vaccine trolled Trials, clinical trials registers and manufacturers’ websites. efficacy against clinical disease caused by HPV16 and HPV18 in We also screened reference lists of included studies, abstracts from 2014 [15], doubts about the validity of the decision have been the European Research Organisation on Genital Infection and 2894 M. D’Addario et al. / Vaccine 35 (2017) 2892–2901

Neoplasia 2013 meeting, regulatory dossiers provided by represen- three doses) the lower 95% CI should be above À0.05. If the lower tatives of the vaccine companies and studies presented at a WHO confidence interval was below the noninferiority margin but the ad hoc meeting in November 2013. The initial search dates were point estimate was within the margin, we called the result incon- the earliest publication date of each electronic database to the last clusive [20]. We present GMC data one month after the last admin- week of January 2014. We updated these searches to the last week istered dose, which was the most consistently reported time point. of February 2016, including specific searches for publications from GMC data at later time points and seroconversion/seropositivity trials that were ongoing. The full search strategy is shown in Sup- data are presented in Supplementary File 2. plementary File 2. In February 2017, we identified countries that had introduced One reviewer conducted the searches and excluded titles that HPV vaccination, using the WHO data on the year of introduction were clearly ineligible, e.g. duplicates and news items. Two review- of selected vaccines [23]. Countries were classified as high income, ers independently screened the titles and abstracts of the remain- upper middle, lower middle or low according to World Bank coun- ing items. Items relating to the same study were grouped together try classifications [24]. We then reviewed data for individual coun- and represented by a primary document (published article, clinical tries in the WHO vaccine preventable diseases monitoring system trial report, clinical register entry or meeting presentation). Two [25] to identify those that recommend a two-dose schedule by the independent reviewers read the full text of potentially eligible end of 2016. We consulted individual country homepages in Febru- items and decided on those to include. Discrepancies were ary 2017 for countries with unclear results [26,27]. resolved by discussion. 3. Results 2.3. Data extraction The initial search strategies yielded a total of 1175 hits relating One reviewer extracted data from relevant items into a pre- to 948 unique items published by January 2014 (Fig. 1, Table S1). piloted form (Epidata, Odensk, Denmark) recording the trial popu- The updated search found 227 unique hits, including four new lation, interventions, and outcomes in intention-to-treat and per- items relating to trials from the initial search (Fig. 1). protocol study populations. At least one other reviewer checked The initial search identified seven trials, all of which used a non- the extracted data. The reviewers discussed discrepancies and inferiority trial design: six [28–33] (43 items) reported on eleven made corrections if necessary. Two reviewers assessed the poten- eligible comparisons between groups of adolescent girls or com- tial risk of bias. pared adolescent girls and women; and one trial compared sched- ules between groups of women [34] (Table 1). The trials were 2.4. Data synthesis and statistical analysis conducted in 11 countries: two trials in high income countries only (Canada and Germany) [28,29], two trials in both high and middle We used data from per-protocol populations. In noninferiority income countries (Canada, Germany, Italy, Romania, Slovakia, Tai- trials, per protocol results are often reported for the primary anal- wan, Thailand) [32,34], two in middle income countries only ysis [20] because this analysis is more likely to show a difference (India, Mexico) [30,31], and one in low income countries only between groups than intention to treat analysis [21]. Within each (Senegal and Tanzania) [33]. The updated search did not identify trial, for HPV16 and HPV18 separately, we calculated: the differ- any new trials but found additional results for two trials ence in log antibody concentrations (two doses minus three doses, (Canada/Germany1 [35], Multinational 2 [36]) and results from a with 95% confidence intervals, CI), which corresponds to a ratio of second arm of a previously included trial in Mexico (Table 1) GMCs on the natural scale; and the difference in proportions [37,38]. seropositive after vaccination. For analyses of proportions we used One trial in India examined clinical outcomes [30]. Unpublished participants who were seronegative at baseline. If seropositivity interim data were presented to the WHO strategic advisory group data were not stratified by pre-vaccination status, we used the of experts in April 2014 [39]. The updated search found published whole study population. results [30]. The RCT was suspended in April 2010 for reasons We used Stata version 13 (StataCorp, Austin, USA) for all analy- unrelated to that trial. We report the results despite disruption of ses. We prepared forest plots and examined heterogeneity the random allocation because this was the only trial with clinical between studies visually and using the I2 statistic [22]. We strati- endpoints. fied results a priori according to the level of economic development All trials reports reported a sample size calculation and used the in countries where the trials were conducted, grouping high and same noninferiority margin. Descriptions of allocation conceal- middle income countries because some trials included both. We ment in RCTs were absent from most RCTs (Tables S2a and S2b). did not compare absolute levels of antibody concentrations between trials that used different vaccines because the methods 3.1. Two-dose vs. three-dose HPV vaccine schedules in adolescent girls for measurement differed. To compare outcomes following vaccination with a two-dose vs. Three RCTs in adolescent girls compared a two-dose (prime- three-dose schedule, we calculated the ratio of GMCs or the differ- boost) with a three-dose (prime-prime-boost) schedule (Table 1) ence in proportions seropositive. For meta-analysis, we used a ran- [28,29]. dom effects model to estimate the weighted average of the effects Fig. 2 shows that one month after the last dose, the trial findings on the log scale. This model allows for variation between studies in for HPV16 in high income countries were heterogeneous (I2 93%). the estimate of effect size, which might occur when trials are con- The GMC for the two-dose schedule was noninferior to the three- ducted in many different countries. The point estimate and lower dose schedule in the Canada1 trial and inconclusive in the and upper CIs were exponentiated to give the ratio of GMCs in nat- Canada/Germany1 trial (Table S3). For HPV18 both trials found ural units. Noninferiority margins were those used in the individ- that GMCs in the two-dose group were lower, but noninferior to ual trials; all trials used the same noninferiority margin (0.5 for those in the three-dose group. In India, GMCs were higher in the the GMC ratio, 0.05 for the proportion seropositive). To interpret two-dose group for both HPV16 and HPV18. the findings, we examined confidence intervals (Fig. S1, Supple- At later time points (Table S3, Fig. S2), GMCs were lower in girls mentary File 2). For GMC ratios (two doses divided by three doses) receiving the two-dose schedule than those receiving three doses noninferiority is demonstrated if the lower 95% CI is greater than in Canada and Germany; noninferiority was demonstrated at 0.5. For the difference in proportions seropositive (two doses minus 36 months for HPV16 in the Canada1 trial but the other results M. D’Addario et al. / Vaccine 35 (2017) 2892–2901 2895

Initial search up to January 2014: 1175 items (PubMed, 410; CENTRAL, 107; Clinicaltrials.gov, 282; WHO trials portal, 167; manufacturer websites, 122; expert references, 59; WHO Consultation Meeting, 18; reference lists, 10) Search update to February 2016: 289 items (PubMed, 119; CENTRAL, 72; Clinicaltrials.gov,65; WHO trials portal,30; handsearch, 3; manufacturer websites, 0 new hits but updated information on 4 from initial search

Excluded: Initial search, 586 items 227 Duplicates 359 Other reasons Update search, 174 items 62 Duplicates 112 Other reasons

Title and abstract screening Initial search, 589 items Update search, 115 items

Excluded: Initial search, 113 items Update search, 24 items

Full text/additional screening Initial search, 476 items Update search, 91 items

Excluded: Initial search, 380 items 17 Reviews (screened for eligible studies) 19 Population ineligible 9 Intervention ineligible 297 Comparison ineligible 12 Outcome ineligible 26 No data could be extracted Update search, 78 items 8 Reviews (screened for eligible studies) 7 Population ineligible 4 Intervention ineligible 47 Comparison ineligible 6 Outcome ineligible 6 No data could be extracted Potentially eligible comparison Initial search, 96 items (27 studies) Update search, 13 items (6 studies) Excluded: Initial search, 53 items (20 studies) Update search, 9 items (3 studies, 1 new) Comparisons not relevant for publication

2- vs. 3-dose schedules 2- vs. 3-dose and 2- vs. 2-dose schedules

Initial search, 5 trials (25 items) Initial search, 2 trials (18 items)

Update search, 2 trials (2 items) Update search, 2 trials (2 items)

Fig. 1. Flow chart of retrieved, excluded and included items, and number of studies according to comparison, as of 22 February 2016. See Table S1 for details of excluded trials.

were inconclusive. In the Canada/Germany1 trial, results at trials in Canada and Germany up to 60 months after vaccination, 36 months were not disaggregated by age. At month 48 in India, but fewer girls receiving the two-dose schedules had antibodies GMCs were lower but noninferior in girls who received two doses. against HPV18 in the Canada1 trial at 24 and 36 months. In India, There were no differences between the schedules in seropositivity the percentage of girls with antibodies fell to about 78% for one month after the last dose (Fig. S3, Table S4). At later time HPV16 and 92% for HPV18 at 48 months but was similar in both points, all participants tested remained seropositive for HPV16 in groups. 2896 M. D’Addario et al. / Vaccine 35 (2017) 2892–2901

Table 1 Summary of schedule comparisons in trials included in the review.

Schedulea First author, year of primary Total vaccinated Primary population with Primary population with non- Secondary Study name, primary publication, additional study cohort (age in randomised comparison randomised comparison comparisons reference,b vaccine identifiers years) Girls vs. girls, age in years Girls vs. women, age in years Women vs. (schedule, months) (schedule, months) women Two-dose vs. three- Primary population and comparison dose Canada/Germany1,29 Romanowski 2011, NCT00541970, 160 (9–14) 9–14 (0,6) vs. 9–14 9–14 (0,6) vs. 15–25 (0,1,6) bivalent GSK110659, 319 (15–25) (0,1,6) HPV-048 Canada1,28 Dobson 2013, NCT00501137, 520 (9–13) 9–13 (0,6) vs. 9–13 9–13 (0,6) vs. 16–26 (0,2,6) quadrivalent H07-00928, 310 (16–26) (0,2,6) BCGov-01 India,30 Sankaranarayanan 2016, 9327 (10–18)c 10–18 (0,6) vs. 10–18 quadrivalent NCT0093702, Sankaranarayanan (0,2,6) 2013, BMGF48979, ISRCTN98283094, REFCTRI-2009 000137 Mexico,31 Lazcano-Ponce 2013, 1026 (9–10) 500 see Table S1 9–10 (0,6) vs. 18–24 (0,1,6) bivalent NCT01717118 (18–24) Mexico, Hernandez-Avila 2016, 150 (9–10) see Table S1 9–10 (0,6) vs. 18–24 (0,1,6) quadrivalent,37 d NCT01717118 150 (18–24) Multinational2,32 e Puthanakit 2013, 2016, 965 (9–14) 9–14 (0,6) vs. 15–25 (0,1,6) 9– bivalent NCT01381575, 482 (15–25) 14 (0,12) vs. 15–25 (0,1,6) GSK114700, HPV-070 Senegal/Tanzania,33 Sow 2013, 232 (10–14) 444 10–14 (0,1) vs. 15–25 (0,1,6) bivalent NCT00481767, (15–25) GSK106069 Two-dose vs. three- Secondary population and comparison dose Europe,34 f Esposito 2011 804 (15–25) 15–25 (0,1) vs. bivalent 15–25 (0,1,6) Two-dose vs. two- dose Canada/Germany1,29 Romanowski 2011, NCT00541970, 159 (9–14) 9–14 (0,6) vs. 9–14 (0,2) 15–19 (0,6) vs. bivalent GSK110659, 164 (15–19) (40 mg) 15–19 (0,2) HPV-048 158 (20–25) (40 mg) 20–25 (0,6) vs. 20–25 (0,2) (40 mg) Multinational2,32 NCT01381575, 965 (9–14) 9–14 (0,12) vs. 9–14 (0,6) bivalent GSK 114700, HPV-070

a Each trial can contribute to more than one comparison. Study names indicated the country or countries in which the trial was done, with numerical suffixes for multiple trials from the same country. b Primary reference for each trial. Additional items associated with the trial cited in the web appendix. c Publication found in updated search in 2016. Total number enrolled, 17,729 but 12,402 did not receive the planned number of vaccine doses because the trial was suspended. d Publication found in updated search in 2016. e Canada, Germany, Italy, Taiwan, Thailand. f Italy, Romania, Slovakia.

In India, the incidence of HPV infections was measured. There [31], Mexico quadrivalent [37], Multinational2 [36]). One RCT, in were four new infections with HPV16 or HPV18 in 526 girls who Senegal and Tanzania, compared the licensed three-dose schedule received two doses and two in 536 girls who received three doses. with placebo in 10 to 25 year old girls and women [33]. Outcomes No infection persisted for more than 12 months. could be compared between 10 to 14 year old girls one month after One RCT in women aged 15–25 years provided immunological the first two doses (prime-prime) and 15 to 25 year old women one outcome data comparing two prime-prime doses (0 and 1 months) month after the third dose (prime-prime-boost). with two three-dose schedules [34] (Tables S3 and S4). One month Fig. 3 shows that GMCs in girls receiving the two-dose prime- after the last dose, GMCs were lower for the prime-prime schedule boost schedule were higher than in women receiving the licensed but seropositivity was high in both groups. three-dose schedule in all comparisons except HPV16 in the Multi- national2 trial. Results were heterogeneous but all trials met the 3.2. Two-dose schedule in girls vs. three-dose schedule in women criteria for noninferiority with combined pooled GMC ratios of 1.44 (1.04, 2.00) for HPV16 and 1.52 (1.25, 1.84) for HPV18. Trials Five trials with six comparisons [28,29,31–33,37] compared the with longer follow up showed that noninferiority was maintained immunogenicity of a two-dose schedule in girls (n = 3053) and a for up to 48 months after the first dose (Figs. S4, S5, Tables S5 and three-dose schedule in women (n = 2205). Four trials with five S6). In Senegal and Tanzania, GMCs were lower after two prime- comparisons compared a two-dose (prime-boost) schedule in ado- prime doses (0, 1 month) than after the three-dose schedule but lescent girls with a three-dose (prime-prime-boost) schedule in all participants had detectable antibody against both HPV16 and women (Canada1 [28], Canada/Germany1 [29], Mexico bivalent HPV18. M. D’Addario et al. / Vaccine 35 (2017) 2892–2901 2897

Income stratum, HPV type Ratio of GMC (95% CI) Study name, vaccine 2 doses vs. 3 doses Schedules

High income, HPV type 16

Canada/Germany1, bivalent, 9-14 y 0.50 (0.38, 0.66) 0, 6 vs 0, 1, 6

Canada1, quadrivalent, 9-13 y 0.98 (0.79, 1.21) 0, 6 vs 0, 2, 6

Subtotal (I-squared = 93.0%, p = 0.000) 0.70 (0.36, 1.36)

High income, HPV type 18

Canada/Germany1, bivalent, 9-14 y 0.75 (0.57, 0.97) 0, 6 vs 0, 1, 6

Canada1, quadrivalent, 9-13 y 0.71 (0.58, 0.86 0, 6 vs 0, 2, 6

Subtotal (I-squared = 0.0%, p = 0.766) 0.72 (0.62, 0.84)

Low-middle income, HPV type 16

India, quadrivalent, 10-18 y 1.12 (1.02, 1.23) 0,6 vs. 0, 2, 6

Low-middle income, HPV type 18

India, quadrivalent, 10-18 y 1.04 (0.92, 1.19) 0,6 vs. 0, 2, 6

0.25 0.5 1 2 4 lower GMC with 2 doses higher GMC with 2 doses

Fig. 2. Forest plot, weighted mean difference between GMCs in girls receiving two-dose and three-dose schedules, one month after last dose, by income level and HPV type; two trials in high income countries and one in a low-middle income country. Dashed line is the noninferiority margin.

Income stratum, HPV type Study name, vaccine, age group GMC ratio (95% CI) Schedules, months

High-middle income, HPV16 Canada/Germany1, bivalent, 9-14 vs 15-25 y 1.07 (0.81, 1.42) 0, 6 (9-14) vs 0, 1, 6 (15-25) Canada1, quadrivalent, 9-13 vs 16-26 y 2.09 (1.68, 2.60) 0, 6 (9-13) vs 0, 2, 6 (16-26) Multinational2, bivalent, 9-14 vs 15-25 y 0.92 (0.82, 1.03) 0, 6 (9-14) vs 0, 1, 6 (15-25) Mexico, bivalent, 9-10 vs 18-24 y 1.49 (1.33, 1.67) 0, 6 (9-10) vs 0, 1, 6 (18-24) Mexico, quadrivalent, 9-10 vs 18-24 y 2.13 (1.57, 2.89) 0, 6 (9-10) vs 0, 2, 6 (18-24) Subtotal (I-squared = 94.2%, p = 0.000) 1.44 (1.04, 2.00)

High-middle income, HPV18 Canada/Germany1, bivalent, 9-14 vs 15-25 y 1.29 (1.01, 1.65) 0, 6 (9-14) vs 0, 1, 6 (15-25) Canada1, quadrivalent, 9-13 vs 16-26 y 1.83 (1.51, 2.21) 0, 6 (9-13) vs 0, 2, 6 (16-26) Multinational2, bivalent, 9-14 vs 15-25 y 1.18 (1.05, 1.32) 0, 6 (9-14) vs 0, 1, 6 (15-25) Mexico, bivalent, 9-10 vs 18-24 y 1.68 (1.50, 1.87) 0, 6 (9-10) vs 0, 1, 6 (18-24) Mexico, quadrivalent, 9-10 vs 18-24 y 1.76 (1.38, 2.25) 0, 6 (9-10) vs 0, 2, 6 (18-24) Subtotal (I-squared = 85.5%, p = 0.000) 1.52 (1.25, 1.84)

Low income, HPV16 Senegal/Tanzania, bivalent, 10-14 vs 15-25 y 0.49 (0.42, 0.58) 0, 1 (10-14) vs 0, 1, 6 (15-25)

Low income, HPV18 Senegal/Tanzania, bivalent, 10-14 vs 15-25 y 0.81 (0.69, 0.94) 0, 1 (10-14) vs 0, 1, 6 (15-25)

0.25 0.5 1 2 4 Favours 3 doses Favours 2 doses

Fig. 3. Forest plot, weighted mean difference between GMCs in girls receiving a two-dose schedule and women receiving the licensed three-dose schedule, one month after last dose, by income level and HPV type; four trials in high or middle income and one trial in low income countries. Dashed line shows the noninferiority margin. 2898 M. D’Addario et al. / Vaccine 35 (2017) 2892–2901

Income stratum, HPV type Study name, vaccine, age group GMC ratio (95% CI) Schedules, Months (dosage)

High income, HPV type 16, 9-14 year old Canada/Germany1, bivalent, 9-14 y 2.06 (1.60, 2.64) 0, 6 (40 µg) vs 0, 2 (40 µg) Multinational2, bivalent, 9-14 y 1.22 (1.11, 1.34) 0, 12 (20 µg) vs 0, 6 (20 µg)

High income, HPV type 18, 9-14 year old Canada/Germany1, bivalent, 9-14 y 1.60 (1.22, 2.10) 0, 6 (40 µg) vs 0, 2 (40 µg) Multinational2, bivalent, 9-14 y 1.13 (1.01, 1.25) 0, 12 (20 µg) vs 0, 6 (20 µg)

High income, HPV type 16, other ages Canada/Germany1, bivalent, 15-19 y 2.15 (1.62, 2.84) 0, 6 (40 µg) vs 0, 2 (40 µg) Canada/Germany1, bivalent, 20-25 y 1.73 (1.25, 2.38) 0, 6 (40 µg) vs 0, 2 (40 µg)

High income, HPV type 18, other ages Canada/Germany1, bivalent, 15-19 y 2.06 (1.52, 2.81) 0, 6 (40 µg) vs 0, 2 (40 µg) Canada/Germany1, bivalent, 20-25 y 1.54 (1.05, 2.26) 0, 6 (40 µg) vs 0, 2 (40 µg)

0.25 0.5 1 2 4 Favours shorter interval Favours longer interval

Fig. 4. Forest plot, weighted mean difference between GMCs one month after the last vaccine dose in girls (9–14 years) and women at older ages (15–19 and 20–25 years) receiving two two-dose schedules with standard or alternative dosage. Noninferiority margin not defined because neither schedule is the current standard schedule.

3.3. Two-dose schedules comparing different intervals between doses 4. Discussion

Two RCTs [29,32] directly compared two-dose schedules in 4.1. Summary of main findings girls between 9 and 14 years (n = 1124) (Tables S7 and S8). The Multinational2 [36] trial compared two two-dose schedules using This review included seven controlled trials in 11 countries the standard dosage (20 mg) of each serotype; the interval with direct comparisons between two-dose and three-dose HPV between the prime and boost doses was longer (doses at 0, vaccine schedules. When compared with women receiving the 12 months) in one group than the other (0, 6 months). The licensed three-dose schedule, adolescent girls receiving a two- Canada/Germany1 [29] trial compared a prime-prime schedule dose HPV vaccine schedule with a six-month interval between (0, 2 months) with a prime-boost (0, 6 months) schedule with a doses had noninferior antibody responses to HPV16 and HPV18 higher dosage (40 mg) of each serotype. In both trials, GMCs were (measured as GMCs or seropositivity) for at least two years after higher when the interval between doses was longer (Fig. 4, the first dose. There was no comparison in which girls who Table S7). GMCs were substantially higher in girls receiving a received a two-dose schedule had antibody responses that were prime-boost schedule than a prime-prime schedule. The differ- inferior to those of girls who received three doses up to 48 months ence between the two prime-boost schedules was more modest, after the last dose. One trial in India found no persistent new HPV but favoured the 12 months interval. In the Canada/Germany1 infections in girls receiving either two or three-doses. Comparisons trial [29], the prime-boost schedule also resulted in higher GMCs between different two-dose schedules show higher GMCs with a than the prime-prime schedule in women aged between 15 and longer interval between doses. 25 years (n = 319) (Table S7). In both trials, all participants receiv- ing any two-dose regimen were HPV antibody seropositive (Fig. S6 and Table S8). 4.2. Strengths and limitations of the review

The strengths of this systematic review include the wide- 3.4. Introduction of two-dose HPV vaccination schedules ranging search to identify data about both immunological and clin- ical outcomes. A systematic review of studies published up to By February 2017, 96 countries had introduced HPV vaccination October 2014 searched for immunological outcomes only [16]. into their national schedule (Table 2). Of these, 48 use a two-dose All trials were designed to show noninferiority between schedules, schedule. Eight countries introduced the two-dose schedule after used the same margins and studied similar age groups, increasing April 2014 (Canada, Chile, United Kingdom, Solomon Islands, Burk- the robustness of results. Nevertheless, only one trial had clinical ina Faso, Liberia, Tanzania, USA) compared with two before April outcome data [30], and many trials were small, resulting in unre- 2014 (Switzerland, The Netherlands). The date of introduction of solved heterogeneity and inconclusive findings. A further limita- the two-dose schedule is not known for the remaining 38 tion is that this review included data available up to February countries. 2016 and only addressed HPV16 and HPV18. A two-dose schedule M. D’Addario et al. / Vaccine 35 (2017) 2892–2901 2899

of a nonavalent vaccine that covers another five oncogenic HPV types (31, 33, 45, 52, 58) was licensed in June 2016 [18] and immunogenicity data, published in November 2016, show non- Totals N = 193 40 27 8 inferiority to the three-dose regimen in girls and boys [19]. Trials of quadrivalent vaccine suggested high levels of protection against the non-oncogenic mucosal HPV types 6 and 11, supported by surveillance data showing large reductions in the incidence of gen- ital warts in Australia since vaccine introduction [40].

4.3. Interpretation of the review findings 021 Low income n=31 3 Rwanda, Senegal, Uganda 8 Benin, Burundi, Ethiopia, Malawi, Mali, Mozambique, Niger, Togo 4 Burkina Faso, Gambia, Liberia, Tanzania We identified four major challenges to extrapolating the review’s findings to the public health impact of two-dose HPV vac- cination schedules. First, only one RCT measured an HPV disease outcome [30]. The RCT in India found no persistent HPV infections after either two or three doses but was suspended, following an investigation into deaths reported during another HPV vaccine study [41]. Second, there is no established immune correlate of protection so the clinical significance of HPV antibody concentra- tions is still uncertain [7,8]. The included trials did not measure 3 Bhutan, Micronesia, Zambia Lower middle income n=50 1 Lesotho 6 Cameroon, Côte d’Ivoire, Honduras, Mauritania, Sao Tome and Principe, Tajikistan Kenya, Laos, Philippines, Solomon Islands memory B cell responses or affinity maturation but waning levels of neutralising antibodies do not appear to result in a loss of pro- tection against clinical disease up to 48 months after receiving the licensed three-dose schedule [42]. Subsequent studies of other measures of the immune response have had mixed results, but none has found consistently poorer responses with a two-dose schedule than with a three-dose schedule [43]. Third, the duration of protection of two-dose HPV vaccine schedules remains unknown. A modelling study suggests that a two-dose schedule that protects for at least 20 years could prevent most vaccine- type preventable cancers [44]. But, if a two-dose schedule only gives protection for 10 years, a third dose could double the number 11 5 7 48 235 Guyana, Libya, Palau, Paraguay, Peru 14 15 96 Upper middle income n=55 11 Argentina, Botswana, Columbia, Brazil, Ecuador, Malaysia, Mexico, Panama, South Africa, Suriname, Macedonia 7 Belize, Fiji, Kazakhstan, Marshall Islands, Romania, Russia, Turkmenistan 04 of cancers prevented [45]. Fourth, the appropriate reference group for two-dose HPV vaccine schedule evaluation is still not clear. If duration of protection is the most important factor, then noninfe- riority of clinical outcomes in randomised comparisons between groups of adolescents is critical. A two-dose HPV vaccine schedule in adolescent girls resulted in vaccine type HPV antibody concentrations that were noninferior to those in women and mostly noninferior to those in girls who received the licensed three-dose schedule. Antibody levels at every time point up to 48 months [35] after the first dose were much higher than those after natural infection. The RCT findings support post hoc analyses of data from incompletely vaccinated women [5,46], some of whom received the first and second doses only one month apart. Further analyses of non-randomised compar- isons estimated high vaccine efficacy against any detection of HPV types 16 or 18 of 76.3% (95% CI 62.0, 85.3%) and against per- sistent infection at 12 months of 89.6% (95% CI 68.9, 97.5%) up to four years after vaccination [46]. These data could help to deter- mine an immune correlate of protection [46]. The effects of two- doses of HPV vaccine on clinical outcomes have also been studied in cohorts of women vaccinated in national programmes, usually when scheduled doses of a three-dose schedule have been missed. In some of these studies, the incidence of lesions was higher in women who received two compared with three doses, but these studies do not take into account the poorer immunogenicity when Australia, Barbados, Brunei Darussalam,Japan, Finland, Greece, Latvia, Monaco, New Zealand,Uruguay Norway, Singapore, Trinidad and Tobago, 44 High income n=57 25 Andorra, Antigua and Barbuda, Austria,Germany, Belgium, Hungary, Canada, Iceland, Chile, Ireland, Denmark, Israel,Portugal, France, Italy, Republic Luxembourg, of Malta, Korea, Netherlands, Slovenia,United States Spain, Sweden, of Switzerland, America United Kingdom, Bahamas, Bahrain, Cook Islands, Czech Republic, San Marino, Seychelles 0 the interval between two doses is one month rather than six months or more and do not control for confounding [43].

4.4. Implications for policy and research

This review was a part of the evidence that resulted in the rec- ommendation in favour of a two-dose HPV vaccine schedule using national schedule coverage coverage Complete country Partial country either bivalent or quadrivalent vaccines in April 2014 [14]. Since 2-dose schedule 25 3-dose schedule 13 HPV included in Unclear schedule 6 Table 2 Summary of HPV vaccine schedules according to World Bank income categories, February 2017. then, trials show that a two-dose schedule of a nonavalent HPV 2900 M. D’Addario et al. / Vaccine 35 (2017) 2892–2901 vaccine is also likely to be noninferior to a nonavalent three-dose [11] Pedersen C, Petaja T, Strauss G, Rumke HC, Poder A, Richardus JH, et al. vaccine schedule [19]. The introduction of two-dose schedules Immunization of early adolescent females with human papillomavirus type 16 and 18 L1 virus-like particle vaccine containing AS04 adjuvant. J Adolesc has remained low, however, and HPV vaccine has still not been Health 2007;40:564–71. approved in India. The reasons for continued use of the three- [12] Block SL, Nolan T, Sattler C, Barr E, Giacoletti KE, Marchant CD, et al. dose schedule are unknown, but most of these are high or upper Comparison of the immunogenicity and reactogenicity of a prophylactic quadrivalent human papillomavirus (types 6, 11, 16, and 18) L1 virus-like middle income countries (Table 2); Zambia is the only country eli- particle vaccine in male and female adolescents and young adult women. gible for support from the Global Vaccine Alliance (GAVI) known to Pediatrics 2006;118:2135–45. use the three-dose schedule. In Switzerland, a two-dose schedule [13] European Medicines Agency. Committee for medicinal products for human use (CHMP) assessment report EMA/789820/2013 Cervarix 7 Westferry Circus, has been recommended for boys as well as girls [47], although Canary Wharf, London E14 4HB. United Kingdom: European Medicines we found no completed studies in boys and no data about the Agency, Committee for Medicinal Products for Human Use; 2013. effects of two-dose schedules on anal, vulvovaginal, penile, or [14] World Health Organization. Meeting of the strategic advisory group of experts on immunization, April 2014 – conclusions and recommendations. Wkly oropharyngeal cancers. Additional HPV vaccine doses are likely to Epidemiol Rec 2014;89:221–36. be needed in HIV-infected people [14]. Trials to evaluate the clini- [15] International Agency for Research in Cancer. Primary end-points for cal efficacy of two-dose and even one-dose schedules have been prophylactic HPV vaccine trials. IARC working group report. World Health proposed [30] and clinical outcome data are needed [48], even Organization International Agency for Research on Cancer; 2014. p. 1–94. [16] Donken R, Knol MJ, Bogaards JA, van der Klis FR, Meijer CJ, de Melker HE. though immunological outcomes in adolescents are judged to pro- Inconclusive evidence for non-inferior immunogenicity of two- compared vide sufficient evidence of bridging to clinical efficacy in adults with three-dose HPV immunization schedules in preadolescent girls: a [15]. The available evidence from this review show that a two- systematic review and meta-analysis. J Infect 2015;71:61–73. [17] Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gotzsche PC, Ioannidis JP, et al. The dose HPV vaccine schedule provides satisfactory immunological PRISMA statement for reporting systematic reviews and meta-analyses of responses in adolescent girls, but uptake globally is limited, partic- studies that evaluate health care interventions: explanation and elaboration. ularly in countries with the highest burden of cervical cancer. PLoS Med 2009;6:e1000100. [18] Joura EA, Giuliano AR, Iversen OE, Bouchard C, Mao C, Mehlsen J, et al. A 9- valent HPV vaccine against infection and intraepithelial neoplasia in women. N Conflict of interest Engl J Med 2015;372:711–23. [19] Iversen OE, Miranda MJ, Ulied A, Soerdal T, Lazarus E, Chokephaibulkit K, et al. Immunogenicity of the 9-valent HPV vaccine using 2-dose regimens in girls None. and boys vs a 3-dose regimen in women. JAMA 2016;316:2411–21. [20] Piaggio G, Elbourne DR, Pocock SJ, Evans SJ, Altman DG. Reporting of noninferiority and equivalence randomized trials: extension of the CONSORT Acknowledgements 2010 statement. JAMA 2012;308:2594–604. [21] Mulla SM, Scott IA, Jackevicius CA, You JJ, Guyatt GH. How to use a noninferiority trial: users’ guides to the medical literature. JAMA This work was supported by grants from the World Health 2012;308:2605–11. Organization (WHO) (2013/372949-0) and the Swiss National [22] Higgins JP, Thompson SG. Quantifying heterogeneity in a meta-analysis. Stat Science Foundation (grant number 320030_135654, EpideMMIC Med 2002;21:1539–58. [23] World Health Organization. Year of introduction of selected vaccines database; project, Epidemiology and Mathematical Modelling for Infectious 2017. disease Control). The views expressed in this article are those of [accessed 20.02.17]. the authors and do not necessarily represent the decisions, policies, [24] The World Bank. Countries and economies; 2017 [accessed 20.02.17]. [25] World Health Organization. WHO vaccine-preventable diseases: monitoring system. 2016 global summary; 2016. [accessed 20.02.17]. Appendix A. Supplementary material [26] Government of Canada. Page 13: canadian immunization guide: Part 1 – Key immunization information; 2017. [accessed 20.02.17]. 096. [27] U.S. Department of Health & Human Services. Recommended immunization schedule for children and adolescents aged 18 years or younger, United States; 2017. References [accessed 20.02.17]. [28] Dobson SR, McNeil S, Dionne M, Dawar M, Ogilvie G, Krajden M, et al. [1] Jit M, Brisson M, Portnoy A, Hutubessy R. Cost-effectiveness of female human Immunogenicity of 2 doses of HPV vaccine in younger adolescents vs 3 doses papillomavirus vaccination in 179 countries: a PRIME modelling study. Lancet in young women: a randomized clinical trial. JAMA 2013;309:1793–802. Glob Health 2014;2:e406–14. [29] Romanowski B, Schwarz TF, Ferguson LM, Peters K, Dionne M, Schulze K, et al. [2] Muñoz N, Castellsagué X, de Gonzalez AB, Gissmann L. Chapter 1: HPV in the Immunogenicity and safety of the HPV-16/18 AS04-adjuvanted vaccine etiology of human cancer. Vaccine 2006;24:S1–S10. administered as a 2-dose schedule compared with the licensed 3-dose [3] Global Burden of Disease Cancer Collaboration, Fitzmaurice C, Dicker D, Pain A, schedule: results from a randomized study. Hum Vaccin 2011;7:1374–86. Hamavid H, Moradi-Lakeh M, et al. The global burden of cancer 013. JAMA [30] Sankaranarayanan R, Prabhu PR, Pawlita M, Gheit T, Bhatla N, Muwonge R, Oncol 2015;1:505–27. et al. Immunogenicity and HPV infection after one, two, and three doses of [4] Agosti JM, Goldie SJ. Introducing HPV vaccine in developing countries–key quadrivalent HPV vaccine in girls in India: a multicentre prospective cohort challenges and issues. N Engl J Med 2007;356:1908–10. study. Lancet Oncol 2016;17:67–77. [5] Kreimer AR, Rodriguez AC, Hildesheim A, Herrero R, Porras C, Schiffman M, [31] Lazcano-Ponce E, Stanley M, Muñoz N, Torres L, Cruz-Valdez A, Salmerón J, et al. Proof-of-principle evaluation of the efficacy of fewer than three doses of a et al. Overcoming barriers to HPV vaccination: non-inferiority of antibody bivalent HPV16/18 vaccine. J Natl Cancer Inst 2011;103:1444–51. response to human papillomavirus 16/18 vaccine in adolescents vaccinated [6] Einstein MH. Module 19: human papillomavirus infection. The immunological with a two-dose vs. a three-dose schedule at 21 months. Vaccine basis for immunization series. Geneva: World Health Organization; 2010. 2014;32:725–32. [7] Stanley MA, Sudenga SL, Giuliano AR. Alternative dosage schedules with HPV [32] GlaxoSmithKline. (GSK_114700) Immunogenicity and safety study of virus-like particle vaccines. Expert Rev Vaccines 2014;13:1027–38. GlaxoSmithKline Biologicals’ HPV-16/18 L1 AS04 vaccine when administered [8] World Health Organization. Human papillomavirus vaccines. WHO position according to alternative 2-dose schedules in 9–14 year old females (results); paper, October 2014. Wkly Epidemiol Rec 2014;89:465–92. 2014. [9] The Future II Study Group. Quadrivalent vaccine against human [accessed 11.08.16]. papillomavirus to prevent high-grade cervical lesions. N Engl Med [33] Sow PS, Watson-Jones D, Kiviat N, Changalucha J, Mbaye KD, Brown J, et al. 2007;356:1915–27. Safety and immunogenicity of human papillomavirus-16/18 AS04-adjuvanted [10] Paavonen J, Naud P, Salmeron J, Wheeler CM, Chow SN, Apter D, et al. Efficacy vaccine: a randomized trial in 10–25-year-old HIV-Seronegative African girls of human papillomavirus (HPV)-16/18 AS04-adjuvanted vaccine against and young women. J Infect Dis 2013;207:1753–63. http://dx.doi.org/10.093/ cervical infection and precancer caused by oncogenic HPV types (PATRICIA): infdis/jis619. Epub 2012 Dec 13. final analysis of a double-blind, randomised study in young women. Lancet [34] Esposito S, Birlutiu V, Jarcuska P, Perino A, Man SC, Vladareanu R, et al. 2009;374:301–14. Immunogenicity and safety of human papillomavirus-16/18 AS04-adjuvanted M. D’Addario et al. / Vaccine 35 (2017) 2892–2901 2901

vaccine administered according to an alternative dosing schedule compared [41] Shetty P. Vaccine trial’s ethics criticized. Collapsed trial fuels unfounded with the standard dosing schedule in healthy women aged 15 to 25 years: vaccine fears. Nature 2011;474:427–8. results from a randomized study. Pediatr Infect Dis J 2011;30:e49–55. [42] Joura EA, Kjaer SK, Wheeler CM, Sigurdsson K, Iversen OE, Hernandez-Avila M, [35] GlaxoSmithKline. (GSK_110659) Evaluation of the safety and immunogenicity et al. HPV antibody levels and clinical efficacy following administration of a of GSK Biologicals’ HPV vaccine 580299 when administered in healthy females prophylactic quadrivalent HPV vaccine. Vaccine 2008;26:6844–51. aged 9–25 years using an alternative schedule and an alternative dosing as [43] Donken R, Bogaards JA, van der Klis FR, Meijer CJ, de Melker HE. An exploration compared to the standard schedule and dosing 110659 (HPV-048 PRI) of individual- and population-level impact of the 2-dose HPV vaccination (results); 2014. [accessed 11.06.16]. [44] Jit M, Brisson M, Laprise JF, Choi YH. Comparison of two dose and three dose [36] Puthanakit T, Huang LM, Chiu CH, Tang RB, Schwarz TF, Esposito S, et al. human papillomavirus vaccine schedules: cost effectiveness analysis based on Randomized open trial comparing 2-dose regimens of the human transmission model. BMJ 2015;350:g7584. papillomavirus 16/18 AS04-adjuvanted vaccine in girls aged 9–14 years [45] Jit M, Choi YH, Laprise JF, Boily MC, Drolet M, Brisson M. Two-dose strategies versus a 3-dose regimen in women aged 15–25 years. J Infect Dis 2016;214 for human papillomavirus vaccination: how well do they need to protect? (4):525–36. http://dx.doi.org/10.1093/infdis/jiw036. Epub 2016 Feb 3. Vaccine 2014;32:3237–42. [37] Hernandez-Avila M, Torres-Ibarra L, Stanley M, Salmeron J, Cruz-Valdez A, [46] Kreimer AR, Struyf F, Del Rosario-Raymundo MR, Hildesheim A, Skinner SR, Munoz N, et al. Evaluation of the immunogenicity of the quadrivalent HPV Wacholder S, et al. Efficacy of fewer than three doses of an HPV-16/18 AS04- vaccine using 2 versus 3 doses at month 21: an epidemiological surveillance adjuvanted vaccine: combined analysis of data from the Costa Rica Vaccine mechanism for alternate vaccination schemes. Hum Vaccin Immun and PATRICIA trials. Lancet Oncol 2015;16(7):775–86. http://dx.doi.org/ 2016;12:30–8. 10.1016/S1470-2045(15)00047-9. Epub 2015 Jun 9. [38] ClinicalTrials.gov. (NCT01717118) Evaluation of immunogenicity levels in [47] Swiss Federal Office of Public Health. HPV-Impfung: ergänzende women with HPV vaccine in Mexico. [accessed 11.08.16]. Bull 2015;10:141–9. [39] Sankaranarayanan R. Trial of two versus three doses of human papillomavirus [48] Kreimer AR, Sherman ME, Sahasrabuddhe VV, Safaeian M. The case for (HPV) vaccine in India; 2009. conducting a randomized clinical trial to assess the efficacy of a single dose of [accessed 11.08.16]. prophylactic HPV vaccines among adolescents. J Natl Cancer Inst 2015;107(3). [40] Ali H, Donovan B, Wand H, Read TR, Regan DG, Grulich AE, et al. Genital warts http://dx.doi.org/10.1093/jnci/dju436. pii: dju436. in young Australians five years into national human papillomavirus vaccination programme: national surveillance data. BMJ 2013;346:f2032. Long-Term Follow-up Observation of the Safety, Immunogenicity, and Effectiveness of GardasilTM in Adult Women

Joaquin Luna1*, Manuel Plata2, Mauricio Gonzalez1, Alfonso Correa3, Ivete Maldonado4, Claudia Nossa5, David Radley6, Scott Vuocolo6, Richard M. Haupt6, Alfred Saah6 1 Instituto Nacional de Cancerologia, Bogota´, Colombia, 2 Fundacion Cardioinfantil, Bogota´, Colombia, 3 Clinica del Country, Bogota´, Colombia, 4 Fundacion Santa Fe de Bogota´, Bogota´, Colombia, 5 Cafam, Bogota´, Colombia, 6 Merck Sharp & Dohme Corp., Whitehouse Station, New Jersey, United States of America

Abstract

Background: Previous analyses from a randomized trial in women aged 24–45 have shown the quadrivalent HPV vaccine to be efficacious in the prevention of infection, cervical intraepithelial neoplasia (CIN) and external genital lesions (EGL) related to HPV 6/11/16/18 through 4 years. In this report we present long term follow-up data on the efficacy, safety and immunogenicity of the quadrivalent HPV vaccine in adult women.

Methods: Follow-up data are from a study being conducted in 5 sites in Colombia designed to evaluate the long-term immunogenicity, effectiveness, and safety of the qHPV vaccine in women who were vaccinated at 24 to 45 years of age (in the original vaccine group during the base study [n = 684]) or 29 to 50 years of age (in the original placebo group during the base study [n = 651]). This analysis summarizes data collected as of the year 6 post-vaccination visit relative to day 1 of the base study (median follow-up of 6.26 years) from both the original base study and the Colombian follow-up.

Results: There were no cases of HPV 6/11/16/18-related CIN or EGL during the extended follow-up phase in the per-protocol population. Immunogenicity persists against vaccine-related HPV types, and no evidence of HPV type replacement has been observed. No new serious adverse experiences have been reported.

Conclusions: Vaccination with qHPV vaccine provides generally safe and effective protection from HPV 6-, 11-, 16-, and 18- related genital warts and cervical dysplasia through 6 years following administration to 24–45 year-old women.

Trial Registration: Clinicaltrials.gov NCT00090220

Citation: Luna J, Plata M, Gonzalez M, Correa A, Maldonado I, et al. (2013) Long-Term Follow-up Observation of the Safety, Immunogenicity, and Effectiveness of GardasilTM in Adult Women. PLoS ONE 8(12): e83431. doi:10.1371/journal.pone.0083431 Editor: Sarah Pett, University of New South Wales, Australia Received May 17, 2013; Accepted November 1, 2013; Published December 31, 2013 Copyright: ß 2013 Luna et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. Funding: The study was designed and funded by the sponsor (Merck & Co, Inc.) in collaboration with external investigators and an external data and safety monitoring board. The sponsor collated the data, monitored the conduct of the study, performed the statistical analysis and coordinated the writing of the manuscript with all authors. The authors were actively involved in the collection, analysis or interpretation of the data, the revising of the manuscript for intellectual content, and approved the final manuscript. All authors had access to data and took part in the decision on where to submit the paper for publication. Competing Interests: M. Gonzalez and J. Luna have grants via their institution to serve as an investigator and colposcopist of protocol 019. J. Luna has received honoraria from Merck, Sharp & Dohmeand Abbott Labs and support for travel from Instituto Nacianal de Cancerologia. I. Maldonado and M. Plata have received grants through her institutions for board membership (IM), study conduct (IM & MP), support for travel (MP). M. Plata has received honoraria from Fundacion Cardioinfantil. D. Radley, S. Vuocolo, R. Haupt and A. Saah are employees of Merck and may own stock and/or stock options. This does not alter the authors’ adherence to all the PLOS ONE policies on sharing data and materials. * E-mail: [email protected]

Introduction 11, 16, or 18-related high-grade intraepithelial neoplasia and condyloma in men and women aged 16 to 26 naı¨ve to the Persistent infection of the uterine cervix by 15 to 20 respective vaccine HPV types at enrollment [9,10] In the pivotal carcinogenic human papillomavirus (HPV) genotypes leads to FUTURE II trial (1095) 12,167 women between the ages of 15 the vast majority of cervical cancers [1,2] and related precursor and 26 received three doses of either HPV-6/11/16/18 vaccine or lesions [3]. While all sexually active women are at risk of HPV placebo, administered at day 1, month 2, and month 6. Subjects infection, the incidence of HPV infection peaks soon after the were followed for an average of 3 years after receiving the first onset of sexual activity in most populations [4–6]. Incidence rates dose of vaccine or placebo. Vaccine efficacy for the prevention of tend to decline thereafter, however some women older than age 25 HPV 16/18 disease was 98% (95% CI: 86–100) in the per- are likely to remain at significant risk for acquisition of new HPV protocol susceptible population. In addition, the efficacy of the infections [7,8]. qHPV vaccine has previously been demonstrated in women 24 to The quadrivalent HPV (qHPV) (types 6, 11, 16, 18) L1 virus- 45 years of age participating in an international double blind like particle (VLP) vaccine is highly effective in preventing HPV 6, clinical trial (FUTURE III) [11]. End of study data (mean follow-

PLOS ONE | www.plosone.org 1 December 2013 | Volume 8 | Issue 12 | e83431 Quadrivalent HPV Vaccine in Adult Women up time of 3.8 years) from FUTURE III demonstrated qHPV the ‘‘catch-up vaccination group’’ (CVG). Because there is no vaccine efficacy of 88.7% (95% CI: 78.1, 94.8) against the placebo group for the long term follow-up phase, vaccine efficacy combined incidence of persistent infection, cervical intraepithelial cannot be measured. In lieu of efficacy measurements, effective- neoplasia (CIN) or external genital lesions (EGL) related to vaccine ness of vaccination with qHPV vaccine is assessed by calculating HPV types in the per-protocol population on women aged 24–45 the incidence of disease endpoints in each of the EVG and CVG [12]. Through the end of study follow-up, 91.5%, 92.0%, 97.4% and comparing these rates with those observed in groups and 47.9% of vaccinated women were still considered seropositive vaccinated in previous efficacy studies within Merck’s qHPV to HPV 6, 11, 16 and 18, respectively. vaccine program. No study vaccinations were provided within the In this report we elaborate on previous FUTURE III results by context of this long-term follow-up study. presenting the findings of a long-term follow-up of that study evaluating the effectiveness, immunogenicity and safety of the Safety Objective qHPV vaccine in women originally enrolled into the trial from To observationally describe the general safety of a 3-dose Colombian study sites. Analyses are cumulative incidence since regimen of qHPV vaccine [Human Papillomavirus (Types 6, 11, vaccination, and therefore include base study data from all 16, 18) Recombinant Vaccine] in women 24 to 50 years of age subjects as well as follow-up data from subjects in Colombia only. during a period of 5 to 10 years following vaccine dose one by describing vaccine- or study procedure-related serious adverse Methods experiences (SAEs), SAEs resulting in death, prespecified medical conditions, and pregnancy follow-up outcomes. Base study design The long-term safety profile will be characterized in terms of Between June 18, 2004 and April 30, 2005, 3,819 women incidence of SAEs that a study investigator considers to be between the ages of 24 and 45 years were enrolled at 38 possibly, probably, or definitely related to prior administration of international study sites into a randomized, placebo-controlled, the qHPV vaccine or to a study procedure; incidence of death; double-blind safety, immunogenicity, and efficacy study prespecified medical conditions; and incidence of pregnancy, (NCT00090220). Subjects were enrolled from community health including pregnancy outcomes and fetal or infant condition. centers, academic health centers, and primary health care providers in Colombia, France, Germany, Philippines, Spain, Immunogenicity objective Thailand, and the United States. Descriptions of treatments, endpoints, hypotheses and case definitions have been published To evaluate the kinetics and age-dependence of anti-HPV 6, 11, [11,12]. The institutional review board (IRB) at each participating 16, and 18 responses following a 3-dose regimen of qHPV vaccine center approved the protocol and written informed consent was at approximately 6, 8, and 10 years after vaccination dose one obtained from all subjects. Studies were conducted in conformance among subjects from the qHPV vaccine group of the base study with applicable country or local requirements regarding ethical and approximately 1, 3, and 5 years after vaccination dose one committee review, informed consent and other statutes or among subjects from the placebo group of the base study. The regulations regarding the protection of the rights and welfare of long-term immunogenicity profile will be characterized in terms of human subjects participating in biomedical research. The names geometric means of vaccine-HPV types 6, 11, 16, and 18 titers and of each participating IRB can be seen in the supplementary file seropositivity to the same four vaccine-HPV types. Two different ‘IRB’. assays will be used; the competitive Luminex immunoassay (cLIA) The protocol for this trial and supporting CONSORT checklist and the total IgG assay. are available as supporting information; see Checklist S1 and Protocol S1. Effectiveness objective To observationally describe the incidence of HPV 6/11/16/18 Follow-up study related genital warts or cervical dysplasia, and of HPV 16/18- V501-Protocol 019-21 is a study being conducted in 5 sites in related CIN2+ (CIN grades 2 or 3, cervical AIS, and cervical Colombia and is designed to evaluate the long-term immunoge- cancer) up to 6 years following administration of qHPV vaccine in nicity, effectiveness, and safety of V501, the quadrivalent Human 24 to 50 year-old women. The analyses are cumulative incidence Papillomavirus (Types 6, 11, 16, 18) recombinant vaccine (qHPV since vaccination. Therefore, the base study data from all subjects vaccine) in women who were vaccinated at 24 to 45 years of age contributes for the first 4 years. Only the Colombian subjects have (for those enrolled in the original vaccine group during the follow-up after 4 years. protocol 019 base study) or 29 to 50 years of age (if they were in HPV type replacement will also be observationally described. the original placebo group during the protocol 019 base study). The incidence of non-HPV 6/11/16/18 related genital warts or This interim analysis report summarizes data collected as of the cervical dysplasia, and of non-HPV 16/18-related CIN2+ (CIN year 6 post-vaccination visit (relative to day 1 of the base study). grades 2 or 3, adenocarcinoma in-situ, and cervical cancer) will be Future analyses are planned at year 8 and year 10 (end-of-study assessed in Colombian women. The HPV types that will be studied analysis). The analyses performed are cumulative incidence since are those studied in the base study, specifically HPV 31/33/35/ vaccination. Base study data from all subjects contributes for the 39/45/51/52/56/58/59 and all high-grade lesions due to any first 4 years. However, only subjects from Colombia contribute non-vaccine type. follow-up data after 4 years. This long-term study does not have a placebo group. Subjects Populations studied who were vaccinated with qHPV vaccine in the base study at 24 to All subjects who received at least one dose of the qHPV vaccine 45 years of age are referred to as the ‘‘early vaccination group’’ and have follow-up data, starting from when they have signed (EVG) in this report. Subjects who were vaccinated with placebo consent to enter the long-term study will be included in the in the base study were later vaccinated with a three dose regimen summaries of safety. of qHPV vaccine during the first extension of the base study at 29 The primary effectiveness analysis approach for the EVG will to 50 years of age (V501 protocol 019-10) and are referred to as be per-protocol-efficacy (PPE), as defined during the base study.

PLOS ONE | www.plosone.org 2 December 2013 | Volume 8 | Issue 12 | e83431 Quadrivalent HPV Vaccine in Adult Women

To be eligible for this population, subjects must (i) have received 3 anti-HPV serology testing. No new vaccinations were provided in doses of GARDASILTM within one year, and (ii) have no protocol this long-term follow-up study. violations. Subjects will be considered cases related to a given Cytology specimens were evaluated using the Bethesda HPV type provided the subject was negative to the respective System—2001. For all cytology diagnoses of Atypical Squamous HPV type by serology and PCR prior to vaccination (i.e., during Cells of Undetermined Significance (ASC-US), a central labora- the base study) and PCR negative through month 7. For purposes tory automatically performed reflex HPV testing on residual of endpoint definition, only the Pathology Panel diagnosis will be ThinPrepTM material, using the Digene Hybrid Capture IITM, considered. Supportive analyses will be done for the group naı¨ve to High Risk/Low Risk Probes. If at least 1 probe was positive or if the relevant HPV type (HNRT) and full analysis set (FAS) no result was obtained, the subject was referred for colposcopy. If populations. The HNRT population consists of all subjects who cervical biopsies and/or endocervical curettage (ECC) specimens received at least one vaccination and were seronegative and PCR were obtained, specimens were sent to the central laboratory for negative to the relevant type prior to vaccination. The FAS analysis. Endpoint adjudication was performed by an independent population includes all vaccinated subjects, regardless of HPV pathology panel. status. For the CVG, no serology or swab samples were taken Competitive Luminex Immunoassay immediately prior to vaccination with qHPV vaccine in the The competitive Luminex immunoassay (cLIA) [13] simulta- extension. However, any positivity recorded during the base study neously evaluates the presence of conformational, neutralizing period (day 1 to month 48) was counted. By comparison, the EVG antibodies to the four HPV types present in the quadrivalent subjects had a single time-point of assessment for baseline HPV vaccine; HPV 6, 11, 16 and 18.(11,27) On each VLP there are positivity. neutralizing epitopes that may or may not be HPV type-specific, as well as epitopes that are non-neutralizing.(1,3–8,14,29) The Procedures cLIA is a multiplex assay that involves the displacement of a No new randomization was performed during this long-term phycoerythrin (PE)-labeled HPV type-specific, VLP-conformation follow-up study. The allocation numbers assigned to study subjects dependent, neutralizing monoclonal antibody (mAb): H6.M48, from Colombia during the base study were carried over to this K11.B2, H16.V5, and H18.J4 for the HPV 6, 11, 16, and 18 study. At visits corresponding to years 6, 8 and 10 after the assays, respectively.(11,27) Luminex microspheres are coated with beginning of the base study, subjects underwent medical and HPV type 6, 11, 16 and 18 L1 VLPs and incubated with the PE- gynecologic histories, pelvic exams, ThinPrepTM Pap tests and a labeled mAbs. Displacement of the PE-labeled mAb is an indirect wart/lesion inspection. In addition, serum samples were taken for measure of human serum antibody binding to the monitored

Figure 1. Subject accounting in base study and extension. 804 EVG subjects and 703 CVG subjects took part in the Colombian extension study. Data were available for 684 subjects in the EVG and 651 CVG subjects. doi:10.1371/journal.pone.0083431.g001

PLOS ONE | www.plosone.org 3 December 2013 | Volume 8 | Issue 12 | e83431 Quadrivalent HPV Vaccine in Adult Women neutralizing epitopes relative to the reference standard. Human HP6043 (Biotrend, Destin, FL) was captured as the raw VLP- serum was diluted 1:4 and tested. Titers are reported in arbitrary specific bound total IgG direct binding data. This mAb binds cLIA milliMerck Units per milliliter (cLIA mMU/mL) determined equally to all four human IgG isotypes, IgG1-IgG4.(17) The by the MFI correlation to the reference standard interpolated correlation of MFI units to an arbitrary IgG milliMerck Unit per through a four parameter curve-fitting algorithm. As a unique milliliter (IgG mMU/mL) of VLP-specific IgG was made by reference standard curve is generated for each HPV type, and serially diluting the reference serum and interpolating the MFI because each HPV type employs a type-specific mAb with a data through a 4-parameter curve fitting algorithm. unique binding affinity, the recorded cLIA mMU/mL titers are not equivalent and cannot be directly compared between HPV Statistical analysis types. In the absence of a control group in the long-term follow-up study, it is not possible to perform analyses of efficacy. Instead, Total IgG Luminex Immunoassay effectiveness will be measured in terms of incidence rates of A total IgG Luminex immunoassay (LIA) was developed disease, with analyses performed periodically until the end of the utilizing yeast-derived L1 VLPs of HPV types 6, 11, 16 and 18 follow-up. Incidence rates based on accrued person-years of coupled to a set of distinct fluorescent Luminex microspheres [14]. follow-up will be estimated together with a corresponding 95% Samples were tested in duplicate in this validated research assay confidence interval, calculated using the exact confidence limits of and titers determined relative to a 12-point standard curve. The the binomial distribution, as an approximation to the Poisson median fluorescent intensity (MFI) of the bound phycoerythrin distribution. Analyses will combine events and follow-up time from (PE) tagged mouse anti-human IgG monoclonal antibody clone the base study and, for those participating, from the extension

Table 1. Baseline characteristics of subjects in Colombia at the time of vaccination dose 1 in the base study.

Early Vaccination Catch-up Vaccination Group Group Total Characteristic at Day 1 (qHPV) (N = 804) (N = 703) (N = 1,507)

Gender - % (m) Female 100% (804) 100% (703) 100% (1,507) Age (years) Mean 34.7 39.8 37.1 Standard Deviation 6.3 6.0 6.7 Median 35 40 38 Range 24 to 45 29 to 50 24 to 50 24 to 34 years old 47.5% (382) 24.6% (173) 36.8% (555) 35 to 45 years old 52.5% (422) 53.6% (377) 53.0% (799) .45 years old 0.0% (0/0) 21.8% (153) 10.2% (153) Race - % (m) Black 0.5% (4) 0.4% (3) 0.5% (7) Hispanic American 99.5% (800) 99.6% (700) 99.5% (1,500) Serostatus{ - % (m/n) Positive to HPV 6/11/16/18 29.9% (240/804) 47.4% (333/703) 38.0% (573/1,507) Positive to HPV 6 14.1% (113/804) 27.6% (194/703) 20.4% (307/1,507) Positive to HPV 11 5.0% (40/804) 12.7% (89/703) 8.6% (129/1,507) Positive to HPV 16 14.9% (120/804) 28.9% (203/703) 21.4% (323/1,507) Positive to HPV 18 5.1% (41/804) 5.0% (35/703) 5.0% (76/1,507) PCR status{ - % (m/n) Positive to HPV 6/11/16/18 9.3% (74/797) 24.3% (171/703) 16.3% (245/1,500) Positive to HPV 6 1.6% (13/796) 8.4% (59/703) 4.8% (72/1,499) Positive to HPV 11 0.4% (3/796) 0.9% (6/703) 0.6% (9/1,499) Positive to HPV 16 5.2% (41/796) 13.7% (96/703) 9.1% (137/1,499) Positive to HPV 18 2.4% (19/797) 5.7% (40/703) 3.9% (59/1,500)

Day 1 (qHPV) is the day of injection of dose 1 of the qHPV vaccine. Unless otherwise indicated the percents shown were calculated as 100*(m/N). N = Number of subjects in the indicated vaccination group who received at least 1 dose of the qHPV vaccine. n = Number of subjects with non-missing Day 1 (qHPV) status corresponding to the indicated HPV type. m = Number of subjects belonging to the indicated category. qHPV = Quadrivalent Human Papillomavirus [Types 6, 11, 16, 18] Recombinant Vaccine. {For Catch-Up Vaccination Group, positivity at any time between Day 1 and Month 48 is used. doi:10.1371/journal.pone.0083431.t001

PLOS ONE | www.plosone.org 4 December 2013 | Volume 8 | Issue 12 | e83431 Quadrivalent HPV Vaccine in Adult Women

Table 2. Effectiveness of qHPV vaccination in women 24–45 years of age against HPV 6/11/16/18-related CIN or condyloma (cumulative incidence in the EVG, day 1 to year 6).

Early Vaccination Group (N = 1,910)

n Cases PYR Rate 95% CI

Per-protocol population (PPE) HPV 6/11/16/18-Related CIN or Condyloma 1,617 1 6,705.6 0.0 (0.0, 0.1) By HPV Type HPV 6-Related CIN or Condyloma 1,330 0 5,515.4 0.0 (0.0, 0.1) HPV 11-Related CIN or Condyloma 1,330 0 5,515.4 0.0 (0.0, 0.1) HPV 16-Related CIN or Condyloma 1,351 1 5,611.5 0.0 (0.0, 0.1) HPV 18-Related CIN or Condyloma 1,524 0 6,314.1 0.0 (0.0, 0.1) By Endpoint Type (HPV 6/11/16/18-Related) CIN (any grade) 1,599 1 6,349.8 0.0 (0.0, 0.1) CIN 1 1,599 0 6,352.4 0.0 (0.0, 0.1) CIN 2 or worse 1,599 1 6,349.8 0.0 (0.0, 0.1) CIN 2 1,599 1 6,349.8 0.0 (0.0, 0.1) CIN 3 1,599 0 6,352.4 0.0 (0.0, 0.1) AIS 1,599 0 6,352.4 0.0 (0.0, 0.1) Cervical Cancer 1,599 0 6,352.4 0.0 (0.0, 0.1) Condyloma 1,617 0 6,696.8 0.0 (0.0, 0.1) Naı¨ve to the relevant type population (HNRT) HPV 6/11/16/18-Related CIN or Condyloma 1,863 4 8,511.0 0.0 (0.0, 0.1) By HPV Type HPV 6-Related CIN or Condyloma 1,535 1 7,041.1 0.0 (0.0, 0.1) HPV 11-Related CIN or Condyloma 1,535 0 7,041.6 0.0 (0.0, 0.1) HPV 16-Related CIN or Condyloma 1,572 3 7,187.2 0.0 (0.0, 0.1) HPV 18-Related CIN or Condyloma 1,760 0 8,052.8 0.0 (0.0, 0.0) By Endpoint Type (HPV 6/11/16/18-Related) CIN (any grade) 1,862 3 8,108.1 0.0 (0.0, 0.1) CIN 1 1,862 1 8,110.7 0.0 (0.0, 0.1) CIN 2 or worse 1,862 3 8,109.8 0.0 (0.0, 0.1) CIN 2 1,862 3 8,109.8 0.0 (0.0, 0.1) CIN 3 1,862 1 8,112.3 0.0 (0.0, 0.1) AIS 1,862 0 8,112.4 0.0 (0.0, 0.0) Cervical Cancer 1,862 0 8,112.4 0.0 (0.0, 0.0) Condyloma 1,863 1 8,509.0 0.0 (0.0, 0.1) Full analysis set population (FAS) HPV 6/11/16/18-Related CIN or Condyloma 1,910 36 8,601.8 0.4 (0.3, 0.6) By HPV Type HPV 6-Related CIN or Condyloma 1,910 10 8,702.7 0.1 (0.1, 0.2) HPV 11-Related CIN or Condyloma 1,910 1 8,728.9 0.0 (0.0, 0.1) HPV 16-Related CIN or Condyloma 1,910 25 8,639.0 0.3 (0.2, 0.4) HPV 18-Related CIN or Condyloma 1,910 3 8,724.9 0.0 (0.0, 0.1) By Endpoint Type (HPV 6/11/16/18-Related) CIN (any grade) 1,909 29 8,303.6 0.3 (0.2, 0.5) CIN 1 1,909 17 8,310.6 0.2 (0.1, 0.3) CIN 2 or worse 1,909 21 8,311.9 0.3 (0.2, 0.4) CIN 2 1,909 11 8,313.7 0.1 (0.1, 0.2) CIN 3 1,909 16 8,317.1 0.2 (0.1, 0.3) AIS 1,909 0 8,319.2 0.0 (0.0, 0.0) Cervical Cancer 1,909 0 8,319.2 0.0 (0.0, 0.0) Condyloma 1,910 7 8,698.1 0.1 (0.0, 0.2)

PLOS ONE | www.plosone.org 5 December 2013 | Volume 8 | Issue 12 | e83431 Quadrivalent HPV Vaccine in Adult Women

N = Number of subjects in the indicated group who received at least 1 dose of the qHPV vaccine. n = Number of subjects in the indicated analysis population. PYR = person years at risk; Rate = rate per 100 person years at risk; AIS = Adenocarcinoma in situ; CI = Confidence interval; CIN = Cervical intraepithelial neoplasia; HPV = Human papillomavirus; qHPV = Quadrivalent Human Papillomavirus (Types 6, 11, 16, 18) Recombinant Vaccine. doi:10.1371/journal.pone.0083431.t002 study. Exact binomial confidence intervals were calculated for the median follow-up time post-dose 1 of the vaccine was 1.18 years. proportions of subjects seropositive Table 1 displays selected subject characteristics at the time of first vaccination for subjects enrolled in Colombia. The difference in Results age between the groups reflects vaccination of the CVG at the end of the base study. Figure 1 displays the subject disposition in the base study and its The rates of positivity prior to vaccination in the CVG are extension. A total of 3,817 study subjects were randomized in a 1:1 generally higher than those observed in the EVG at the start of the ratio and vaccinated with either qHPV vaccine (N = 1,910) or base study, partly as a result of increased ascertainment of placebo (N = 1,907), in the context of the base study. This infection and partly because of persistent risk of infection in this extension study is being conducted only in Colombia. Enrollment older age group of women. into the base study in Colombia was 1,610 in total (804 Table 2 displays the cumulative incidence of HPV 6/11/16/18- randomized to and vaccinated with qHPV vaccine, 806 random- related genital warts or cervical dysplasia, from the start of the base ized to and vaccinated with placebo). A total of 1,360 Colombian study through all visits completed before the cutoff date, in the subjects participated in this extension (84% of the subjects enrolled EVG cohort. There was a single case of this endpoint, an HPV 16- in base study in that country). related CIN2, observed in the PPE population during the base This report is comprised of follow-up data from study start study. To date, no additional cases of this endpoint have occurred through the cutoff date of 02-Nov-2011. The median follow-up in follow-up visits of the EVG PPE population. Four (4) cases of time from day 1 of the base study in the EVG was 6.26 years. For HPV 6/11/16/18-related genital warts or cervical dysplasia have the CVG who received qHPV vaccine after the base study, the occurred in the EVG HNRT population (Table 2); all had been

Table 3. Effectiveness of qHPV vaccination in women 24 to 45 years of age against HPV 31/33/35/39/45/51/52/56/58/59-related CIN or condyloma (cumulative incidence, day 1 to year 6 in the EVG FAS population).

Early Vaccination Group (N = 1,910)

Endpoint n Cases PYR Rate 95% CI

HPV 31/33/35/39/45/51/52/56/58/59-Related CIN or Condyloma 1,910 93 8,403.7 1.1 (0.9, 1.4) By HPV Type HPV 31-Related CIN or Condyloma 1,910 16 8,678.1 0.2 (0.1, 0.3) HPV 33-Related CIN or Condyloma 1,910 7 8,711.7 0.1 (0.0, 0.2) HPV 35-Related CIN or Condyloma 1,910 4 8,717.4 0.0 (0.0, 0.1) HPV 39-Related CIN or Condyloma 1,910 17 8,668.3 0.2 (0.1, 0.3) HPV 45-Related CIN or Condyloma 1,910 5 8,720.4 0.1 (0.0, 0.1) HPV 51-Related CIN or Condyloma 1,910 17 8,686.5 0.2 (0.1, 0.3) HPV 52-Related CIN or Condyloma 1,910 13 8,698.5 0.1 (0.1, 0.3) HPV 56-Related CIN or Condyloma 1,910 23 8,654.5 0.3 (0.2, 0.4) HPV 58-Related CIN or Condyloma 1,910 17 8,664.4 0.2 (0.1, 0.3) HPV 59-Related CIN or Condyloma 1,910 6 8,712.4 0.1 (0.0, 0.1) HPV 31/33/35/39/45/51/52/56/58/59-Related CIN or Condyloma 1,910 93 8,403.7 1.1 (0.9, 1.4) By Endpoint Type CIN (any grade) 1,909 93 8,191.1 1.1 (0.9, 1.4) CIN 1 1,909 72 8,210.3 0.9 (0.7, 1.1) CIN 2 or worse 1,909 40 8,296.4 0.5 (0.3, 0.7) CIN 2 1,909 28 8,302.2 0.3 (0.2, 0.5) CIN 3 1,909 23 8,311.0 0.3 (0.2, 0.4) AIS 1,909 0 8,319.2 0.0 (0.0, 0.0) Cervical Cancer 1,909 0 8,319.2 0.0 (0.0, 0.0) Condyloma 1,910 0 8,721.0 0.0 (0.0, 0.0)

N = Number of subjects in the indicated group who received at least 1 dose of the qHPV vaccine. n = Number of subjects in the indicated analysis population. PYR = person years at risk; Rate = rate per 100 person years at risk; AIS = Adenocarcinoma in situ; CI = Confidence interval; CIN = Cervical intraepithelial neoplasia; HPV = Human papillomavirus; qHPV = Quadrivalent Human Papillomavirus (Types 6, 11, 16, 18) Recombinant Vaccine. doi:10.1371/journal.pone.0083431.t003

PLOS ONE | www.plosone.org 6 December 2013 | Volume 8 | Issue 12 | e83431 LSOE|wwpooeog7Dcme 03|Vlm su 2|e83431 | 12 Issue | 8 Volume | 2013 December 7 www.plosone.org | ONE PLOS Table 4. Comparison of 2-year incidence rates of vaccine and non-vaccine HPV type-related endpoints.

A. Per-protocol efficacy (PPE) population (vaccine HPV types only)

Early Vaccination Group Catch-up Vaccination Group (N = 1,910) (N = 1,907)

Endpoint Period n Cases P-Y Rate 95% CI n Cases P-Y Rate 95% CI

HPV 6/11/16/18-Related CIN or Condyloma Day 1 - Year 2 1602 1 2276 0.0 (0.0, 0.2) 1599 8 2265 0.4 (0.2, 0.7) Year 2 - Year 4 1559 0 3024 0.0 (0.0, 0.1) 1550 11 2990 0.4 (0.2, 0.7) Year 4 - Year 6 927 0 1226 0.0 (0.0, 0.3) - - - - - HPV 16/18-Related CIN2 or Worse Day 1 - Year 2 1570 1 2191 0.0 (0.0, 0.3) 1558 2 2174 0.1 (0.0, 0.3) Year 2 - Year 4 1483 0 2828 0.0 (0.0, 0.1) 1475 4 2806 0.1 (0.0, 0.4) Year 4 - Year 6 842 0 1109 0.0 (0.0, 0.3) - - - - - HPV 6/11-Related Condyloma Day 1 - Year 2 1316 0 1872 0.0 (0.0, 0.2) 1316 3 1868 0.2 (0.0, 0.5) Year 2 - Year 4 1285 0 2491 0.0 (0.0, 0.1) 1285 4 2484 0.2 (0.0, 0.4) Year 4 - Year 6 751 0 1002 0.0 (0.0, 0.4) - - - - -

B. Full analysis set (FAS) population.

Early Vaccination Group Catch-up Vaccination Group (N = 1,910) (N = 1,907)

Endpoint Period n Cases P-Y Rate 95% CI n Cases P-Y Rate 95% CI

HPV 6/11/16/18-Related CIN or Condyloma Day 1 to Year 2 1886 29 3627 0.8 (0.5, 1.1) 1884 41 3614 1.1 (0.8, 1.5) Year 2 to Year 4 1757 6 3402 0.2 (0.1, 0.4) 1744 19 3352 0.6 (0.3, 0.9)

Year 4 to Year 6 1040 1 1371 0.1 (0.0, 0.4) - - - - - Women Adult in Vaccine HPV Quadrivalent HPV 16/18-Related CIN2 or Worse Day 1 to Year 2 1862 17 3577 0.5 (0.3, 0.8) 1862 19 3573 0.5 (0.3, 0.8) Year 2 to Year 4 1712 4 3259 0.1 (0.0, 0.3) 1706 8 3236 0.2 (0.1, 0.5) Year 4 to Year 6 978 0 1284 0.0 (0.0, 0.3) - - - - - HPV 6/11-Related Condyloma Day 1 to Year 2 1884 5 3662 0.1 (0.0, 0.3) 1883 8 3657 0.2 (0.1, 0.4) Year 2 to Year 4 1781 2 3449 0.1 (0.0, 0.2) 1776 4 3426 0.1 (0.0, 0.3) Year 4 to Year 6 1041 0 1390 0.0 (0.0, 0.3) - - - - - HPV 31/33/35/39/45/51/52/56/58/59-Related CIN or Condyloma Day 1 to Year 2 1886 63 3593 1.8 (1.3, 2.2) 1884 61 3595 1.7 (1.3, 2.2) Year 2 to Year 4 1727 28 3312 0.8 (0.6, 1.2) 1725 24 3308 0.7 (0.5, 1.1) Quadrivalent HPV Vaccine in Adult Women

reported during the base study. Compared to the per-protocol population, this population additionally includes protocol viola- tors, including those receiving less than 3 vaccinations, and those who became HPV-positive at or prior to month 7 of the base study. In follow-up visits to date, no additional cases of this endpoint have occurred in the EVG HNRT population. A total of 36 cases of HPV 6/11/16/18-related genital warts or cervical dysplasia have been observed in the EVG FAS population (Table 2); all had been reported during the base study. Compared to the HNRT population, this population additionally includes subjects positive to the relevant HPV type at day 1 of the base study. The cumulative incidence of non-vaccine HPV type (HPV 31/ 33/35/39/45/51/52/56/58/59)-related genital warts or cervical dysplasia, from the start of the base study through all visits completed before the cutoff date, in the EVG FAS population can be seen in Table 3. Of the total 93 cases of this endpoint, a single case has been reported subsequent to the base study (an HPV 31- related CIN1 lesion). Table 4 displays the incidence of the vaccine type-related composite effectiveness endpoints in 2-year intervals covering the base study and follow-up period. This table provides context for the incidence of these endpoints observed in the follow-up period to date for the EVG and CVG (no data are available for the CVG from years 4–6) in both the PPE and FAS populations. For HPV 6/11/16/18-related CIN or condyloma (Table 4A), the incidence rate within the PPE CVG during the base study was 0.4/100 person-years for day 1 to year two, and 0.4/100 person-years for year two to year four. By comparison, the two-year incidence rate in year four to year six in the PPE EVG was 0.0/100 person-years,

Early Vaccination Group(N = 1,910) Catch-up Vaccination Group (N = 1,907) with an upper 95% confidence limit of 0.3. No cases have been reported during extension study visits. Similar data were seen for vaccine-related CIN or condyloma in the FAS population (Table 4B). For non-vaccine (HPV 31/33/35/ 39/45/51/52/56/58/59)-related disease, the incidence rate with- in the FAS CVG during the base study was 1.7/100 person-years for day 1 to year 2, and 0.7/100 person-years for year 2 to year 4. By comparison, the incidence rate in year 4 to year 6 in the EVG PeriodYear 4 to Year 6Day 1 to Year 2Year 2 1002 to n Year 4Year 4 to Year 6 1 1862 1707 Cases P-Y 1304 976 32 Rate 0.1 6 95% CI 3570 (0.0, 0.4) 2 0.9 3254is n 0.1, (0.6, - 1.3) 0.2 1280 (0.1, with 0.4) 0.2 1862 (0.0, 0.6) an Cases 1707 P-Yupper - - 18 Rate 95% CI 95% - 6 3573 0.5 confidence 3237 - - (0.3, 0.8) 0.2 - (0.1, 0.4) - limit - of 0.4. - The proportion of subjects who are seropositive to HPV 6, 11, 16 and 18 at month 72 (year 6) from first vaccination, as measured by cLIA and total IgG assays can be seen in Table 5. Month 7 cLIA seropositivity for HPV types 6, 11, 16 and 18 was 97.6%, 97.0%, 97.9% and 96.8%, respectively (data not shown). While some declines are seen in seropositivity after the vaccination course, month 72 cLIA seropositivity was comparable to that observed at month 48. As noted in previous studies, the proportion of subjects seropositive to HPV 18 by the cLIA declines over time because of the nature of the assay. At month 72, overall seropositivity is 45%, whereas for HPV 6, 11 and 16, seropositivity is maintained at approximately 90% or higher. Of note, no cases of HPV 18-related disease have been observed in the EVG PPE population, during the base study or during follow-up to date. Month 72 total IgG seropositivity for HPV 6, 11, 16 and 18 was 87.8%, 84.4%, 98.8% and 81.5%, respectively. It is noticeable that the overall seropositivity for HPV 18 exceeds 80% (Table 5), which is much higher than cLIA seropositivity and due to the particular features of each assay. Subjects 24 to 34 years of age appear to have a slightly higher response to all HPV types in this

Cont. assay, compared to subjects 35 to 45 years of age, although the 95% CI for age-specific GMTs overlap. No serious adverse experiences have been reported in the context of the long-term extension study period. Table S1 displays Endpoint HPV 31/33/35/39/45/51/52/56/58/59-Related CIN2 or Worse N = Number of subjectsP-Y in = person the years; indicated Rate group =doi:10.1371/journal.pone.0083431.t004 rate who per received 100 at person least years 1 at dose risk; of CI the = Confidence qHPV interval; vaccine. CIN = Cervical intraepithelial neoplasia. Table 4. B. Full analysis set (FAS) population. outcomes for pregnancies reported in the long-term extension

PLOS ONE | www.plosone.org 8 December 2013 | Volume 8 | Issue 12 | e83431 Quadrivalent HPV Vaccine in Adult Women

Table 5. Summary of month 72 HPV seropositivity rates by age (per-protocol immunogenicity population participating in follow- up).

Early Vaccination Group (N = 1,910)

24 to 34 Year-olds 35 to 45 Year-olds All Subjects (N = 953) (N = 957) (N = 1,910)

Assay (cLIA) SPR SPR SPR n (95% CI) N (95% CI) n (95% CI)

Anti-HPV 6 cLIA 198 89.4% (84.2%, 93.3%) 270 88.9% (84.5%, 92.4%) 468 89.1% (85.9%, 91.8%) Total IgG 198 87.9% (82.5%, 92.1%) 270 87.8% (83.3%, 91.4%) 468 87.8% (84.5%, 90.6%) Anti-HPV 11 cLIA 196 94.4% (90.2%, 97.2%) 270 90.4% (86.2%, 93.6%) 466 92.1% (89.2%, 94.3%) Total IgG 198 87.4% (81.9%, 91.7%) 270 82.2% (77.1%, 86.6%) 468 84.4% (80.8%, 87.6%) Anti-HPV 16 cLIA 197 97.5% (94.2%, 99.2%) 276 97.1% (94.4%, 98.7%) 473 97.3% (95.3%, 98.5%) Total IgG 197 100% (98.1%, 100%) 276 99.6% (98.0%, 100%) 473 99.8% (98.8%, 100%) Anti-HPV 18 cLIA 237 48.5% (42.0%, 55.1%) 293 42.7% (36.9%, 48.5%) 530 45.3% (41.0%, 49.6%) Total IgG 237 84.4% (79.1%, 88.8%) 293 79.2% (74.1%, 83.7%) 530 81.5% (77.9%, 84.7%)

N = Number of subjects in the indicated group who received at least 1 dose of the qHPV vaccine. n = Number of subjects with non-missing titer in the indicated analysis population. CI = Confidence interval; cLIA = Competitive Luminex immunoassay; total IgG = total IgG assay. doi:10.1371/journal.pone.0083431.t005 study period, after the base study. Of the four known outcomes to The immunogenicity data from the cLIA showed persisting high date, all have resulted in normal live births. Approximately 13% of titers over 72 months and showed no significant reduction in all subjects had at least one new medical condition reported in the GMT from month 48 to month 72. Similarities in GMTs between long-term extension study period. The most commonly reported the two age strata across all HPV types and time points could be new medical conditions were bacterial vaginitis, hypothyroidism, seen when comparing the data over 72 months. While some and uterine leiomyoma. The number of subjects with cancer or differences between the two age strata can be seen, these conditions of potential autoimmune etiology was low. The two differences are small and the 95% CIs overlap. Additionally, vaccination groups were generally well balanced with regard to the when the entire PPI population and those women who entered the proportions of subjects reporting specific categories of medical long-term follow-up segment of the study are compared, the conditions. GMTs are similar for all HPV types and time points. No significant reduction in seropositivity from month 48 to Discussion month 72 was seen. Moreover, seropositivity at month 72 is similar to month 24 for each of the HPV types, including HPV 18. This Previous reports have demonstrated the efficacy, safety and observation indicates that effectiveness against HPV 18 related immunogenicity of the qHPV vaccine in women aged 24 to 45 disease persisted when the cLIA was positive in approximately [11,12]. While these earlier reports had a maximum mean follow- 50% of women for the last two years of the base study and for the up time of just under 4 years; the current analysis extends this first two years of the long-term follow-up study. follow-up to a mean of 6.26 years. As described [15,16], the nature of the cLIA in measuring There were no cases of HPV 6/11/16/18-related CIN or antibodies (including HPV 18) is a function of the monoclonal condyloma, HPV 16/18-related CIN 2 or worse or HPV 6/11- antibody used in the competitive assay. When the total IgG assay related condyloma during the follow-up visits in years 4 to 6 in all is used, the proportion of women who are positive at month 72 is of the analysis populations. This finding suggests that qHPV substantially higher (81.5% overall). Despite these immunogenicity vaccine effectiveness continues to be high in this population of findings, effectiveness continued to be observed by the sustained women through 6 years following vaccination. The observed absence of disease due to HPV 18, as well as due to the other HPV person-time of follow-up after the base study was not large enough vaccine types. to demonstrate statistically significant protection against HPV 16/ A drawback of the current study is its localization to subjects 18-related CIN 2 or worse and HPV 6/11-related condyloma. enrolled only from Colombia. While the data are interesting and However, the upper 95% CI of the incidence of HPV 6/11/16/ observational in nature, the generalizability of the findings would 18-related CIN or condyloma was below the 4-year incidence of be greater if enrolling a larger and/or more geographically diverse the same endpoint in the placebo group of the base study. cohort. There was no evidence of HPV type replacement in those Administration of the qHPV vaccine in the base study was women who were vaccinated with qHPV vaccine. Accordingly, generally well tolerated. Moreover, no serious adverse experiences the incidence of non-vaccine HPV types did not increase have been reported in the context of the long-term extension study proportionally over time. period. The proportion of subjects who reported serious adverse

PLOS ONE | www.plosone.org 9 December 2013 | Volume 8 | Issue 12 | e83431 Quadrivalent HPV Vaccine in Adult Women experiences was comparable between the qHPV vaccine group administration of qHPV vaccine to women 24 to 45 years of age is and the placebo group. Of the four known pregnancy outcomes to generally safe and well tolerated through year 6 post-vaccination. date, all have resulted in normal live births. It is clear that HPV vaccination is likely to be beneficial to Supporting Information sexually active adult women as they are at risk of acquiring new HPV infection and related sequelae [8]. However, public health Table S1 Pregnancy outcomes. There were 4 live births in the recommendations for mass vaccination must take into consider- EVG population and none in the CVG population. No fetal losses ation the cost-effectiveness of vaccination programs. Current have occurred. vaccine and implementation cost modelling studies have shown (DOC) that vaccination becomes less cost-effective with the increasing age Checklist S1 CONSORT Checklist. of the target vaccination group, likely due to prior HPV exposure (DOC) in this case. Since the overall cost–benefit becomes progressively less favorable with increasing age, most health authorities have not Protocol S1 Trial Protocol. widely recommended routine vaccination of older women. (DOC) Nevertheless, as documented in this trial and others, sexually active women over the age of 26 also have the potential to benefit Acknowledgments from vaccination and should be allowed the opportunity to choose to be vaccinated on an individual basis. The authors thank all study participants. Merck provided support and funding for the conduct of the study. In summary, we have demonstrated that vaccination with qHPV vaccine provides durable protection from HPV 6-, 11-, 16-, and 18-related genital warts and cervical dysplasia through 6 years Author Contributions following administration to 24–45 year-old women. In addition, Conceived and designed the experiments: RMH AS. Analyzed the data: the qHPV vaccine shows no tendency to select for disease due to DR. Contributed reagents/materials/analysis tools: JL MP MG AC CN HPV types that are not present in the vaccine through 6 years IM. Wrote the paper: SV JL. following vaccination in women 24 to 45 years of age. Lastly,

References 1. Walboomers JMM, Jacobs MV, Manos MM, Bosch FX, Kummer A, et al. 10. Giuliano AR, Palefsky JM, Goldstone S, Moreira ED, Jr., Penny ME, et al. (1999) Human papillomavirus is a necessary cause of invasive cervical cancer (2011) Efficacy of quadrivalent HPV vaccine against HPV Infection and disease worldwide. J Pathol 189: 12–19. in males. N Engl J Med 364: 401–411. 2. Mun˜oz N, Bosch FX, de Sanjose´ S, Herrero R, Castellsague¨ X, et al. (2003) 11. Munoz N, Manalastas R, Pitisuttihum P, Tresukosol D, Monsonego J, et al. Epidemiologic classification of human papillomavirus types associated with (2009) Safety, immunogenicity, and efficacy of quadrivalent HPV (types 6, 11, cervical cancer. N Engl J Med 348: 518–527. 16, 18) recombinant vaccine in adult women between 24 and 45 years of age: a 3. International Agency for Research on Cancer Working Group (2007) Human randomized, double-blind trial. Lancet 373: 1949–1957. Papillomaviruses. 12. Castellsague X, Munoz N, Pitisuttithum P, Ferris D, Monsonego J, et al. (2011) 4. Dunne EF, Unger ER, Sternberg M, McQuillan G, Swan DC, et al. (2007) End-of-study safety, immunogenicity, and efficacy of quadrivalent HPV (types 6, Prevalence of HPV infection among females in the United States. JAMA 297: 11, 16, 18) recombinant vaccine in adult women 24-45 years of age. Br J Cancer 813–819. 105: 28–37. 5. Schiffman M, Kjaer SK (2003) Chapter 2: Natural history of anogenital human 13. Smith JF, Kowalski R, Esser MT, Brown MJ, Bryan JT (2008) Evolution of type- papillomavirus infection and neoplasia. J Natl Cancer Inst Monogr 31: 14–19. specific immunoassays to evaluate the functional immune response to Gardasil: a 6. Jacobs MV, Walboomers JMM, Snijders PJF, Voorhorst FJ, Verheijen RHM, et vaccine for human papillomavirus types 16, 18, 6 and 11. Hum Vaccin 4: 134– al. (2000) Distribution of 37 mucosotropic HPV types in women with 142. cytologically normal cervial smears: the age-related patterns for high-risk and 14. Brown DR, Garland SM, Ferris DG, Joura E, Steben M, et al. (2011) The low-risk types. Int J Cancer 87: 221–227. humoral response to Gardasil(R) over four years as defined by Total IgG and 7. Winer RL, Lee S-K, Hughes JP, Adam DE, Kiviat NB, et al. (2003) Genital competitive Luminex immunoassay. Hum Vaccin 7: epub ahead of print. human papillomavirus infection: incidence and risk factors in a cohort of female university students. Am J Epidemiol 157: 218–226. 15. Schiller JT, Lowy DR (2009) Immunogenicity testing in human papillomavirus 8. Castellsague X, Schneider A, Kaufmann AM, Bosch FX (2009) HPV virus-like-particle vaccine trials. J Infect Dis 200: 166–171. vaccination against cervical cancer in women above 25 years of age: key 16. Rositch AF, Hudgens MG, Backes DM, Moses S, Agot K, et al. (2012) Vaccine- considerations and current perspectives. Gynecol Oncol 115: S15–S23. relevant human papillomavirus (HPV) infections and future acquisition of high- 9. The FUTURE II Study Group (2007) Quadrivalent vaccine against human risk HPV types in men. J Infect Dis 206: 669–677. papillomavirus to prevent high-grade cervical lesions. N Engl J Med 356: 1915– 1927.

PLOS ONE | www.plosone.org 10 December 2013 | Volume 8 | Issue 12 | e83431 Prevalence of HPV After Introduction of the Vaccination Program in the United States Lauri E. Markowitz, MD,a Gui Liu, MPH,a Susan Hariri, PhD,a Martin Steinau, PhD,b Eileen F. Dunne, MD, MPH,a Elizabeth R. Unger, MD, PhDb

BACKGROUND: Since mid-2006, human papillomavirus (HPV) vaccination has been abstract recommended for females aged 11 to 12 years and through 26 years if not previously vaccinated. METHODS: HPV DNA prevalence was analyzed in cervicovaginal specimens from females aged 14 to 34 years in NHANES in the prevaccine era (2003–2006) and 4 years of the vaccine era (2009–2012) according to age group. Prevalence of quadrivalent HPV vaccine (4vHPV) types (HPV-6, -11, -16, and -18) and other HPV type categories were compared between eras. Prevalence among sexually active females aged 14 to 24 years was also analyzed according to vaccination history. RESULTS: Between the prevacccine and vaccine eras, 4vHPV type prevalence declined from 11.5% to 4.3% (adjusted prevalence ratio [aPR]: 0.36 [95% confidence interval (CI): 0.21–0.61]) among females aged 14 to 19 years and from 18.5% to 12.1% (aPR: 0.66 [95% CI: 0.47–0.93]) among females aged 20 to 24 years. There was no decrease in 4vHPV type prevalence in older age groups. Within the vaccine era, among sexually active females aged 14 to 24 years, 4vHPV type prevalence was lower in vaccinated (≥1 dose) compared with unvaccinated females: 2.1% vs 16.9% (aPR: 0.11 [95% CI: 0.05–0.24]). There were no statistically significant changes in other HPV type categories that indicate cross-protection. CONCLUSIONS: Within 6 years of vaccine introduction, there was a 64% decrease in 4vHPV type prevalence among females aged 14 to 19 years and a 34% decrease among those aged 20 to 24 years. This finding extends previous observations of population impact in the United States and demonstrates the first national evidence of impact among females in their 20s.

a Division of STD Prevention, National Center for HIV/AIDS, Viral Hepatitis, STD, and TB Prevention, and bDivision of High-Consequence Pathogens and Pathology, National Center for Emerging and Zoonotic Infectious Diseases, WHAT’S KNOWN ON THIS SUBJECT: Previous Centers for Disease Control and Prevention, Atlanta, Georgia studies have found declines in vaccine type human Dr Markowitz conceptualized and designed the analyses and drafted the manuscript; Ms Liu papillomavirus (HPV) prevalence and genital warts conducted the data analyses and reviewed and revised the manuscript; Dr Hariri assisted with among young females in the United States after study design and data analyses and reviewed and revised the manuscript; Dr Steinau supervised introduction of the HPV vaccination program. laboratory testing and reviewed and revised the manuscript; Dr Dunne assisted with study design WHAT THIS STUDY ADDS: This study extends and data collection and reviewed and revised the manuscript; and Dr Unger assisted with study design, supervised laboratory testing, and reviewed and revised the manuscript. All authors previous observations of quadrivalent HPV vaccine approved the fi nal manuscript as submitted. impact and examines cross-protection. Within 6 years of vaccine introduction, there were decreases The fi ndings and conclusions in this report are those of the authors and do not necessarily in national vaccine type HPV prevalence of 64% and represent the offi cial position of the Centers for Disease Control and Prevention. 34% among females aged 14 to 19 years and 20 to 24 Dr Hariri’s current affi liation is Division of Viral Hepatitis, National Center for HIV/AIDS, Viral years, respectively. Hepatitis, STD, and TB Prevention, Centers for Disease Control and Prevention, Atlanta, GA. Dr Dunne’s current affi liation is Division of HIV/AIDS Prevention, National Center for HIV/AIDS, Viral Hepatitis, STD, and TB Prevention, Centers for Disease Control and Prevention, Atlanta, GA. Dr To cite: Markowitz LE, Liu G, Hariri S, et al. Prevalence of HPV After Introduction of the Vaccination Program in the United States. Pediatrics. 2016;137(2):e20151968

Downloaded from www.aappublications.org/news by guest on March 12, 2019 PEDIATRICS Volume 137 , number 3 , March 2016 :e 20151968 ARTICLE Three prophylactic human METHODS (80.6%) samples were adequate papillomavirus (HPV) vaccines are for DNA typing (as discussed in the available, and they have been shown Survey Design and Population Specimen Collection and Laboratory Methods section). In 2009 to 2012, in clinical trials to have high efficacy NHANES is an ongoing series of a total of 2473 females aged 14 to for prevention of HPV vaccine-type cross-sectional surveys conducted 1–3 34 years were interviewed; 2403 infection and associated disease. by the National Center for Health (97.2%) received an examination The bivalent vaccine targets HPV-16 Statistics, Centers for Disease in an MEC. Of those, 2070 (86.1%) and -18; the quadrivalent vaccine Control and Prevention (CDC). submitted a cervicovaginal swab, (4vHPV) targets HPV-6, -11, -16, The surveys are designed to be and 2061 (85.8%) were adequate for and -18; and the 9-valent vaccine nationally representative of the DNA typing. HPV prevalence testing (9vHPV), licensed at the end of civilian, noninstitutionalized US among males was not included in 2014, targets HPV-6, -11, -16, and population. Surveys are conducted NHANES during this time period. -18 as well as 5 additional HPV in ∼15 counties, which vary each types (31, 33, 45, 52, and 58). HPV year. Consenting participants Demographic, Behavioral, and HPV vaccination has been recommended undergo a household interview Vaccination History for females in the United States followed by a physical examination Demographic information was since mid-2006 and for males since in a mobile examination center ascertained during household 20114; through 2014, almost all (MEC). To increase the precision of interviews. Sexual history 5 estimates, NHANES oversampled vaccines used were 4vHPV. Routine information was collected among certain subpopulations. In 1999 vaccination is recommended for participants aged 14 to 59 years by to 2006, Mexican-Americans, non- females and males aged 11 or 12 using audio computer-assisted self- Hispanic blacks, low-income non- years and for females through age interviews in an MEC. Respondents Hispanic whites and others, as well 26 years and males through age 21 who reported ever having sex as adolescents aged 12 to 19 years, years if not previously vaccinated. (described as vaginal, oral, or anal were oversampled. Adolescents Although rates of HPV vaccination sex) were asked additional questions were not oversampled after 2006. have been increasing in the United about their sexual history, including In 2009 to 2012, all Hispanics were States, coverage is still low; in 2013, age at first sexual encounter, also oversampled and in 2011 to a national survey found that 57% number of lifetime sex partners, and 2012, Asians were also oversampled. of 13- to 17-year-old females had number of sex partners in the past Informed consent or assent was 12 months. NHANES 2003–2004 received at least 1 dose and 38% obtained from all participants and 5 did not collect information about had received 3 doses. Despite consent from guardians of minors. partners in the past 12 months from this moderate coverage, data from This survey was approved by the persons aged 14 to 17 years. HPV NHANES exhibited a 56% decrease National Center for Health Statistics/ vaccination history was collected in 4vHPV type prevalence among CDC Research Ethics Review Board. females aged 14 to 19 years in the beginning in 2007. Persons aged ≥ first 4 years of the vaccine era (2007– NHANES data from 2003 to 2006 16 years and emancipated minors 2010) compared with the prevaccine and 2009 to 2012 were analyzed. were interviewed directly. Parents/ era.6 Because adolescents were not guardians were interviewed oversampled after 2006, there was regarding vaccination history for a reduced number of 14- to 19-year- those aged <16 years. The present report analyzes data olds in the second time period. Years from the 4 most recent years 2003 to 2006 were considered the Specimen Collection and Laboratory available from NHANES (2009–2012) prevaccine era because vaccination Methods and compares HPV prevalence with was first recommended in June 2006. Females aged 14 to 59 years who the prevaccine era (2003–2006). Analyses for the present report were were examined in an MEC were In these more recent years, vaccine limited to participants aged 14 to 34 asked to self-collect a cervicovaginal coverage was higher than in the first years with adequate self-collected sample.8,9 Extractions and testing 4 years of the vaccine era evaluated cervicovaginal samples. In 2003 to were performed at the CDC as previously.6,7 We also explore 2006, a total of 3325 females aged 14 previously described.9 Briefly, vaccine effectiveness, by analyzing to 34 years were interviewed; 3210 extracted DNA was tested by using HPV prevalence according to report (96.5%) received an examination the Research Use Only Linear of vaccination, herd effects, and in an MEC. Of those, 2649 (82.5%) Array HPV Genotyping Test (Roche potential cross- protection against submitted a self-collected Molecular Diagnostics, Indianapolis, non-4vHPV types. cervicovaginal swab, and 2587 IN) with supplementary HPV-52

Downloaded from www.aappublications.org/news by guest on March 12, 2019 2 MARKOWITZ et al quantitative polymerase chain was defined as report of at least 1 Between 2003–2006 and 2009–2012, reaction, as previously described.9 HPV vaccine dose. there were no significant changes This assay uses L1 consensus All estimates were weighted by in the percentage of females who polymerase chain reaction followed using sample weights to account for reported having had sex or in past by type-specific hybridization for unequal probabilities of selection year or lifetime sexual partners in qualitative detection of 37 HPV types and adjustment for nonresponse.12 any age group except 20- to 24-year- (6, 11, 16, 18, 26, 31, 33, 35, 39, 40, Variance estimates were calculated olds. In this age group, a higher 42, 45, 51, 52, 53, 54, 55, 56, 58, 59, by using a Taylor series linearization percentage of participants in 2009– 61, 62, 64, 66, 67, 68, 69, 70, 71, to account for the complex 2012 than in 2003–2006 reported ≥ 72, 73, 81, 82, 83, 84, 89, and IS39) survey design.13 Logit confidence having 2 sex partners in the past β ≥ and -globin (control for sample intervals (CIs) were calculated for year and 3 lifetime sex partners. amplification). Samples that tested prevalence estimates, with an α of β Among females aged 14 to 19 years, negative for both HPV and -globin .05. Prevalence estimates with a were considered inadequate. there were statistically significant relative SE (RSE) >30% or based declines in 4vHPV type prevalence, on <10 cases are noted; these are Data Analysis from 11.5% to 4.3% (aPR: 0.36 considered unstable and should [95% CI: 0.21–0.61]), as well as The 2 most recent NHANES cycles be interpreted with caution. We prevalence of HPV-16, -18, from 7.1% of the vaccine era (2009–2010 calculated prevalence ratios, adjusted to 2.8% between the prevaccine and 2011–2012) were combined prevalence ratios (aPRs), and their and vaccine eras (Table 2). There to achieve stable estimates. The 95% CIs, adjusted for race/ethnicity were no significant differences in 5-year age groups included in the (non-Hispanic black, non-Hispanic prevalence of HPV-31, -33, -45 in present report are those in which an white, Mexican American, and the prevaccine (4.3%) and vaccine impact of vaccination would first be other), and lifetime and past year (2.6%, RSE >30%) eras or the 5 observed (14–19, 20–24, and 25–29 number of sex partners. Because additional 9vHPV types (8.4% in years) and the next oldest age group data on past year sex partners were the prevacccine era and 6.2% in the (30–34 years). We analyzed report of not available for 14- to 17-year- vaccine era). Among females aged 20 at least 1 dose and 3-dose vaccination olds in 2003 to 2004, this variable to 24 years, there were statistically history in 2009–2012 and compared was not included in some models. significant declines in 4vHPV type sexual behavior in 2009–2012 versus The prevalence ratio was the prevalence, from 18.5% to 12.1% 2003–2006. predicted probability calculated (aPR: 0.66 [95% CI: 0.47–0.93]), from the logistic regression model and prevalence of HPV-16, -18, from HPV prevalence was compared by using the PREDMARG statement 15.2% to 10.5%. No statistically among females in 2003–2006 and 14 in SAS-Callable SUDAAN. Vaccine significant changes were observed 2009–2012 according to age group. effectiveness was calculated as 100 × in 4vHPV type prevalence among HPV type categories investigated P (1 – aPR). Throughout the analyses, females aged 25 to 29 years or 30 include any of 37 HPV types, 4vHPV values were not adjusted for multiple to 34 years. For individual types types (6, 11, 16, and 18), 4vHPV comparisons. Statistical analyses examined in the 2 youngest age high-risk types (16 and 18), any were conducted in SAS version 9.3 groups, significant differences non-4vHPV types, non-4vHPV high- (SAS Institute, Inc, Cary, NC) and between the prevaccine and vaccine risk types (31, 33, 35, 39, 45, 51, 52, SUDAAN version 11.0 (Research eras were observed for HPV-6, -16, 56, 58, 59, 66, and 68), 3 high-risk Triangle Institute, Research Triangle and -56 among females aged 14 to 19 types (31, 33, and 45) for which Park, NC). years and for HPV-51 among those some cross-protection has been 20 to 24 years (Supplemental Fig 1, suggested10,11; and the 5 additional Supplemental Tables 5 and 6). high-risk types in 9vHPV (31, 33, RESULTS 45, 52, and 58).3 In analyses limited to sexually active females aged 14 HPV Prevalence Among All Females HPV Prevalence Among Sexually Active Females Aged 14 to 24 Years, to 24 years, we compared lifetime Aged 14 to 34 Years According to Age Group Overall and According to Vaccination sex partners and race/ethnicity History according to vaccination status in the In NHANES 2009–2012, receipt of vaccine era and compared these data at least 1 HPV vaccine dose was Analyses according to vaccination with data from the prevaccine era. reported by 51.4% of females aged history were limited to sexually Vaccine effectiveness was evaluated 14 to 19 years, 32.6% aged 20 to 24 active females aged 14 to 24 for prevention of vaccine type HPV years, 14.7% aged 25 to 29 years, and years, combining the 2 age groups detection in 2009–2012; vaccination 3.3% aged 30 to 34 years (Table 1). (14–19 years and 20–24 years)

Downloaded from www.aappublications.org/news by guest on March 12, 2019 PEDIATRICS Volume 137 , number 3 , March 2016 3 TABLE 1 Selected Sexual Behavior and Reported Vaccination History in Females Aged 14 to 34 Years, Within the vaccine era, there was According to Age Group, NHANES 2003–2006 and 2009–2012 no difference in the percentages of Age Group/Characteristic Prevaccine Era Vaccine Era sexually active females reporting ≥3 2003–2006 2009–2012 lifetime sex partners in vaccinated % (95% CI) % (95% CI) compared with those unvaccinated 14–19 y n = 1363 n = 736 (Table 3). A larger percentage of HPV vaccination history vaccinated was non-Hispanic white. ≥1 dose NA 51.4 (47.3–55.3) The 4vHPV type prevalence was 3 doses NA 34.6 (30.3–39.2) lower among vaccinated compared Sexual behavior with unvaccinated females (2.1% Ever had sex 53.8 (50.6–56.9) 50.4 (45.0–55.8) ≥2 sex partners in past year 40.4 (35.1–46.0)a 42.3 (36.8–48.1) vs 16.9%; aPR: 0.11 [95% CI: 0.05– ≥3 lifetime sex partners 47.6 (42.3–52.9) 52.2 (45.6–58.8) 0.24]), corresponding to a vaccine 20–24 y n = 432 n = 470 effectiveness of 89% (Table 4). There HPV vaccination history were no statistically significant ≥ 1 dose NA 32.6 (26.5–39.3) differences between vaccinated 3 doses NA 18.1 (13.0–24.6) Sexual behavior and unvaccinated females in the Ever had sex 91.3 (86.1–94.7) 90.8 (85.9–94.1) prevalence of any HPV; non-4vHPV ≥2 sex partners in past year* 25.6 (21.0–30.7) 34.6 (29.4–40.3) high-risk types; HPV-31, -33, -45; or ≥3 lifetime sex partners* 66.4 (60.8–71.5) 77.4 (72.8–81.4) the 5 additional 9vHPV types. 25–29 y n = 403 n = 424 HPV vaccination history ≥1 dose NA 14.7 (10.9–19.5) 3 doses NA 8.8 (5.2–14.4) DISCUSSION Sexual behavior Ever had sex 95.0 (91.8–97.0) 94.6 (91.1–96.7) In this study of HPV prevalence in ≥2 sex partners in past year 22.6 (18.5–27.4) 20.6 (15.5–27.0) cervicovaginal specimens from a ≥3 lifetime sex partners 77.0 (69.8–83.0) 76.1 (70.1–81.3) nationally representative sample 30–34 y n = 389 n = 428 of females in the United States, we HPV vaccination history extend our previous findings of ≥1 dose NA 3.3b (1.5–6.9) 3 doses NA 1.2b (0.3–5.9) vaccine impact with 2 additional Sexual behavior years of data from NHANES.6 Our Ever had sex 97.6 (95.0–98.9) 99.2 (98.1–99.6) analysis using the 4 most recent ≥ 2 sex partners in past year 12.8 (9.5–17.1) 11.9 (8.5–16.4) years of data from the vaccine era ≥3 lifetime sex partners 74.8 (69.9–79.1) 74.1 (67.8–79.5) (2009–2012) showed that among Number of past year and lifetime partners among those who reported ever having sex. HPV vaccination history is all females aged 14 to 19 years according to self-report or parent report. NA, not applicable. a Data limited to 2005 to 2006. and 20 to 24 years, 4vHPV type b RSE >30% or <10 cases. prevalence decreased 64% and * χ2 P < .05 from the Wald test. 34%, respectively, compared with the prevaccine era. There was no in which declines in 4vHPV type era, 4vHPV type prevalence was statistically significant change in prevalence were observed between significantly lower overall in the prevalence of any HPV or other the prevaccine and vaccine eras. vaccine era (18.6% vs 10.8%; aPR: categories of HPV types in these age Compared with the prevaccine era 0.53 [95% CI: 0.40–0.71]) and among groups and no decline in 4vHPV type (58.4%), in the vaccine era, a larger those vaccinated (18.6% vs 2.1%; prevalence among females aged 24 to percentage of females aged 14 to aPR: 0.09 [95% CI: 0.05–0.18]). The 29 years or 30 to 34 years. 24 years overall (67.7%) and those 4vHPV type prevalence among those The greatest decline in 4vHPV unvaccinated (70.0%) reported unvaccinated in the vaccine era was ≥ type prevalence observed in 14- to 3 lifetime sex partners (Table 3). not significantly different from the There were no statistically significant 19-year-olds is consistent with the prevaccine era (18.6% vs 16.9%). differences in race/ethnicity (percent highest reported vaccine coverage Overall or according to vaccination non-Hispanic white shown in Table in this age group. Of note, the history, there were no statistically 3). percentage of females aged 14 to 19 significant differences in the vaccine years who reported having received Any HPV type prevalence was similar era compared with the prevaccine at least 1 vaccine dose in NHANES in the prevaccine era (54.4%) and era in the prevalence of non-4vHPV 2009–2012 (51%) is similar to data the vaccine era (58.1%) (Table high-risk types; HPV-31, -33, -45; or from national coverage surveys. 4). Compared with the prevaccine the 5 additional 9vHPV types. Among females aged 13 to 17 years

Downloaded from www.aappublications.org/news by guest on March 12, 2019 4 MARKOWITZ et al TABLE 2 HPV Prevalence Among Females Aged 14 to 34 Years, According to Age Group, NHANES 2003–2006 and 2009–2012 Age Group/HPV Types Prevaccine Era 2003–2006 Vaccine Era 2009–2012 Comparison of Vaccine Era With Prevaccine Era % (95% CI) % (95% CI) PR (95% CI) aPR (95% CI) 14–19 y n = 1363 n = 736 Any HPV 32.9 (29.5–36.4) 29.0 (24.5–33.9) 0.88 (0.73–1.07) 0.93 (0.79–1.09) Non-4vHPV 31.2 (28.0–34.7) 28.4 (24.0–33.2) 0.91 (0.75–1.11) 0.96 (0.82–1.14) Non-4vHPV HR 20.7 (17.9–23.9) 18.6 (14.7–23.4) 0.90 (0.68–1.18) 0.99 (0.79–1.26) HPV-31, -33, -45 4.3 (3.1–6.1) 2.6a (1.2–5.5) 0.59 (0.25–1.38) 0.66 (0.27–1.59) HPV-31, -33, -45, -52, -58 8.4 (6.7–10.5) 6.2 (4.1–9.3) 0.74 (0.46–1.19) 0.82 (0.53–1.28) 4vHPV 11.5 (9.2–14.4) 4.3 (2.7–6.8) 0.37 (0.22–0.63)** 0.36 (0.21–0.61)** HPV-16, -18 7.1 (5.8–8.7) 2.8 (1.6–4.7) 0.39 (0.22–0.68)** 0.37 (0.20–0.67)** 20–24 y n = 432 n = 470 Any HPV 53.7 (46.0–61.3) 57.9 (52.5–63.2) 1.08 (0.91–1.28) 1.02 (0.88–1.18) Non-4vHPV 50.7 (43.6–57.9) 56.1 (50.4–61.6) 1.10 (0.93–1.32) 1.05 (0.90–1.23) Non-4vHPV HR 32.9 (26.8–39.6) 36.8 (30.8–43.3) 1.12 (0.87–1.45) 1.11 (0.85–1.44) HPV-31, -33, -45 7.8 (5.0–12.0) 5.4 (3.6–8.2) 0.70 (0.38–1.27) 0.85 (0.48–1.50) HPV-31, -33, -45, -52, -58 16.5 (11.4–23.2) 12.7 (8.6–18.2) 0.77 (0.46–1.29) 0.85 (0.51–1.41) 4vHPV 18.5 (14.9–22.7) 12.1 (9.1–16.0) 0.66 (0.46–0.93)* 0.66 (0.47–0.93)* HPV-16, -18 15.2 (11.7–19.5) 10.5 (7.7–14.2) 0.69 (0.47–1.03) 0.66 (0.45–0.97)* 25–29 y n = 403 n = 424 Any HPV 46.8 (42.9–50.7) 49.1 (43.8–54.5) 1.05 (0.92–1.20) 1.06 (0.90–1.24) Non-4vHPV 43.8 (39.0–48.8) 47.5 (42.1–53.0) 1.08 (0.92–1.27) 1.09 (0.91–1.31) Non-4vHPV HR 24.6 (19.1–31.0) 28.1 (23.9–32.9) 1.14 (0.86–1.53) 1.09 (0.79–1.50) HPV-31, -33, -45 5.8 (3.7–9.0) 6.2 (3.7–10.0) 1.07 (0.55–2.08) 1.27 (0.68–2.38) HPV-31, -33, -45, -52, -58 10.8 (7.3–15.6) 13.4 (10.1–17.6) 1.25 (0.78–1.99) 1.34 (0.81–2.21) 4vHPV 11.8 (8.8–15.5) 11.7 (8.7–15.4) 0.99 (0.66–1.48) 1.04 (0.72–1.50) HPV-16, -18 8.1 (6.1–10.7) 9.9 (7.3–13.3) 1.22 (0.81–1.85) 1.17 (0.79–1.73) 30–34 y n = 389 n = 428 Any HPV 47.9 (42.4–53.4) 40.3 (34.7–46.1) 0.84 (0.70–1.01) 0.80 (0.66–0.98)* Non-4vHPV 44.5 (39.2–50.0) 37.9 (32.1–44.1) 0.85 (0.70–1.04) 0.83 (0.67–1.02) Non-4vHPV HR 21.0 (15.6–27.6) 21.6 (17.6–26.3) 1.03 (0.73–1.46) 0.98 (0.67–1.42) HPV-31, -33, -45 4.1a (2.2–7.5) 4.2 (2.8–6.2) 1.02 (0.48–2.14) 0.85 (0.40–1.80) HPV-31, -33, -45, -52, -58 9.8 (7.1–13.4) 7.5 (5.5–10.1) 0.76 (0.49–1.18) 0.58 (0.35–0.97)* 4vHPV 9.5 (6.8–13.1) 8.0 (5.8–10.9) 0.84 (0.53–1.32) 0.77 (0.48–1.24) HPV-16, -18 7.6 (5.0–11.2) 6.6 (4.5–9.5) 0.87 (0.50–1.50) 0.78 (0.45–1.36) Data are for all females, including those who did not report having had sex. aPR was adjusted for race/ethnicity and lifetime and past year sex partners (14- to 19-year-old age group not adjusted for past year partners). Non-4vHPV high-risk (HR) includes types -31, -33, -35, -39, -45, -51, -52, -56, -58, -59, -66, and -68. 4vHPV includes types -6, -11, -16, and -18. PR, prevalence ratio. *P < .05; **P < .01 obtained by using the logistic regression model. a RSE >30%. with provider-verified records in the from 44.3% to 53.8% between 2009 available from national surveys for National Immunization Survey–Teen and 2012.7 Although provider- persons aged ≥18 years, the 2012 receipt of at least 1 dose ranged verified vaccine history is not National Health Interview Survey

TABLE 3 Characteristics of Sexually Active Females Aged 14 to 24 Years, Overall and According to Vaccination History, NHANES 2003–2006 and 2009–2012 Variable % (95% CI) Prevalence Ratio (95% CI) Prevaccine Era 2003–2006 Vaccine Era 2009–2012 Comparison of Vaccine Era Comparison Within (n = 1092) (n = 753a) With Prevaccine Erab Vaccine Erac ≥3 lifetime sex partners Overall 58.4 (54.9–61.7) 67.7 (64.1–71.1) 1.16 (1.07–1.25) NA Vaccinated NA 64.9 (56.7–72.3) 1.11 (0.98–1.27) 0.93 (0.78–1.11) Unvaccinated NA 70.0 (63.9–75.4) 1.20 (1.09–1.32) Ref Non-Hispanic white Overall 64.0 (56.7–70.7) 56.8 (49.1–64.2) 0.89 (0.75–1.05) NA Vaccinated NA 63.9 (54.0–72.8) 1.00 (0.83–1.20) 1.19 (1.01–1.40) Unvaccinated NA 53.7 (45.4–61.7) 0.84 (0.70–1.01) Ref NA, not applicable. a A total of 287 sexually active females were vaccinated (defi ned as a history of receipt of ≥1 vaccine dose), and 439 were unvaccinated. Data for 27 sexually active females who had no information on vaccination status are included in the overall group. b Overall, vaccinated, and unvaccinated in 2009–2012 compared with overall in 2003–2006. c Vaccinated compared with unvaccinated in 2009–2012.

Downloaded from www.aappublications.org/news by guest on March 12, 2019 PEDIATRICS Volume 137 , number 3 , March 2016 5 TABLE 4 HPV Prevalence Among Sexually Active Females Aged 14 to 24 Years, Overall and According to Vaccination History, NHANES 2003–2006 and 2009–2012 HPV Types/Vaccination History Prevalence, % (95% CI) aPR (95% CI) Prevaccine Era 2003–2006 Vaccine Era 2009–2012 Comparison of Vaccine Era Comparison Within (n = 1092) (n = 753a) With Prevaccine Erab Vaccine Erac Any HPV Overall 54.4 (49.5–59.2) 58.1 (52.9–63.1) 1.00 (0.90–1.11) NA Vaccinated NA 56.7 (50.6–62.6) 1.01 (0.89–1.14) 0.99 (0.85–1.15) Unvaccinated NA 59.5 (52.3–66.4) 1.00 (0.87–1.16) Ref Non-4vHPV HR Overall 32.8 (28.9–37.0) 38.0 (32.8–43.4) 1.09 (0.93–1.28) NA Vaccinated NA 37.2 (31.1–43.7) 1.11 (0.91–1.34) 0.98 (0.75–1.27) Unvaccinated NA 39.1 (31.6–47.1) 1.10 (0.88–1.36) Ref HPV-31, -33, and -45 Overall 6.6 (4.7–9.2) 5.3 (3.7–7.6) 0.76 (0.48–1.20) NA Vaccinated NA 4.9 (2.9–8.1) 0.74 (0.41–1.34) 0.95 (0.50–1.83) Unvaccinated NA 5.8 (3.6–9.1) 0.80 (0.46–1.37) Ref HPV-31, -33, -45, -52, and -58 Overall 14.7 (11.8–18.1) 13.1 (10.2–16.7) 0.83 (0.60–1.15) NA Vaccinated NA 14.3 (10.1–19.9) 0.95 (0.64–1.42) 1.22 (0.81–1.84) Unvaccinated NA 12.5 (9.2–16.8) 0.76 (0.53–1.10) Ref 4vHPV Overall 18.6 (16.1–21.3) 10.8 (8.2–14.2) 0.53 (0.40–0.71) NA Vaccinated NA 2.1 (1.1–3.7) 0.09 (0.05–0.18) 0.11 (0.05–0.24) Unvaccinated NA 16.9 (12.3–22.6) 0.83 (0.61–1.12) Ref Non-4vHPV high-risk (HR) includes types -31, -33, -35, -39, -45, -51, -52, -56, -58, -59, -66, and -68. 4vHPV includes types -6, -11, -16, and -18. Vaccinated assessment was made by self-report or parent-reported receipt of at least 1 HPV vaccine dose. NA, not applicable. a A total of 287 sexually active females were vaccinated (defi ned as a history of receipt of ≥1 vaccine dose), and 439 were unvaccinated. Data for 27 sexually active females who had no information on vaccination status are included in the overall group. b Prevalence overall, among vaccinated and among unvaccinated in 2009–2012 compared with overall in 2003–2006, adjusted for race/ethnicity and number of lifetime sexual partners. c Prevalence in vaccinated compared with unvaccinated in 2009–2012, adjusted for race/ethnicity and number of lifetime sexual partners. found that 44% of women aged 19 the first 4 years of the vaccine era, significant decline compared with to 21 years and 28% aged 22 to 26 it was difficult to assess herd effects the prevaccine era. We also found years reported receipt of at least 1 because there were differences no vaccine effectiveness against dose of HPV vaccine.15 In NHANES in sexual behavior and lower these 3 types among females aged 2009–2012, 33% of women aged 20 prevalence of any HPV type among 14 to 24 years in the vaccine era. to 24 years reported receipt of at those unvaccinated compared with Prelicensure clinical trials of both least 1 dose. the prevaccine era.6 In the present quadrivalent and bivalent vaccines analysis, a similar percentage of investigated cross-protection against In addition to our analysis of vaccinated and unvaccinated females persistent infection and cervical prevalence overall in the prevaccine reported >3 lifetime lifetime sex intraepithelial neoplasia due to and vaccine eras, we estimated partners as well as prevalence of nonvaccine high-risk types.10,18–20 vaccine effectiveness and explored any HPV type. Although we cannot Trials of bivalent HPV vaccine herd effects and potential cross- claim evidence of herd effects in found more evidence of cross- protection among sexually active this analysis, herd effects have been protection than did trials of 4vHPV. 14- to 24-year-olds. High vaccine observed for both genital warts and Other postlicensure prevalence effectiveness was found for 4vHPV type prevalence in countries evaluations have investigated cross- prevention of 4vHPV types within in which higher vaccination coverage protection.16,21,22 In England and the vaccine era (89%) and there was has been achieved.16,17 Scotland, where the bivalent HPV a large decline in prevalence among vaccine was introduced, decreases those vaccinated compared with the We also examined the prevalence were observed in HPV-16, -18, as overall prevalence in the prevaccine of 3 HPV types for which there well as related types, in the vaccine era (18.6% to 2.1%). Among those has been some evidence of cross- era.21,22 In Australia, where 4vHPV unvaccinated, 4vHPV type prevalence protection.10,11 The prevalence was introduced, HPV-6, -11, -16, 18 was 18.6% in the prevaccine era and estimate for any HPV-31, -33, -45 prevalence among women aged 18 16.9% in the vaccine era; any HPV in the vaccine era among all female to 24 years decreased from 37.6% to prevalence remained stable. In our subjects aged 14 to 19 years was 6.5%.16 For HPV-31, -33, -45, there previous analysis, using data from unstable; there was no statistically was no statistically significant change

Downloaded from www.aappublications.org/news by guest on March 12, 2019 6 MARKOWITZ et al between the prevaccine and vaccine between the prevaccine and vaccine in vaccine type HPV might not be eras; within the vaccine era, however, eras, these unadjusted comparisons observed, particularly if there is any a significant 58% effectiveness should be interpreted with caution. increase in sexual risk behavior in the was observed. Further analyses of Prevalence of individual types will population (as we found in women NHANES data will assess changes in continue to be evaluated in future aged 20–24 years). HPV prevalence in the United States cycles of NHANES. Our data confirm previous findings as HPV vaccine coverage increases. of an early impact of HPV vaccination However, introduction of 9vHPV will Limitations of NHANES data have 6 in the United States among females make it more difficult to evaluate been described. First, vaccination history is by self-report in NHANES, aged 14 to 19 years and extend the changes due to potential 4vHPV findings to females in their early cross-protection for these 3 types and overreporting or underreporting 27,28 20s. The decline in vaccine type beyond 2015.23 could have occurred. In the United States, there is wide variation of prevalence after introduction of HPV Finally, we also examined prevalence HPV vaccine coverage by state, with vaccination is greater than expected of the 5 additional 9vHPV types to coverage of ≥1 dose ranging from based on current 3-dose coverage. provide a baseline before potential 29% to 73% in 2012.29 NHANES This outcome could be due to herd introduction of this vaccine. In the design does not allow state-specific protection or effectiveness of less than prevaccine era, the prevalence of prevalence estimates; each cycle is a complete 3-dose series, for which 30,31 any 5 additional types in 9vHPV designed to include a representative there is accumulating evidence. was 8.4% and 16.5% in females sample of the US population. Third, There are now data on population aged 14 to 19 years and 20 to 24 starting in 2007, adolescents were not impact of HPV vaccines from many 32 years, respectively, similar to the oversampled. Some of our analyses countries ; most reports of declines prevalence of HPV-16 and -18 in were limited by small sample size, in HPV vaccine type prevalence are those age groups. Of note, these 5 and some prevalence estimates were from countries with higher coverage 16,21,22 types cause substantially less HPV- unstable. Although we adjusted than the United States. Data associated cancer than HPV-16, the analyses for race/ethnicity and from NHANES contribute to the -18 because they are less likely to some sexual behaviors, we cannot increasing body of evidence on HPV 24 progress to cancer. In the United exclude the possibility of changes or vaccine impact and will continue to States, it is estimated that 66% of differences in sexual behaviors that provide important information as cervical cancers are attributable were not measured by NHANES. coverage increases and vaccine policy to HPV-16 and -18 compared with changes in the United States.23 15% attributable to the 5 additional Ongoing analyses of NHANES will 9vHPV types.25,26 There was no allow monitoring of HPV vaccine statistically significant difference in impact on HPV prevalence, duration ABBREVIATIONS prevalence of the 5 additional 9vHPV of protection, possible cross- 4vHPV: quadrivalent HPV vaccine types in the prevaccine/vaccine era protection, or type replacement. 9vHPV: 9-valent HPV vaccine comparison or in the comparison of To date, there is no clear indication aPR: adjusted prevalence ratio vaccinated and unvaccinated sexually that type replacement is occurring. CDC: Centers for Disease Control active females aged 14 to 24 years Given the low contribution of vaccine and Prevention within the vaccine era. Although types to the overall prevalence of CI: confidence interval there were some statistically HPV in the population and because HPV: human papillomavirus significant differences in prevalence co-infections occur, a decrease in any MEC: mobile examination center of individual non-4vHPV types HPV prevalence due to the declines

Markowitz's and Ms Liu's current affi liations are Division of Viral Diseases, National Center for Immunization and Respiratory Diseases, Centers for Disease Control and Prevention, Atlanta, GA. DOI: 10.1542/peds.2015-1968 Accepted for publication Oct 30, 2015 Address correspondence to Lauri E. Markowitz, MD, Centers for Disease Control and Prevention, 1600 Clifton Rd NE, MS A-34, Atlanta, GA 30329. E-mail: lem2@cdc. gov PEDIATRICS (ISSN Numbers: Print, 0031-4005; Online, 1098-4275). PEDIATRICS (ISSN Numbers: Print, 0031-4005; Online, 1098-4275); published in the public domain by the American Academy of Pediatrics FINANCIAL DISCLOSURE: The authors have indicated they have no fi nancial relationships relevant to this article to disclose.

Downloaded from www.aappublications.org/news by guest on March 12, 2019 PEDIATRICS Volume 137 , number 3 , March 2016 7 FUNDING: Supported by the Centers for Disease Control and Prevention. POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential confl icts of interest to disclose.

REFERENCES 1. Paavonen J, Jenkins D, Bosch FX, et States. MMWR Morb Mortal Wkly Rep. The near disappearance of genital al; HPV PATRICIA study group. Effi cacy 2013;62(29):591–595 warts in young women 4 years after of a prophylactic adjuvanted bivalent 8. Dunne EF, Unger ER, Sternberg M, et commencing a national human L1 virus-like-particle vaccine against al. Prevalence of HPV infection among papillomavirus (HPV) vaccination infection with human papillomavirus females in the United States. JAMA. programme. Sex Transm Infect. types 16 and 18 in young women: an 2007;297(8):813–819 2011;87(7):544–547 interim analysis of a phase III double- 18. Wheeler CM, Kjaer SK, Sigurdsson blind, randomised controlled trial. 9. Hariri S, Unger ER, Sternberg M, K, et al. The impact of quadrivalent Lancet. 2007;369(9580):2161–2170 et al. Prevalence of genital human papillomavirus among females in the human papillomavirus (HPV; types 6, 2. FUTURE II Study Group. Quadrivalent United States, the National Health And 11, 16, and 18) L1 virus-like particle vaccine against human papillomavirus Nutrition Examination Survey, 2003- vaccine on infection and disease due to prevent high-grade cervical lesions. 2006. J Infect Dis. 2011;204(4):566–573 to oncogenic nonvaccine HPV types N Engl J Med. 2007;356(19):1915–1927 in sexually active women aged 16-26 10. Brown DR, Kjaer SK, Sigurdsson K, years. J Infect Dis. 2009;199(7):936–944 3. Joura EA, Giuliano AR, Iversen OE, et al. The impact of quadrivalent 19. Wheeler CM, Castellsagué X, Garland et al; Broad Spectrum HPV Vaccine human papillomavirus (HPV; types 6, SM, et al; HPV PATRICIA Study Group. Study. A 9-valent HPV vaccine 11, 16, and 18) L1 virus-like particle Cross-protective effi cacy of HPV-16/18 against infection and intraepithelial vaccine on infection and disease due AS04-adjuvanted vaccine against neoplasia in women. N Engl J Med. to oncogenic nonvaccine HPV types in cervical infection and precancer 2015;372(8):711–723 generally HPV-naive women aged 16-26 years. J Infect Dis. 2009;199(7):926–935 caused by non-vaccine oncogenic HPV 4. Markowitz LE, Dunne EF, Saraiya types: 4-year end-of-study analysis of M, et al; Centers for Disease 11. Schiller JT, Castellsagué X, Garland the randomised, double-blind PATRICIA Control and Prevention (CDC). SM. A review of clinical trials of human trial. Lancet Oncol. 2012;13(1):100–110 Human papillomavirus vaccination: papillomavirus prophylactic vaccines. Vaccine. 2012;30(suppl 5):F123–F138 20. Herrero R, Hildesheim A, Bratti C, et recommendations of the Advisory al. Population-based study of human Committee on Immunization 12. Design and estimation for the National papillomavirus infection and cervical Practices (ACIP). MMWR Recomm Rep. Health Interview Survey, 1995-2004. neoplasia in rural Costa Rica. J Natl 2014;63(RR-05):1–30 Vital Health Stat 2. 2000;(130):1–31 Cancer Inst. 2000;92(6):464–474 5. Stokley S, Jeyarajah J, Yankey D, et 13. Korn EL, Graubard BI. Epidemiologic 21. Kavanagh K, Pollock KG, Potts A, al; Immunization Services Division, studies utilizing surveys: accounting et al. Introduction and sustained National Center for Immunization and for the sampling design. Am J Public high coverage of the HPV bivalent Respiratory Diseases, CDC; Centers for Health. 1991;81(9):1166–1173 vaccine leads to a reduction in Disease Control and Prevention (CDC). 14. Bieler GS, Brown GG, Williams RL, prevalence of HPV 16/18 and closely Human papillomavirus vaccination Brogan DJ. Estimating model-adjusted related HPV types. Br J Cancer. coverage among adolescents, 2007- risks, risk differences, and risk ratios 2014;110(11):2804–2811 2013, and postlicensure vaccine from complex survey data. Am J 22. Mesher D, Soldan K, Howell-Jones safety monitoring, 2006-2014—United Epidemiol. 2010;171(5):618–623 R, et al. Reduction in HPV 16/18 States. MMWR Morb Mortal Wkly Rep. 15. Williams WW, Lu PJ, O’Halloran A, prevalence in sexually active young 2014;63(29):620–624 et al; Centers for Disease Control women following the introduction of 6. Markowitz LE, Hariri S, Lin C, et al. and Prevention (CDC). Noninfl uenza HPV immunisation in England. Vaccine. Reduction in human papillomavirus vaccination coverage among adults— 2013;32(1):26–32 (HPV) prevalence among young women United States, 2012. MMWR Morb 23. Petrosky E, Bocchini JA Jr, Hariri S, following HPV vaccine introduction in Mortal Wkly Rep. 2014;63(5):95–102 et al; Centers for Disease Control the United States, National Health and 16. Tabrizi SN, Brotherton JM, Kaldor and Prevention (CDC). Use of 9-valent Nutrition Examination Surveys, 2003- JM, et al. Assessment of herd human papillomavirus (HPV) 2010. J Infect Dis. 2013;208(3):385–393 immunity and cross-protection after vaccine: updated HPV vaccination recommendations of the Advisory 7. Centers for Disease Control a human papillomavirus vaccination Committee on Immunization Practices. and Prevention (CDC). Human programme in Australia: a repeat MMWR Morb Mortal Wkly Rep. papillomavirus vaccination coverage cross-sectional study. Lancet Infect 2015;64(11):300–304 among adolescent girls, 2007-2012, Dis. 2014;14(10):958–966 and postlicensure vaccine safety 17. Read TR, Hocking JS, Chen MY, 24. Serrano B, Alemany L, Tous S, et al. monitoring, 2006-2013—United Donovan B, Bradshaw CS, Fairley CK. Potential impact of a nine-valent

Downloaded from www.aappublications.org/news by guest on March 12, 2019 8 MARKOWITZ et al vaccine in human papillomavirus state vaccination coverage among HPV vaccine in younger adolescents related cervical disease. Infect Agent adolescents aged 13 through vs 3 doses in young women: a Cancer. 2012;7(1):38 17 years—United States, 2010. randomized clinical trial. JAMA. 25. Saraiya M, Unger ER, Thompson TD, et MMWR Morb Mortal Wkly Rep. 2013;309(17):1793–1802 2011;60(33):1117–1123 al; HPV Typing of Cancers Workgroup. 31. Blomberg M, Dehlendorff C, Sand C, US assessment of HPV types in 28. Dorell CG, Jain N, Yankey D. Validity of Kjaer SK. Dose-related differences cancers: implications for current and parent-reported vaccination status for in effectiveness of human 9-valent HPV vaccines. J Natl Cancer adolescents aged 13-17 years: National papillomavirus vaccination against Inst. 2015;107(6):djv086 Immunization Survey-Teen, 2008. Public genital warts: a nationwide study of 26. Hopenhayn C, Christian A, Christian Health Rep. 2011;126(suppl 2):60–69 550,000 young girls. Clin Infect Dis. WJ, et al. Prevalence of human 29. Centers for Disease Control and 2015;61(5):676–682 papillomavirus types in invasive Prevention (CDC). National and 32. Drolet M, Bénard É, Boily MC, et cervical cancers from 7 US state vaccination coverage among al. Population-level impact and cancer registries before vaccine adolescents aged 13-17 years—United herd effects following human introduction. J Low Genit Tract Dis. States, 2012. MMWR Morb Mortal Wkly papillomavirus vaccination 2014;18(2):182–189 Rep. 2013;62(34):685–693 programmes: a systematic review 27. Centers for Disease Control and 30. Dobson SR, McNeil S, Dionne M, and meta-analysis. Lancet Infect Dis. Prevention (CDC). National and et al. Immunogenicity of 2 doses of 2015;15(5):565–580

Downloaded from www.aappublications.org/news by guest on March 12, 2019 PEDIATRICS Volume 137 , number 3 , March 2016 9 Prevalence of HPV After Introduction of the Vaccination Program in the United States Lauri E. Markowitz, Gui Liu, Susan Hariri, Martin Steinau, Eileen F. Dunne and Elizabeth R. Unger Pediatrics 2016;137; DOI: 10.1542/peds.2015-1968 originally published online February 22, 2016;

Updated Information & including high resolution figures, can be found at: Services http://pediatrics.aappublications.org/content/137/3/e20151968 References This article cites 32 articles, 1 of which you can access for free at: http://pediatrics.aappublications.org/content/137/3/e20151968#BIBL Subspecialty Collections This article, along with others on similar topics, appears in the following collection(s): Infectious Disease http://www.aappublications.org/cgi/collection/infectious_diseases_su b Vaccine/Immunization http://www.aappublications.org/cgi/collection/vaccine:immunization _sub Sexually Transmitted Infections http://www.aappublications.org/cgi/collection/sexually_transmitted_i nfections_new_sub Permissions & Licensing Information about reproducing this article in parts (figures, tables) or in its entirety can be found online at: http://www.aappublications.org/site/misc/Permissions.xhtml Reprints Information about ordering reprints can be found online: http://www.aappublications.org/site/misc/reprints.xhtml

Downloaded from www.aappublications.org/news by guest on March 12, 2019 Prevalence of HPV After Introduction of the Vaccination Program in the United States Lauri E. Markowitz, Gui Liu, Susan Hariri, Martin Steinau, Eileen F. Dunne and Elizabeth R. Unger Pediatrics 2016;137; DOI: 10.1542/peds.2015-1968 originally published online February 22, 2016;

The online version of this article, along with updated information and services, is located on the World Wide Web at: http://pediatrics.aappublications.org/content/137/3/e20151968

Data Supplement at: http://pediatrics.aappublications.org/content/suppl/2016/02/17/peds.2015-1968.DCSupplemental

Pediatrics is the official journal of the American Academy of Pediatrics. A monthly publication, it has been published continuously since 1948. Pediatrics is owned, published, and trademarked by the American Academy of Pediatrics, 141 Northwest Point Boulevard, Elk Grove Village, Illinois, 60007. Copyright © 2016 by the American Academy of Pediatrics. All rights reserved. Print ISSN: 1073-0397.

Downloaded from www.aappublications.org/news by guest on March 12, 2019 Human Papillomavirus Chelse Spinner,a​ Lili Ding,b,​ ​c David I. Bernstein,​b,​c Darron R. Brown,​d Eduardo VaccineL. Franco,​e Courtney Covert, Effectiveness​c Jessica A. Kahn, MD, MPHb,​c and Herd Protection in Young Women BACKGROUND: abstract

Clinical trials of the 4-valent human papillomavirus (HPV) vaccine demonstrate high efficacy, but surveillance studies are essential to examine the long-term impact of vaccine introduction on HPV prevalence in community settings. The aims of this study were to determine during the 11 years after vaccine introduction the prevalence of (1) vaccine- type HPV in adolescent and young adult women who were vaccinated (to assess vaccine effectiveness) and (2) vaccine-type HPV in women who were unvaccinated (to assess herd METHODS: protection). Young women 13 to 26 years of age were recruited from hospital-based and community health clinics for 4 surveillance studies from 2006 to 2017. We determined the proportion of vaccinated and unvaccinated women who were positive for vaccine-type HPV – across the studies, and the odds of positivity for vaccine-type HPV using logistic regression; all analyses were propensity score adjusted to control for between-wave differences in RESULTS: participant characteristics. – Vaccination rates increased from 0% to 84.3% (97% of study participants received the 4-valent vaccine). Among women who were vaccinated, 4-valent vaccine type HPV – detection decreased from 35% to 6.7% (80.9% decline; odds ratio 0.13, 95% confidence interval 0.08 to 0.22). Among women who were unvaccinated, 4-valent vaccine type HPV detection decreased from 32.4% to 19.4% (40% decline; odds ratio 0.50, 95% confidence interval 0.26 to 0.97). Estimated vaccine effectiveness was 90.6% in wave 3 and 80.1% in CONCLUSIONS: wave 4. In this study in which trends in HPV in a US community >10 years after 4-valent HPV vaccine introduction and after 9-valent vaccine introduction were examined, we – found evidence of vaccine effectiveness and herd protection. Further research is needed to NIH examine trends in 9-valent vaccine type HPV after higher rates of vaccination are achieved. aUniversity of Cincinnati, bCollege of Medicine, Cincinnati, Ohio; cCincinnati Children’s Hospital Medical Center, WHAT’S KNOWN ON THIS SUBJECT: Researchers in clinical Cincinnati, Ohio; dIndiana University, Indianapolis, Indiana; and eMcGill University, Montreal, Canada trials of the 4-valent human papillomavirus (HPV) vaccine demonstrate high efficacy, but surveillance studies are Ms Spinner assisted with the design of the analyses, did the literature search, interpreted the essential to examine the long-term impact of vaccine results, and codrafted the initial manuscript; Dr Ding designed and conducted the statistical introduction on HPV prevalence in community settings. analyses, interpreted the results, and critically revised the manuscript for intellectual content; Drs Bernstein and Franco assisted with study conceptualization and design, interpreted the WHAT THIS STUDY ADDS: In this study in which trends results, and critically revised the manuscript for intellectual content; Dr Brown assisted in HPV in a US community >10 years after 4-valent with study conceptualization and design, conducted the human papillomavirus DNA analyses, HPV vaccine introduction and after 9-valent vaccine interpreted the results, and critically revised the manuscript for intellectual content; Ms Covert introduction are examined, we found evidence of vaccine recruited participants for the study and interpreted the results; Dr Kahn conceptualized and effectiveness and herd protection. designed the study, obtained funding for the study, interpreted the results, codrafted the initial

To cite: Spinner C, Ding L, Bernstein DI, et al. Human Papillomavirus Vaccine Effectiveness and Herd Protection in Young Women. Pediatrics. 2019;143(2):e20181902

Downloaded from www.aappublications.org/news by guest on February 5, 2019 PEDIATRICS Volume 143, number 2, February 2019:e20181902 ARTICLE METHODS

Infection with human papillomavirus prevalence after vaccine introduction (HPV) may cause genital warts and among women who are vaccinated cancers. In women, HPV infection and women who are unvaccinated, The study population comprised may cause cervical, vaginal, vulvar, to examine HPV prevalence among young women recruited from a anal, and oropharyngeal cancers, younger women who are in the university-affiliated, hospital-based ’ whereas –in men, infection may cause target age group for vaccination, and primary care clinic (Cincinnati anal, penile,1 3 and oropharyngeal to examine the impact of 9-valent Children s Hospital Teen Health cancers. ‍‍ The first prophylactic vaccine introduction on these trends. Center) and the Cincinnati Health HPV vaccine, a 4-valent vaccine that Assessments of vaccine effectiveness Department (obstetrics and prevents HPV-6, -11, -16, and -18, and herd protection are essential gynecology clinic and a sexually ’ was licensed in 2006 in the United to guide public health messaging, transmitted disease clinic). The study 4 ’ States. A 2-valent vaccine that clinical counseling, vaccination was approved by the hospital s and prevents HPV-165 and -18 was recommendations, and cervical health department s institutional licensed in 2009,​ and a 9-valent cancer screening recommendations. review boards, and participants vaccine that prevents HPV-6, -11, provided written informed consent.

-16, -18, -31, -33, -45, 6-52, and -58 To evaluate effectiveness and Parental consent was waived for was licensed in 2014. The 9-valent herd protection after 4-valent and participants <18 years of age to vaccine, the only vaccine available 9-valent HPV vaccine introduction protect patient confidentiality in the United States as of the end in a community over the 11 years because history of sexual contact was of 2016, prevents 5 additional after vaccine introduction, we an inclusion criterion. To participate, oncogenic HPV types (-31, -33, -45, ∼ – designed a study to15, 22​extend our individuals had to be 13 to 26 years -52, and -58) and could prevent7 10 previous findings ‍ with the of age and sexually experienced, 90% of cervical cancers. ‍ following primary specific aims: (1) which was defined as having had to determine trends in vaccine-type sexual contact (oral-genital or

Through evidence from clinical trials, HPV (types targeted by the 4-valent genital-genital, with22 a male or researchers have demonstrated and 9-valent HPV vaccines) among female partner). Individuals were that all 3 HPV vaccines have high young women who are vaccinated to not eligible if they had participated efficacy in preventing infection and– examine vaccine effectiveness and in previous surveillance studies. disease caused by the specific HPV7,11​ 14 (2) to determine trends in vaccine- Participants were enrolled by using types targeted by the vaccines. ‍ ‍ type HPV among young women a sequential recruitment strategy; However, vaccine effectiveness in who are unvaccinated to assess for 95% to 98% of those approached in community settings may be lower; evidence of herd protection. We each study agreed to participate. We – – N women in the community may hypothesized that (1) the prevalence collected the following 4 waves of – N have been infected with vaccine- of 4-valent vaccine type HPV would data: wave 1 (2006 2007, = 371), – N type HPV before vaccination and decrease significantly from 2006 wave 2 (2009 2010, = 409), wave – N may have lower compliance with to 2017 among women who were 3 (2013 2014, = 400), and wave the vaccination series or be less vaccinated, which would indicate 4 (2016 2017, = 400). A survey – healthy than those in the clinical vaccine effectiveness, and that (2) the instrument was used to collect data trials. Researchers in studies have prevalence of 4-valent vaccine type on sociodemographic and behavioral demonstrated that after introduction HPV would decrease significantly variables that may be risk factors for of the 2-valent and 4-valent vaccines, in women who were unvaccinated, type-specific HPV infection; details there is a substantial reduction which would suggest herd protection. about survey development and in the prevalence of vaccine-type As an exploratory specific aim, we validity 15,are22​ described in previous HPV as well as precancers in real- examined trends in the prevalence of articles. ‍ Cervicovaginal swabs world settings among– women who the 5 additional types in the 9-valent were collected by self-swab or are vaccinated,15 indicating19 vaccine vaccine (HPV-31, -33, -45, -52, and clinician swab from each female effectiveness. ‍ Studies are also -58). Decreases in the prevalence participant, and samples were emerging in which researchers of the 5 additional types in the genotyped for HPV by using a Roche demonstrate a reduction in vaccine- 9-valent vaccine may be driven Linear Array test, a polymerase chain type HPV among women who are either by direct protection after reaction amplification technique unvaccinated,15,16,​ suggesting18​ herd vaccination with the 9-valent vaccine that uses an L1 consensus primer protection,​ ‍ ‍ but17,20,​ findings21​ or crossprotection after vaccination system and a reverse line blot are not consistent. ‍ Longer- with the 4-valent vaccine, given that detection strip23 to identify 36 HPV term surveillance is essential to these 5 types are genetically related genotypes. In previous studies, establish trends in vaccine-type HPV to HPV-16 and HPV-18. researchers have demonstrated Downloaded from www.aappublications.org/news by guest on February 5, 2019 2 SPINNER et al .5 — — — — — .69 .07 .50 .55 .97 .58 .29 .29 .15 .45 .42 .78 .22 .53 Adjusted Propensity Score ​ a P, — — — — — .18 .39 .22 .02 .48 .01 .02 .13 .69 .0001 .0014 .0006 ˂ <.0001 < .0001 < .0001 Unadjusted a ​ P, — — — — — — — — — — — — — — — — — — — 19.1 (2.6) Wave 4, N = 400 — N (%) Mean (SD) 6 (1.5) 52 (13) 64 (16) 21 (5.3) 284 (71) 44 (11.1) 83 (21.3) 47 (11.8) 68 (17.4) 67 (16.9) 93 (23.3) 91 (22.8) 307 (76.8) 215 (53.8) 311 (78.7) 314 (79.3) 353 (88.3) 105 (26.3) 309 (77.3) — — — — — — — — — — — — — — — — — — — 19.1 (2.8) Wave 3, N = 400 — N (%) Mean (SD) 9 (2.3) 35 (8.8) 28 (7.0) 63 (15.8) 81 (20.3) 58 (14.5) 77 (19.3) 86 (21.9) 96 (24.0) 150 (37.5) 108 (27.0) 269 (67.3) 361 (90.3) 342 (85.5) 250 (62.5) 202 (50.5) 327 (82.2) 139 (34.8) 292 (73.0) — — — — — — — — — — — — — — — — — — — 18.8 (2.9) Wave 2, N = 409 — N (%) Mean (SD) 16 (3.9) 24 (5.9) 85 (20.8) 93 (22.7) 76 (18.6) 63 (15.4) 86 (21.0) 141 (34.5) 114 (27.9) 217 (53.1) 380 (92.9) 333 (81.4) 268 (65.5) 212 (52.0) 354 (87.8) 146 (38.5) 117 (29.1) 295 (72.1) 129 (31.5) — — — — — — — — — — — — — — — — — — — 18.7 (3.0) Wave 1, N = 371 — N (%) Mean (SD) 24 (6.7) 25 (6.9) 32 (8.6) 76 (21.5) 89 (25.3) 73 (19.7) 71 (20.1) 132 (35.6) 110 (30.1) 196 (52.8) 319 (89.1) 298 (80.3) 239 (64.4) 170 (46.5) 282 (80.6) 121 (37.5) 114 (31.8) 255 (69.9) 143 (38.5) ticipant Characteristics Across All 4 Study Waves, Unadjusted and Adjusted for Propensity Score test, Fisher ’ s exact Kruskal-Wallis or analysis of variance. 2 tners in tners in tner male ≤ 13 y of age tment tner tner in the past 3 mo  Comparison of Par

multiracial White or Asian American Medicaid Less than every time Every time intercourse male par par intercourse cigarettes in lifetime lifetime ≥ 2 the past 3 mo ≥ 2 African American or Private None or not sure Teen Health Center Health depar Race and/or ethnicity History of any STI Age of first sexual No. male sexual par No. male sexual par Main sexual par Ever had anal sex with a Condom use with main Condom use at last sexual Smoked at least 100 and medical history Age, y Appalachian descent Hispanic ethnicity Health insurance plan

Enrollment site Behaviors Characteristic Demographic characteristics

P value calculated by using χ TABLE 1 — , not applicable. a

Downloaded from www.aappublications.org/news by guest on February 5, 2019 PEDIATRICS Volume 143, number 2, February 2019 3 b b b 48.1] b b b b 6.4) [ − [ − 80.9] [ − 68.8] [ − 61.8] [ − 75.1] Wave 1 – 4 17.2 to − − 28.2 ( 33.8 to 22.7) − 16.1 ( 21.3 to 11.0) − 28.5 ( 34.8 to 22.2) − 26.0 ( 31.6 to 20.4) 11.8 ( − − 1.3 ( 14.3 to 11.8) [ 3.0] − 13 ( 23.8 to 2.2) [ 40.1] − − 33.1 ( 39.4 to 26.8) [ 71] b b 48.2] 1.9) [ − Wave 3 – 4 11.8 to − +0.5 ( − 3.4 to 4.3) [4.9] − 4.7 ( 9.6 to 0.2) [ 28] +3.5 (+0.05 to 6.9) [103] − 2.9 ( 15.2 to 9.4) [ 13] 6.8 ( − − 3.2 ( 8.9 to 2.5) [ 19.2] − 3.4 ( 18.5 to 11.7) [ 7.5] − 4.7 ( 10.2 to 0.8) [ 21.1] − b b b 32.3] 66.7] b b b 2.5) [ − 2.3) [ − [ − 47] [ − 34] [ − 48.8] Wave 2 – 3 10.9 to − 13.6 to − − 12.4 ( 19.3 to 5.5) − 11.5 ( 17.6 to 5.4) − 15.9 ( 23.2 to 8.5) 6.6 ( − +5.3 ( − 4.3 to 14.9) [31.2] 8.1 ( − − − 5.3 ( 9.6 to 1.0) [ 39.3] − b b b b b [ − 61] [ − 26.7] [ − 71.7] [ − 47.5] Between-Wave Changes in HPV Prevalence, Propensity Score Adjusted, % (95% CI) [% Decline or Increase] Wave 1 – 2 − 5.1 to +7.0) [3.8] − 6.7 to +9.0) [4.8] +6.4 ( − 4.4 to 17.1) [26.7] +5.8 ( − 8.7 to 20.2) [18.8] +13.2 ( − 0.8 to +25.7) [57.6] − 3.9 to +10.2) [13.7] 12.3 ( − 19.2 to 5.5) 15.4 ( − 22.9 to 7.9) 21.1 ( − 27.0 to 15.3) 25.1 ( − 31.2 to 18.9) − − − − 8.3 ( − 17.3 to 0.6) [ 19.4] +10.5 ( − 1.3 to 22.2) [30] +1.0 ( +1.1 ( 14 ( − 21.8 to 6.2) [ 30] +3.1 ( − − 400 8.6 6.7 Wave 4, N = 400 22.316.7 17.6 45.5 13.5 8.2 42.1 3.3 22.3 19.4 16.814.1 12.1 7.3 30.4 36.1 Wave 3, N = Adjusted, % 409 Wave 2, N = 371 46.146.6 33.8 43.4 32.6 34.6 35.0 35.0 13.5 32.4 9.9 17 23.923.4 24.8 26.6 22.9 24 Wave 1, N = c a d  Prevalence of Vaccine-Type HPV and Between-Wave Changes by Vaccination Status Adjusted for Propensity Score

All Vaccinated Unvaccinated 4-valent vaccine All Vaccinated Unvaccinated 9-valent but not 4-valent vaccine All Vaccinated Unvaccinated 9-valent vaccine Vaccination Status Prevalence of HPV Types Across Waves, Propensity Score Types in the Types in the

Types in the P < .05. -33, -45, -52, and -58. HPV-31, HPV-6, -11, -16, -18, -31, -33, -45, -52, and -58. HPV-6, HPV-6, -11, -16, and -18. HPV-6, TABLE 2 CI, confidence interval. a b c d

Downloaded from www.aappublications.org/news by guest on February 5, 2019 4 SPINNER et al χ participant characteristics using ’ 2 univariable methods (eg, test, Fisher s exact test, Kruskal-Wallis test, or analysis of variance). Because there were between-wave differences for some variables, we conducted a propensity score analysis adjusted by inverse probability of treatment weighting. This adjusts for selection bias, ensuring that any differences noted in HPV prevalence across the study waves were due to HPV vaccine introduction instead of differences in measured participant factors, which FIGURE 1 may have been confounders. These Proportions of women who were vaccinated across the 4 study waves infected with any vaccine- type HPV, adjusted for propensity scores. HPV 9m4v, 5 additional HPV types only in 9-valent vaccine factors included sociodemographic (HPV-31, -33, -45, -52, and -58); 4v HPV, 4-valent vaccine–type human papillomavirus; 9v HPV, 9-valent characteristics, gynecologic history, vaccine–type human papillomavirus. sexual history, and enrollment site (see Table 1 for all factors included). We then determined HPV prevalence for all women in the study, women who were vaccinated, and women who were unvaccinated across the 4 study waves; these results were unadjusted and adjusted for the propensity score.

Methodologic details15, are22​ described in previous articles. ‍ Total vaccine effectiveness (the relative infection risk in individuals who were vaccinated versus the infection risk in individuals who were unvaccinated before the introduction of a vaccine) was assessed by comparing vaccine- type HPV prevalence in women who were vaccinated in waves 2, 3, and 4

FIGURE 2 with women in wave28 1, all of whom Proportions of women who were unvaccinated across the 4 study waves infected with any vaccine- were unvaccinated. type HPV, adjusted for propensity scores. HPV 9m4v, 5 additional HPV types only in 9-valent vaccine (HPV-31, -33, -45, -52, and -58); 4v HPV, 4-valent vaccine–type human papillomavirus; 9v HPV, 9-valent vaccine–type human papillomavirus. Finally, we conducted logistic regression analyses to determine the odds of HPV prevalence across the study waves, unadjusted and that the results of self- and – participants who were vaccinated. adjusted for the propensity score. clinician testing for vaginal HPV24 in26 Vaccination status was determined The independent variables were women are highly concordant. ‍‍ by a positive record in either or both study waves (eg, wave 4 versus wave Vaccination status was defined databases and, in the small number 1), and the dependent variables were as having received at least 1 HPV of cases in which no information was prevalence of at least 1 of the HPV vaccine dose before enrollment, and available, by self-report. types included in the 9-valent vaccine it was assessed by using the Ohio (HPV-6, -11, -16, -18, -31, -33, -45, statewide immunization registry We first examined participant -52, and -58), at least 1 of the HPV and the electronic health record, 27 characteristics and behaviors using types included in the 4-valent vaccine both systems with high reliability. descriptive statistical methods. (HPV-6, -11, -16, and -18), or at least Documentation in at least 1 of these We then determined if there were 1 of the 5 HPV types in the 9-valent systems was available for 98% of the differences by study wave in but not the 4-valent vaccine (-31, -33, Downloaded from www.aappublications.org/news by guest on February 5, 2019 PEDIATRICS Volume 143, number 2, February 2019 5 ≥ – proportion infected with 1 4-valent vaccine type HPV decreased from ≥ 35.0% to 6.7% (80.9% decline), and the proportion infected with 1 of the 5 types in the 9-valent but not the 4-valent vaccine decreased from 23.4% to 7.3% (68.8% decline); all were statistically significant. Among women who were unvaccinated, ≥ – the proportion of women infected with 1 9-valent vaccine type HPV decreased from 43.4% to 42.1% (3.0% decline; nonsignificant) from ≥ – waves 1 to 4, the proportion infected with 1 4-valent vaccine type HPV decreased from 32.4% to 19.4% ≥ FIGURE 3 (40.1% decline; significant), and Proportions of all women across the 4 study waves infected with any vaccine-type HPV, adjusted for the proportion infected with 1 propensity scores. HPV 9m4v, 5 additional HPV types only in 9-valent vaccine (HPV-31, -33, -45, -52, and -58); 4v HPV, 4-valent vaccine–type human papillomavirus; 9v HPV, 9-valent vaccine–type human of the 5 types in the 9-valent but papillomavirus. not the 4-valent vaccine increased from 22.9% to 36.1% (57.6% increase; nonsignificant). Vaccine effectiveness was 71.7% in wave -45, -52, and -58). Separate models wave 1 to 59.2% in wave 2, 71.5% 2 versus wave 1, 90.6% in wave 3 were estimated for all women in the in wave 3, and 84.3% in wave 4. versus wave 1, and 80.1% in wave 4 study, those who were unvaccinated, Virtually all participants in waves 1 versus wave 1. and those who were vaccinated. Data to 3 received the 4-valent vaccine; in were analyzed by using SAS software wave 4, 88% received the 4-valent The logistic regression model results version 9.4 (SAS Institute, Inc, Cary, and 12% the 9-valent vaccine. NC). Overall, 97% of participants received are shown in Table 3. In these models, we demonstrate the odds RESULTS the 4-valent vaccine. – of infection (in waves 4 vs 1, 4 vs 2, – A propensity score analysis was and 4 vs 3) with 9-valent vaccine performed to adjust for between- type HPV, 4-valent vaccine type A total of 1580 participants were Pwave differences in participant HPV, and the 5 additional types in enrolled across the 4 studies. characteristics (Table 1). The the 9-valent vaccine by vaccination Participant characteristics are shown values after propensity score status. Among women who were in Table 1. The range of mean ages adjustment were all >.05, and the vaccinated, the odds of infection across all waves was 18.7 to 19.1 – standardized differences for most of decreased significantly from wave 1 years. Between 62.5% and 76.8% the variables were <20%, indicating to wave 4 for 9-valent vaccine type of participants were recruited from – that variables were successfully HPV (adjusted odds ratio [aOR] the Teen Health Center and 23.3% balanced. 0.18), 4-valent vaccine type HPV to 37.5% were recruited from the – (aOR 0.13), and the 5 additional HPV health department. The majority The proportions of women across of participants (69.9% 77.3%) the 4 study waves who were positive types included only in the 9-valent identified as African American or for 9-valent HPV types, 4-valent HPV vaccine (aOR 0.26). Among women multiracial, 48.9% to 56.8% reported types, and the 5 HPV types included who were unvaccinated, the odds of infection did not change significantly at least 2 male lifetime sexual in the 9-valent but not the 4-valent – partners, 26.3% to 38.5% used vaccine are shown adjusted for from wave 1 to wave 4 for 9-valent – vaccine type HPV; it decreased condoms during their last sexual propensity score in Table 2 and in – intercourse, and 16.9% to 31.8% ‍Figs 1 3. Among women who were significantly from wave 1 to wave 4 ≥ – had smoked at least 100 cigarettes vaccinated, the proportion infected for 4-valent vaccine type HPV (aOR in their lifetime. The rate of HPV with 1 9-valent vaccine type HPV 0.50) and increased significantly for vaccination (defined as receipt of at decreased from waves 1 to 4 from the 5 additional HPV types only in the least 1 dose) increased from 0% in 46.6% to 13.5% (71% decline), the 9-valent vaccine (aOR 1.90). Downloaded from www.aappublications.org/news by guest on February 5, 2019 6 SPINNER et al DISCUSSION b

To our knowledge, this is the first – study in which trends in 4-valent 0.74 (0.52 to 1.05) and 9-valent vaccine type HPV among women who are vaccinated and women who are unvaccinated Wave 3 – 4 b in the United States >10 years after HPV vaccine introduction are examined, and it is the first in which trends after 9-valent HPV 0.77 (0.50 to 1.19)0.66 (0.38 to 1.16) 0.77 (0.50 to 1.21) 1.06 (0.64 to 1.75) 0.69 (0.46 to 1.04)0.68 (0.42 to 1.10) 0.68 (0.46 to 1.02) 0.48 (0.28 to 0.82) 0.66 (0.46 to 0.94) vaccine introduction are examined. Consistent with our hypotheses, we – found that from 2006 to 2017, the –

b b b b b prevalence of 4-valent vaccine type HPV and 9-valent vaccine type HPV decreased significantly among – women who were vaccinated, and the

aOR (95% CI) Unadjusted OR (95% CI) aOR (95% CI) prevalence of 4-valent vaccine type 1.17 (0.56 to 2.46) 0.55 (0.22 to 1.39)1.79 (0.96 to 3.33) 0.84 (0.39 to 1.79) 0.93 (0.43 to 2.02) 1.30 (0.68 to 2.48) 1.35 (0.75 to 2.43) 0.68 (0.35 to 1.35) 0.87 (0.47 to 1.61) 0.66 (0.36 to 1.20) 1.15 (0.52 to 2.54) 2.13 (0.97 to 4.69) 0.42 (0.30 to 0.58) 0.32 (0.21 to 0.49) 0.61 (0.39 to 0.95) 0.42 (0.29 to 0.60) 0.22 (0.13 to 0.36) HPV decreased significantly among women who were unvaccinated. In exploratory analyses, we Wave 2 – 4

b b b b b b demonstrated a significant decrease in the 5 additional types targeted by the 9-valent vaccine among women who were vaccinated and, unexpectedly, a significant increase 0.55 (0.23 to 1.32) 0.77 (0.37 to 1.59) 0.66 (0.35 to 1.25) 0.35 (0.25 to 0.49) 0.30 (0.20 to 0.46) 0.36 (0.22 to 0.60) 0.39 (0.20 to 0.75) 0.38 (0.26 to 0.55) 0.30 (0.19 to 0.47) in the 5 additional types targeted by the 9-valent vaccine among women who were unvaccinated.

b b b b b b b b – The significant decline (81%) in 4-valent vaccine type HPV in women who were vaccinated and the high degree of vaccine effectiveness 0.95 (0.56 to 1.62) 0.25 (0.18 to 0.35) 0.18 (0.12 to 0.26) 0.18 (0.12 to 0.27) 0.13 (0.08 to 0.22) 0.50 (0.26 to 0.97) 0.44 (0.30 to 0.64) 0.26 (0.16 to 0.42) when comparing women who were vaccinated with women who were unvaccinated (90.6% in wave 3 Wave 1 – 4 b b b b b b b b versus wave 1 and 80.1% in wave 4 versus wave 1) provide evidence of direct protection by the 4-valent vaccine among the women who were 0.76 (0.39 to 1.49) 1.90 (1.09 to 3.34) 0.26 (0.12 to 0.60) 0.24 (0.17 to 0.34) 0.12 (0.08 to 0.20) 0.43 (0.29 to 0.63) 0.46 (0.26 to 0.84) 0.10 (0.06 to 0.17) 0.37 (0.24 to 0.57) 0.20 (0.14 to 0.29) vaccinated in this community, and it

Unadjusted OR (95% CI) aOR (95% CI) Unadjusted OR (95% CI) suggests high vaccine effectiveness tions of Women Who Are Positive for 9-Valent HPV Types, 4-Valent and the 5 Types Included in But N ot Vaccine Across 4 Study in a real-world setting. This degree of effectiveness is remarkable given ≥ the fact that vaccination was defined c a

d as having received 1 dose (ie, was not defined as having completed the vaccination series) and that women in this study were likely  Comparisons of the Propor Waves by Vaccination Status: Results of Unadjusted and Adjusted Logistic Regression Analyses at a substantially higher risk for the 4-valent vaccine Unvaccinated All All All Unvaccinated Unvaccinated Vaccinated Vaccinated Vaccinated preexisting HPV infection than those 5 HPV types in the 9-valent but not 4-valent vaccine – type HPV

Vaccination Status 9-valent vaccine – type HPV

P < .05. -33, -45, -52, and -58. HPV-31, HPV-6, -11, -16, -18, -31, -33, -45, -52, and -58. HPV-6, HPV-6, -11, -16, and -18. HPV-6, CI, confidence interval; OR, odds ratio. a b c d TABLE 3 in the HPV vaccine clinical trials because of their reported sexual behaviors. The high efficacy of the Downloaded from www.aappublications.org/news by guest on February 5, 2019 PEDIATRICS Volume 143, number 2, February 2019 7 – licensed HPV vaccines and high to the 2-valent33 35 or 4-valent HPV cost-effectiveness analysis evaluating rates of vaccination in this study vaccines ‍ ‍ ; however, it is not well cervical cancer screening strategies. sample likely contributed to this established whether crossprotection substantial decrease in HPV infection, occurs among younger women We unexpectedly found a significant even in young women at a high risk recruited from broader settings or increase from 2006 to 2017 in for sexually transmitted infections after introduction of the 9-valent the prevalence of the 5 additional (STIs). Our findings of a decrease in HPV vaccine. In this study, only 12% types in the 9-valent vaccine among vaccine-type HPV among women who of women in wave 4 received at women who were unvaccinated. were vaccinated extend the findings A theoretical explanation could be least 1 dose of the 9-valent vaccine; – of those studies conducted over type replacement, which would – therefore, it is unlikely that these – different time frames and16 in19, different21,​ 29​ findings were due only to direct be an increase in non 4-valent populations and settings,​ ‍ ‍ ‍ protection against the types in the vaccine type HPV created by an and they support the real-world 9-valent vaccine among women ecological niche after the 4-valent – effectiveness of the 4-valent vaccine, who were vaccinated. Instead, the vaccine was introduced. However, especially in younger age groups decreases noted in 9-valent vaccine this phenomenon is thought to be and countries16,21​ with high vaccination type HPV and the 5 additional types unlikely given the genetic stability of coverage. ‍ This decline in HPV in the 9-valent vaccine may represent HPV and that HPV types do not seem prevalence is expected to result in evidence of crossprotection against to compete38 for trophic epithelial a significant decrease in cervical genetically related types (-31, -33, niches. A more likely explanation precancers and cancers in the future. -45, -52, and -58). In a separate is differences between women Of note, the findings should not analysis, we are examining trends who are unvaccinated and women be interpreted as suggesting that in nonvaccine-type HPV in more who are vaccinated. For example, if current recommendations for a 2- or depth by exploring the prevalence women who are unvaccinated versus 3-dose series and vaccinating before of nonvaccine-type HPV genetically women who are vaccinated are more sexual initiation are not necessary. In related to HPV-16 and HPV-18 likely to practice riskier behaviors additional exploratory analyses, we that would increase their risk of among women who received the – have demonstrated that receipt of 1 acquiring HPV, they would be more 4-valent vaccine to examine for vs 3 doses was associated with 3.2 – likely to acquire non vaccine-type P crossprotection and by exploring times the adjusted odds of vaccine- HPV. In a previous analysis of the non vaccine-type HPV genetically type HPV infection ( = .04) and that first 3 waves of data from this study, unrelated to vaccine-type HPV young women who did versus did we found that women who were to examine for type replacement not have sex before vaccination were unvaccinated did differ significantly P (C.S., L.D., D.B., et al, unpublished more likely to be positive for vaccine- from women who were vaccinated observations). type HPV (8.8% vs 3.8%, = .0021). in ways that could increase their risk for HPV acquisition, supporting this There was a significant decline of 39 We also demonstrated a significant – explanation. For example, women decline among women who were 40.1% in the prevalence of 4-valent who were unvaccinated versus – vaccinated in the prevalence of vaccine type HPV among women women who were vaccinated were 9-valent vaccine type HPV (71%), as who were unvaccinated, suggesting more likely to lack health insurance hypothesized, and in the prevalence herd protection. Evidence of herd and to have had at least 1 new of the 5 additional HPV types protection was found in our previous sexual partner in the past 3 months. included in the 9-valent vaccine study during15 the first 3 waves of data Continued community-level research (68.8%). All 5 additional types in collection. In addition, evidence is needed to determine if this trend the 9-valent vaccine are genetically of herd protection was found in reverses and if the prevalence of the related to HPV-16 (HPV-31, -33, 2 studies conducted in Scotland 5 additional types included in the -52, and -58) and HPV-18 (HPV- among adult women attending33,35​ 9-valent vaccine begin to decline 45). In clinical trials, researchers cervical cancer screening,​ ‍ in once a higher proportion of young have demonstrated evidence of studies in Australia among adult women have received the 9-valent crossprotection– against HPV types women attending19 cervical cancer vaccine. In addition, the findings genetically14,30​ 32 related to HPV-16 and screening and recruited18 through a of differences between women -18,​ ‍ ‍‍ and in studies conducted social networking site,​ and in the who are unvaccinated and women in clinical and community settings, United States29, before36​ 9-valent vaccine who are vaccinated underscore researchers have also demonstrated introduction. 37‍ As noted in a the importance of understanding evidence of crossprotection against recent review,​ evidence about herd predictors of nonvaccination and HPV types genetically related protection will be a key component of designing clinical interventions Downloaded from www.aappublications.org/news by guest on February 5, 2019 8 SPINNER et al CONCLUSIONS to reach those youth who are and the impact of 9-valent vaccine unvaccinated and who may be at an Eleven years after the introduction introduction. elevated risk for HPV. of the HPV vaccine, we noted ACKNOWLEDGMENTS – – A limitation of this study was the significant decreases in 4-valent small proportion of women who vaccine type HPV, 9-valent vaccine received the 9-valent vaccine in type HPV, and the 5 additional HPV We acknowledge the support wave 4, which limited our ability types included only in 9-valent provided by Charlene Morrow, RN, to examine the trends in HPV vaccine among women who were Susan Glynn, BA, Rachel Thomas, prevalence due to the effects of the vaccinated, suggesting 4-valent BS, and Lisa Higgins, RPh, as well ’ 9-valent vaccine. Also, the clinic- vaccine effectiveness in a real-world as the support of the Cincinnati based recruitment strategy could setting and possible crossprotection Children s Hospital Teen Health limit generalizability to all young against genetically related HPV types. Center and Cincinnati Health women in this age group in the The significant decrease in 4-valent Department staff. United States. Instead, the sample HPV types among women who ABBREVIATIONS could be viewed as generalizable were unvaccinated suggests herd to a group of adolescent and young protection. Although these findings adult women at a relatively high risk are important for clinical care and aOR: adjusted odds ratio for HPV and other STIs and with a public health policy, continued HPV: human papillomavirus fairly high vaccination rate. Finally, surveillance will be important STI: sexually transmitted risk behaviors were assessed by self- to assess for waning vaccine infection report, which may limit validity. effectiveness, herd protection, manuscript, and revised the final manuscript; and all authors approved the final manuscript as submitted and agree to be accountable for all aspects of the work. Ms Spinner was an undergraduate at the University of Cincinnati when this research was conducted; she is currently an entering graduate student at the University of South Florida. DOI: https://​doi.​org/​10.​1542/​peds.​2018-​1902 Accepted for publication Oct 3, 2018 Address correspondence to Jessica A. Kahn, MD, MPH, Cincinnati Children’s Hospital Medical Center, 3333 Burnet Ave, Cincinnati, OH 45229. E-mail: jessica.kahn@ cchmc.org PEDIATRICS (ISSN Numbers: Print, 0031-4005; Online, 1098-4275). Copyright © 2019 by the American Academy of Pediatrics FINANCIAL DISCLOSURE: Dr Brown owns stock in shares of Merck & Co, Inc; the other authors have indicated they have no financial relationships relevant to this article to disclose. FUNDING: Supported by 2 grants from the National Institutes of Health National Institute of Allergy and Infectious Diseases (R01 AI073713 and R01 AI104709) and a grant from the National Institutes of Health National Center for Advancing Translational Sciences (UL1 TR001425). Funded by the National Institutes of Health (NIH). POTENTIAL CONFLICT OF INTEREST: Dr Brown has received an investigator initiated studies program award from Merck entitled “Cervical Cancer Prevention in Kenya” and owns stock in shares of Merck & Co, Inc. Dr Franco has occasionally served as consultant to companies involved with human papillomavirus (HPV) diagnostics (Roche, BD, and Abbott) and HPV vaccines (Merck and GSK). Dr Kahn served as cochair of a study of HPV vaccines in men with HIV. The study was funded by the National Institutes of Health, but Merck provided vaccine and serology testing; the other authors have indicated they have no potential conflicts of interest to disclose.

REFERENCES 1. Bosch FX, de Sanjosé S. Chapter 1: human 4. Markowitz LE, Dunne EF, Saraiya M, Prevention. Human papillomavirus papillomavirus and cervical cancer– Lawson HW, Chesson H, Unger ER; Centers vaccination: recommendations of the burden and assessment of causality. for Disease Control and Prevention; Advisory Committee on Immunization J Natl Cancer Inst Monogr. 2003;(31):3–13 Advisory Committee on Immunization Practices (ACIP) [published correction Practices. Quadrivalent human appears in MMWR Recomm Rep. 2. Bosch FX, Lorincz A, Muñoz N, Meijer CJ, Shah KV. The causal relation papillomavirus vaccine: recommendations 2014;63(49):1182]. MMWR Recomm Rep. between human papillomavirus of the Advisory Committee on 2014;63(RR-05):1–30 Immunization Practices (ACIP). MMWR and cervical cancer. J Clin Pathol. 6. Petrosky E, Bocchini JA Jr, Hariri S, Recomm Rep. 2007;56(RR-2):1–24 2002;55(4):244–265 et al; Centers for Disease Control 3. Palefsky JM. HPV infection in men. Dis 5. Markowitz LE, Dunne EF, Saraiya M, and Prevention (CDC). Use of 9-valent Markers. 2007;23(4):261–272 et al; Centers for Disease Control and human papillomavirus (HPV)

Downloaded from www.aappublications.org/news by guest on February 5, 2019 PEDIATRICS Volume 143, number 2, February 2019 9 vaccine: updated HPV vaccination of a double-blind, randomised vaccine introduction. Pediatrics. recommendations of the advisory study in young women [published 2012;130(2). Available at: www.​ committee on immunization practices. correction appears in Lancet. pediatrics.​org/​cgi/​content/​full/​130/​2/​ MMWR Morb Mortal Wkly Rep. 2010;376(9746):1054]. Lancet. e249 2015;64(11):300–304 2009;374(9686):301–314 23. Gravitt PE, Peyton CL, Alessi TQ, et al. 7. Joura EA, Giuliano AR, Iversen OE, 15. Kahn JA, Widdice LE, Ding L, et al. Improved amplification of genital et al; Broad Spectrum HPV Vaccine Substantial decline in vaccine-type human papillomaviruses. J Clin Study. A 9-valent HPV vaccine human papillomavirus (HPV) among Microbiol. 2000;38(1):357–361 against infection and intraepithelial vaccinated young women during 24. Kahn JA, Slap GB, Huang B, et al. neoplasia in women. N Engl J Med. the first 8 years after HPV vaccine Comparison of adolescent and young 2015;372(8):711–723 introduction in a community. Clin adult self-collected and clinician- Infect Dis. 2016;63(10):1281 1287 8. Paz-Zulueta M, Álvarez-Paredes L, – collected samples for human Rodríguez Díaz JC, et al. Prevalence of 16. Drolet M, Bénard É, Boily MC, et papillomavirus. Obstet Gynecol. high-risk HPV genotypes, categorised al. Population-level impact and 2004;103(5, pt 1):952–959 by their quadrivalent and nine-valent herd effects following human 25. Sellors JW, Lorincz AT, Mahony JB, HPV vaccination coverage, and the papillomavirus vaccination et al. Comparison of self-collected genotype association with high-grade programmes: a systematic review vaginal, vulvar and urine samples lesions. BMC Cancer. 2018;18(1):112 and meta-analysis. Lancet Infect Dis. with physician-collected cervical 9. Luckett R, Feldman S. Impact of 2-, 2015;15(5):565–580 samples for human papillomavirus 4- and 9-valent HPV vaccines on 17. Markowitz LE, Liu G, Hariri S, Steinau M, testing to detect high-grade squamous morbidity and mortality from cervical Dunne EF, Unger ER. Prevalence of HPV intraepithelial lesions. CMAJ. cancer. Hum Vaccin Immunother. after introduction of the vaccination 2000;163(5):513–518 2016;12(6):1332–1342 program in the United States. 26. Gravitt PE, Lacey JV Jr, Brinton LA, 10. Brisson M, Laprise JF, Chesson HW, Pediatrics. 2016;137(3):e20151968 et al. Evaluation of self-collected et al. Health and economic impact 18. Garland SM, Cornall AM, Brotherton cervicovaginal cell samples for human of switching from a 4-valent to a JML, Wark JD, Malloy MJ, Tabrizi SN; papillomavirus testing by polymerase 9-valent HPV vaccination program in VACCINE Study Group. Final analysis chain reaction. Cancer Epidemiol the United States. J Natl Cancer Inst. of a study assessing genital human Biomarkers Prev. 2001;10(2):95–100 2015;108(1):djv282 papillomavirus genoprevalence 27. Thomas R, Higgins L, Ding L, Widdice 11. Harper DM, Franco EL, Wheeler CM, in young Australian women, LE, Chandler E, Kahn JA. Factors et al; HPV Vaccine Study Group. Sustained following eight years of a national associated with HPV vaccine initiation, efficacy up to 4.5 years of a bivalent vaccination program. Vaccine. vaccine completion, and accuracy L1 virus-like particle vaccine against 2018;36(23):3221–3230 of self-reported vaccination status human papillomavirus types 16 and 18: 19. Machalek DA, Garland SM, Brotherton among 13- to 26-year-old men. follow-up from a randomised control JML, et al. Very low prevalence Am J Men Health. 2018;12(4):819–827 trial. Lancet. 2006;367(9518):1247–1255 of vaccine human papillomavirus 28. Shim E, Galvani AP. Distinguishing 12. Joura EA, Kjaer SK, Wheeler CM, et al. types among 18- to 35-year old vaccine efficacy and effectiveness. HPV antibody levels and clinical Australian women 9 years following Vaccine. 2012;30(47):6700–6705 implementation of vaccination. J Infect efficacy following administration of a 29. Oliver SE, Unger ER, Lewis R, et al. Dis. 2018;217(10):1590 1600 prophylactic quadrivalent HPV vaccine. – Prevalence of human papillomavirus Vaccine. 2008;26(52):6844–6851 20. Markowitz LE, Hariri S, Lin C, et al. among females after vaccine 13. Joura EA, Leodolter S, Hernandez-Avila Reduction in human papillomavirus introduction-National Health and M, et al. Efficacy of a quadrivalent (HPV) prevalence among young women Nutrition Examination Survey, United prophylactic human papillomavirus following HPV vaccine introduction in States, 2003-2014. J Infect Dis. (types 6, 11, 16, and 18) L1 virus- the United States, National Health and 2017;216(5):594–603 like-particle vaccine against high- Nutrition Examination Surveys, 2003- 30. Brown DR, Kjaer SK, Sigurdsson K, grade vulval and vaginal lesions: 2010. J Infect Dis. 2013;208(3):385–393 et al. The impact of quadrivalent a combined analysis of three 21. Mesher D, Panwar K, Thomas SL, human papillomavirus (HPV; types 6, randomised clinical trials. Lancet. Beddows S, Soldan K. Continuing 11, 16, and 18) L1 virus-like particle 2007;369(9574):1693–1702 reductions in HPV 16/18 in a vaccine on infection and disease due population with high coverage of to oncogenic nonvaccine HPV types in 14. Paavonen J, Naud P, Salmerón J, et al; HPV PATRICIA Study Group. bivalent HPV vaccination in England: generally HPV-naive women aged 16-26 Efficacy of human papillomavirus an ongoing cross-sectional study. BMJ years. J Infect Dis. 2009;199(7):926–935 Open. 2016;6(2):e009915 (HPV)-16/18 AS04-adjuvanted 31. Wheeler CM, Castellsagué X, Garland vaccine against cervical infection 22. Kahn JA, Brown DR, Ding L, et al. SM, et al; HPV PATRICIA Study Group. and precancer caused by oncogenic Vaccine-type human papillomavirus Cross-protective efficacy of HPV-16/18 HPV types (PATRICIA): final analysis and evidence of herd protection after AS04-adjuvanted vaccine against

Downloaded from www.aappublications.org/news by guest on February 5, 2019 10 SPINNER et al cervical infection and precancer 34. Tabrizi SN, Brotherton JM, Kaldor 37. Malagón T, Laurie C, Franco EL. Human caused by non-vaccine oncogenic HPV JM, et al. Assessment of herd papillomavirus vaccination and the types: 4-year end-of-study analysis of immunity and cross-protection after role of herd effects in future cancer the randomised, double-blind PATRICIA a human papillomavirus vaccination control planning: a review. Expert Rev trial [published correction appears in programme in Australia: a repeat Vaccines. 2018;17(5):395–409 Lancet Oncol. 2012;13(1):e1]. Lancet cross-sectional study. Lancet Infect 38. Tota JE, Ramanakumar AV, Jiang M, Oncol. 2012;13(1):100–110 Dis. 2014;14(10):958–966 et al. Epidemiologic approaches to 32. Malagón T, Drolet M, Boily MC, et al. 35. Cameron RL, Kavanagh K, Pan J, et al. evaluating the potential for human Cross-protective efficacy of two human Human papillomavirus prevalence papillomavirus type replacement papillomavirus vaccines: a systematic and herd immunity after introduction postvaccination. Am J Epidemiol. review and meta-analysis. Lancet Infect of vaccination program, Scotland, 2013;178(4):625–634 Dis. 2012;12(10):781–789 2009-2013. Emerg Infect Dis. 39. Ding L, Widdice LE, Kahn JA. 2016;22(1):56 64 33. Kavanagh K, Pollock KG, Potts A, et al. – Differences between vaccinated Introduction and sustained high 36. Berenson AB, Hirth JM, Chang M. and unvaccinated women explain coverage of the HPV bivalent vaccine Change in human papillomavirus increase in non-vaccine-type human leads to a reduction in prevalence of prevalence among U.S. women aged papillomavirus in unvaccinated women HPV 16/18 and closely related HPV types. 18-59 years, 2009-2014. Obstet Gynecol. after vaccine introduction. Vaccine. Br J Cancer. 2014;110(11):2804–2811 2017;130(4):693–701 2017;35(52):7217–7221

Downloaded from www.aappublications.org/news by guest on February 5, 2019 PEDIATRICS Volume 143, number 2, February 2019 11 Human Papillomavirus Vaccine Effectiveness and Herd Protection in Young Women Chelse Spinner, Lili Ding, David I. Bernstein, Darron R. Brown, Eduardo L. Franco, Courtney Covert and Jessica A. Kahn Pediatrics 2019;143; DOI: 10.1542/peds.2018-1902 originally published online January 22, 2019;

Updated Information & including high resolution figures, can be found at: Services http://pediatrics.aappublications.org/content/143/2/e20181902 References This article cites 39 articles, 7 of which you can access for free at: http://pediatrics.aappublications.org/content/143/2/e20181902#BIBL Subspecialty Collections This article, along with others on similar topics, appears in the following collection(s): Infectious Disease http://www.aappublications.org/cgi/collection/infectious_diseases_su b Vaccine/Immunization http://www.aappublications.org/cgi/collection/vaccine:immunization _sub Sexually Transmitted Infections http://www.aappublications.org/cgi/collection/sexually_transmitted_i nfections_new_sub Permissions & Licensing Information about reproducing this article in parts (figures, tables) or in its entirety can be found online at: http://www.aappublications.org/site/misc/Permissions.xhtml Reprints Information about ordering reprints can be found online: http://www.aappublications.org/site/misc/reprints.xhtml

Downloaded from www.aappublications.org/news by guest on February 5, 2019 Human Papillomavirus Vaccine Effectiveness and Herd Protection in Young Women Chelse Spinner, Lili Ding, David I. Bernstein, Darron R. Brown, Eduardo L. Franco, Courtney Covert and Jessica A. Kahn Pediatrics 2019;143; DOI: 10.1542/peds.2018-1902 originally published online January 22, 2019;

The online version of this article, along with updated information and services, is located on the World Wide Web at: http://pediatrics.aappublications.org/content/143/2/e20181902

Pediatrics is the official journal of the American Academy of Pediatrics. A monthly publication, it has been published continuously since 1948. Pediatrics is owned, published, and trademarked by the American Academy of Pediatrics, 141 Northwest Point Boulevard, Elk Grove Village, Illinois, 60007. Copyright © 2019 by the American Academy of Pediatrics. All rights reserved. Print ISSN: 1073-0397.

Downloaded from www.aappublications.org/news by guest on February 5, 2019 Clinical Infectious Diseases MAJOR ARTICLE

Declines in Human Papillomavirus (HPV)–Associated High-Grade Cervical Lesions After Introduction of HPV Vaccines in Connecticut, United States, 2008–2015 Linda M. Niccolai,1,2 James I. Meek,1,2 Monica Brackney,1,2 James L. Hadler,1,2 Lynn E. Sosa,3 and Daniel M. Weinberger2 Downloaded from https://academic.oup.com/cid/article-abstract/65/6/884/3829989 by Yale University Law School user on 12 March 2019 1Connecticut Emerging Infections Program at Yale and 2Department of Epidemiology of Microbial Diseases, Yale School of Public Health, New Haven, and 3Connecticut Department of Public Health, Hartford

(See the Editorial Commentary by Garland and Machalek on pages 890–2.) Background. Trends in human papillomavirus (HPV)–associated cervical lesions can provide an indication of vaccine impact. Our purpose was to measure trends in cervical lesions during 2008–2015 and to consider possible explanations including vaccina- tion coverage, changes in screening for cervical cancer, and risk behaviors for acquiring HPV. Methods. Connecticut (CT) implemented mandatory reporting of cervical intraepithelial neoplasia grades 2/3 and adenocar- cinoma in situ (cervical intraepithelial neoplasia grade 2 or higher [CIN2+]) in 2008. Trends by age and birth cohort were modeled using negative binomial regression and change-point methods. To evaluate possible explanations for changes, these trends were compared to changes in HPV vaccination coverage, cervical cancer screening, an antecedent event to detection of a high-grade lesion, and changes in sexual behaviors and Chlamydia trachomatis, an infection with similar epidemiology to and shared risk factors for HPV. Results. A significant decline in CIN2+ was first evident among women aged 21 years in 2010, followed by successive declines in women aged 22–26 years during 2011–2012. During 2008–2015, the rates of CIN2+ declined by 30%–74% among women aged 21–26 years, with greater declines observed in the younger women. Birth cohorts between 1985 and 1994 all experienced significant declines during the surveillance period, ranging from 25% to 82%. Ecological comparisons revealed substantial increases in HPV vaccination during this time period, and more modest reductions in cervical cancer screening and sexual risk behaviors. Conclusions. The age and cohort patterns in our data suggest that declines in CIN2+ during 2008–2015 are more likely driven by HPV vaccination, introduced in 2006, than by changes in screening or risk behavior. Keywords. human papillomavirus; cervical lesions; vaccination; vaccine impact; surveillance.

Postlicensure monitoring of human papillomavirus (HPV) vac- they are most closely related to the development of cervical can- cines is important to demonstrate the population-level impact cer and because they are diagnoses that incur substantial mor- on clinical outcomes, to provide data to inform policy and prac- bidity, psychological distress, and treatment costs [5, 6]. tice, and to measure progress of immunization programs [1]. In countries that have national or regional healthcare and dis- Using a variety of data sources and approaches, studies from ease registries (eg, Australia and Denmark), data linkage studies Australia, Europe, North America, and New Zealand have are able to examine rates of HPV-associated outcomes in relation been largely consistent in showing significant declines in HPV- to individual vaccination status to assess direct effectiveness of vac- related outcomes among young women, particularly in settings cines [7–11]. Unfortunately, registries for these types of studies are that have achieved high coverage with HPV vaccines [2–4]. not widely available in many settings, including the United States. Though evidence of the impact of HPV vaccines on cervical In such areas, evidence of vaccine impact may be assessed by infections and genital warts is growing, less is known about examining population-level trends in HPV-associated outcomes trends in precancerous high-grade cervical lesions [2]. Cervical over time. A strength of this approach is that these trends reflect lesions are clinically important outcomes to monitor because both direct and indirect effects of vaccination [12]. However, a challenge with this type of trend analysis is that declines due to Received 11 January 2017; editorial decision 24 March 2017; accepted 12 May 2017; published the vaccine may not be easily disentangled from declines that online May 17, 2017. Correspondence: L. M. Niccolai, Yale School of Public Health, 60 College St, New Haven, CT might occur for other reasons. For example, in the United States, 06520 ([email protected]). recent changes to cervical cancer screening guidelines that now Clinical Infectious Diseases® 2017;65(6):884–9 recommend less frequent screening may also contribute to fewer © The Author 2017. Published by Oxford University Press for the Infectious Diseases Society of America. All rights reserved. For permissions, e-mail: [email protected]. diagnoses of cervical lesions [13]. First put forth in 2009 and DOI: 10.1093/cid/cix455 more widely adopted in 2012, guidelines changed to recommend

884 • CID 2017:65 (15 September) • Niccolai et al no screening prior to age 21 years and to lengthen the screen- for human subjects approval. Population denominators esti- ing interval to every 3 years (instead of annually). Furthermore, mated for each year were obtained from the US Census Bureau because HPV is a sexually transmitted infection, declines may also American Community Survey. We restricted these analyses to be due to underlying changes in sexual behaviors. women diagnosed at ages 21–39 years. Age 21 years was used We conducted the present analyses to describe trends in cer- for the lower limit because of changes to screening guidelines vical lesions in Connecticut (CT) during 2008–2015 by single during the surveillance period that no longer recommend cer- year of age and birth cohort. In CT, as in the rest of the United vical cancer screening prior to that age. States, HPV vaccination was introduced in 2006 for routine use We examined changes in rates of CIN2+ by single year of in females aged 11–12 years and catch-up vaccination to age age because each subsequent cohort of women will have had a 26 years. Administration of HPV vaccine is practice-based and greater opportunity to be vaccinated at younger ages and prior Downloaded from https://academic.oup.com/cid/article-abstract/65/6/884/3829989 by Yale University Law School user on 12 March 2019 typically provided in medical offices. All vaccines in use during to natural infections, thus producing declines in the rate of the study period covered HPV types 16 and 18, associated with CIN2+ in consecutive years. To test whether the year in which approximately 70% of cervical cancers and approximately 50% declines might begin could vary by age, we fit change-point of high-grade cervical lesions. To disentangle changes associ- models to estimate both the timing of when a decline begins and ated with vaccination from other factors that could influence the rate of decline after the change-point 17[ ]. For these change- reported rates, we also evaluated changes in HPV vaccina- point analyses, we used negative binomial regression models, tion coverage; rates of Papanicolaou (Pap) testing, a necessary where the outcome was the rate of CIN2+ for an age group (sin- screening test for the diagnosis of cervical intraepithelial neo- gle years of age). These models were fit separately for each age plasia grade 2 or higher (CIN2+); and changes in the sexual group. Eight alternative regression models were fit, one for each behaviors and the rate of infection with Chlamydia trachomatis, year in which the declines could begin: an intercept-only model a common sexually transmitted infection that is a marker of (no trend), and models where the rates were steady and then sexual risk. This analysis makes use of a unique data set from began declining (log) linearly in each year during the period the state of CT (population 3.6 million) that enacted mandatory 2009–2015. The Akaike information criterion scores were used reporting of CIN grades 2, 2/3, and 3 and adenocarcinoma in to generate a weight for each of these alternative models (bet- situ (CIN2+) in 2008. These data add to our knowledge about ter-fitting models receive a higher weight), and the predicted the population-level impact of HPV vaccines from countries values and parameter estimates were then averaged together. with high coverage of >70% for all 3 doses among adolescent Negative binomial regression models were also used to ana- females by showing trends in a geographic area with moderate lyze changes in rates of CIN2+ by birth cohorts. We included HPV vaccine coverage of approximately 50% [14]. Furthermore, dummy variables for the birth cohorts and controlled for age in by testing whether the trends in CIN2+ vary by individual years years using a cubic spline. The exponentiated regression coeffi- of age and by birth cohorts and by examining different possi- cient for the dummy variables gave the relative decline in cases ble explanations for trends (likelihood of detection, underlying among the indicated birth cohort compared with the referent risk), we may better understand the potential role of HPV vac- birth cohort of 1979. Women in this birth cohort were 27 years cine impact on cervical lesions in the United States. old in 2006 when vaccines were first available in the United States and are not likely to have experienced declines attributa- METHODS ble to vaccination because they were not in the recommended Surveillance methods have been previously described [15, age group postlicensure. 16]. In brief, in 2008, the Centers for Disease Control and To explore possible factors that could explain changes in Prevention began to monitor the impact of HPV vaccination CIN2+, we gathered data from other sources for ecological com- through population-based surveillance of CIN2+ conducted by parisons. First, we considered HPV vaccination coverage from the Emerging Infections Program network. To facilitate imple- 2 sources. State-specific estimates of coverage were available for mentation in CT, the state’s Department of Public Health added adolescents aged 13–17 years from the National Immunization CIN2+ to the list of mandatory reportable diseases, effective 1 Survey–Teen, a population-based telephone survey that verifies January 2008. All 34 pathology laboratories that serve CT res- vaccination histories with review of medical provider records idents are in compliance with the reporting requirement. All (https://www.cdc.gov/vaccines/imz-managers/nis/about.html). laboratories are regularly contacted to ensure ongoing, com- National estimates of coverage for women aged 19–26 years were plete, and timely reporting, and quality assurance protocols available from the National Health Interview Survey, a national include logic and range checks and double data entry for a sub- household in-person survey of the noninstitutionalized US civil- set of cases. Pathology laboratories that report >80% of cases ian population (https://www.cdc.gov/nchs/nhis/). Both of these are routinely audited for completeness. This work has been sources were used to cover the full age range for which HPV deemed public health surveillance by university, state, and fed- vaccination has been recommended. Second, because CIN2+ is eral institutional review boards and thus exempt from the need only detected after an abnormal cervical cancer screening test

Declines in HPV-Associated Lesions • CID 2017:65 (15 September) • 885 such as a Pap test, trends in CIN2+ may also be influenced by Table 1. Characteristics of Women Aged 21–39 Years Reported to the Connecticut Surveillance System for High-Grade Cervical Lesions During recent changes to guidelines that now recommend less frequent 2008–2015 (N = 15 218) screening (every 3 years instead of annually). Therefore, we esti- mated the percentage of CT women who reported a Pap test in Characteristic No. (%) the past year from data available through the Behavioral Risk Age, y Factor Surveillance System (BRFSS; https://www.cdc.gov/brfss/). 21–24 4291 (28) BRFSS is a federally funded, population-based national survey 25–29 5150 (34) that collects state-level data about a variety of health status indi- 30–34 3658 (24) cators. Data were analyzed for the following age groups to cor- 35–39 2119 (14) Diagnosis respond to rates we estimated for CIN2+: 21–24, 25–29, 30–34, CIN grade 2 9372 (62) Downloaded from https://academic.oup.com/cid/article-abstract/65/6/884/3829989 by Yale University Law School user on 12 March 2019 and 35–39 years during 2008–2014 (the last year for which these CIN grade 2/3 2269 (15) data are currently available). Finally, we considered changes in CIN grade 3 3398 (22) sexual behaviors, a risk factor for acquiring HPV. State-level data AIS (with or without CIN) 177 (1) on sexual behaviors are readily available from the Youth Risk Missing 2 (<1) Year of diagnosis Behavior Surveillance System, a school-based survey of repre- 2008 2163 (14) sentative samples of students in grades 9–12 (https://www.cdc. 2009 2195 (14) gov/healthyyouth/data/yrbs/). We included measures of ever 2010 2037 (13) having had sexual intercourse, having had sex with ≥4 persons, 2011 2044 (13) and not using a condom at last sexual intercourse. We also col- 2012 1818 (12) lected data about C. trachomatis diagnoses that share epidemi- 2013 1726 (11) 2014 1695 (11) ologic features with HPV (eg, sexually transmitted, common, 2015 1540 (10) and disproportionately affecting young women) because these Abbreviations: AIS, adenocarcinoma in situ; CIN, cervical intraepithelial neoplasia. estimates are readily available for comparable age groups includ- ing young adult women. Chlamydia trachomatis is a reportable infection, and the number of cases among women in CT by year total number of cases reported during this surveillance period and age were obtained from CT Department of Public Health. declined from 2163 in 2008 to 1540 in 2015. Data were available for the following standard age groups: 20–24, Significant declines in CIN2+ after 2008 were observed 25–29, 30–34, and 35–39 years. Annual counts were available for among women between the ages of 21 and 26 years, whereas 2008 through 2015 and rates estimated by dividing the number no significant declines were observed among women aged of cases by the estimated annual population size obtained from ≥27 years (Figure 1A and 1B; Supplementary Table 1). The tim- the US Census Bureau American Community Survey. For all ing of when declines began varied by single year of age, with measures, we calculated the average annual percentage change the earliest significant decline beginning among 21-year-olds during the study period years for which data were available. in 2010 and declines among women aged 22–26 years in 2011 and 2012. Comparing estimated rates in 2015 with the period RESULTS prior to the change-point in each age group, we observed sta- A total of 15 218 cases of CIN2+ were reported during 2008– tistically significant declines of 74% (95% confidence interval 2015 among residents of CT aged 21–39 years (Table 1). The [CI], 67%–79%) among women aged 21 years, 73% (95% CI,

Figure 1. Trends in observed (A) and predicted (B) incidence rates (per 100 000) of high-grade cervical lesions by age in years, Connecticut, 2008–2015. Figures restricted to ages 21–29 years for ease of viewing because no declines were observed among women aged 30–39 years. B, Predicted rates obtained from the negative binomial regression model presented for increased visualization of trend patterns.

886 • CID 2017:65 (15 September) • Niccolai et al 51%–85%) among women aged 22 years, 62% (95% CI, 51%– approaches of examining individual years of age and change- 70%) among women aged 23 years, 50% (95% CI, 28%–65%) point regression models, and by considering other possible fac- among women aged 24 years, 34% (95% CI, 0%–56%) among tors that could be contributing to trends in CIN2+. women aged 25 years, and 30% (95% CI, 13%–46%) among In places like the United States where data linkage studies women aged 26 years. with immunization registries are not generally possible, inter- Compared to the 1979 birth cohort, the first birth cohort pretation of these trends requires consideration of other pos- to see a significant decline was born in 1985 (22% [95% CI, sible explanations in addition to HPV vaccine impact. One 4%–36%]) (Figure 2). Increasingly larger declines were also possible explanation for declines in high-grade cervical lesions observed in subsequent birth cohorts through 1994 (82% [95% is that changes in screening guidelines for cervical cancer now CI, 74%–88%]), the last birth cohort for which data were avail- recommend less frequent screening among women included Downloaded from https://academic.oup.com/cid/article-abstract/65/6/884/3829989 by Yale University Law School user on 12 March 2019 able during the study period. in our analysis [13]. The extent to which these guidelines, first Ecological comparisons are presented in Table 2. Among recommended in 2009 and adopted more widely by 2012, have women aged 21–24 years, rates of CIN2+ declined by 14.8% per been implemented in practice is likely to vary and has not year, concomitant with a 1.6% per year decrease in Pap screen- yet been fully described. Using population-based survey data ing and <1% annual change in reported C. trachomatis rates. (BRFSS), we noted only a 1.6% annual decline in screening Sexual risk behaviors including ever having had sex and hav- among women aged 21–24 in CT during 2008–2014, during ing had sex with ≥4 people decreased among adolescents in CT which time CIN2+ declined in this age group by 14.8% per during this time period (declines of 3.1% and 7.4% per year). year. Additionally, the patterns of sequential declines by age HPV vaccination coverage with at least 1 dose increased 6.7% and birth cohort suggest that the declines are not principally per year among adolescents aged 13–17 years in CT during driven by changes in screening, which might be expected to 2008–2015 and 18.0% per year among females aged 19–26 years affect all age groups similarly. Thus, it is not likely that changes in the United States during 2010–2014. in screening can fully explain the decline in CIN2+. The rela- tively small decline in screening in this age group could reflect DISCUSSION the new guidelines to begin screening at age 21 years; thus, all We found that rates of precancerous high-grade cervical lesions women in this age group should have a Pap smear. We are not declined significantly and substantially among women aged aware of other published estimates of changes in screening by 21–26 years in CT during 2008–2015, with the earliest and larg- comparable age groups (eg, 21–24 years, 25–29 years, or even est declines occurring among younger women. Birth cohorts individual year of age). Such data would be very useful to inter- that experienced significant declines began in 1985 and con- preting trends in precancerous cervical lesions. Furthermore, tinued with larger declines through 1994. These results build we note that our observations are consistent with data from on a small but growing body of research in the United States other countries, including Australia and Denmark, that experi- on trends in CIN2+ [18, 19] by using more robust analytic enced comparable declines in high-grade cervical lesions after

Figure 2. Relative change in incidence rates of high-grade cervical lesions and 95% confidence intervals (CIs) by birth cohort 1980–1994 compared to 1979, Connecticut, 2008–2015. Dot for each birth cohort indicates point estimate relative difference from 1979 (referent) birth cohort, and lines indicate 95% CIs.

Declines in HPV-Associated Lesions • CID 2017:65 (15 September) • 887 Table 2. Annual Percentage Changes in Cervical Intraepithelial sexual exposure to HPV) age into the high-risk group for cervi- Neoplasia Grade 2+, Human Papillomavirus Vaccination Coverage, cal lesions, we expect progressive declines in the rate of disease Cervical Cancer Screening, and Indicators of Sexual Behavior in Female Residents of Connecticut, by Age by individual year. This is consistent with our results. Previous studies that examined trends in high-grade cervical lesions did Annual not report data by individual years of age or birth cohorts but Years of Data Percentage Measure Available Age Group, y Change rather broader age categories (eg, 21–29 years) that likely con- found the effects of vaccine impact and changes to cervical can- Rate of CIN2+ 2008–2015 21–24 –14.8% 25–29 –3.8% cer screening guidelines [16, 24]. 30–34 –1.0% Our findings should be interpreted in light of certain con- 35–39 0.4% siderations. First, this is an ecological analysis that does not Downloaded from https://academic.oup.com/cid/article-abstract/65/6/884/3829989 by Yale University Law School user on 12 March 2019 HPV vaccination coverage 2008–2015 13–17 (CT) 6.7% include vaccination histories for cases. Other study designs with at least 1 dose 2010–2014 19–26 (US) 18.0% such as vaccine effectiveness studies that compare vaccina- Proportion of women who 2008–2014 21–24 –1.6% tion histories at the individual level between women with and had a Pap test last year 25–29 –4.6% 30–34 –3.5% without HPV-related outcomes are also important approaches 35–39 –0.6% and should be conducted when possible. However, an impor- Indicators of sexual behavior tant strength of our design is that it measures population-level Ever had sexual 2009–2015 Grades 9–12 –3.8% impact and can assess both direct and indirect effects of vac- intercourse Had sex with ≥4 2009–2015 Grades 9–12 –7.3% cination, thereby capturing any possible effect of herd immu- persons nity. Second, the extent to which these findings that are specific No condom use during 2009–2015 Grades 9–12 0.9% to CT may apply to other regions of the United States is not last sex known. Finally, some of our ecological measures are not directly Rate of Chlamydia tra- 2008–2015 20–24 –0.1% chomatis diagnoses 25–29 2.0% comparable due to limitations in available data sources such as 30–34 2.7% different age groups, years, and/or regions. 35–39 3.3% Our results add to the small but growing body of literature about Abbreviations: CIN2+, cervical intraepithelial neoplasia grade 2 or higher; CT, Connecticut; trends in CIN2+, an important clinical outcome, since introduc- HPV, human papillomavirus; US, United States. tion of HPV vaccines. Though it is encouraging that declines in CT have been achieved with only moderate vaccine coverage, HPV vaccine introduction in the absence of similar changes to these results should promote renewed efforts to increase vaccina- screening recommendations during the same period [20–22]. tion coverage to achieve the vaccines full prevention potential. Changes in sexual risk behaviors are also not likely to explain the declines we observed in CIN2+. Rates of C. trachomatis Supplementary Data among women aged 20–24 years in CT remained steady during Supplementary materials are available at Clinical Infectious Diseases online. Consisting of data provided by the authors to benefit the reader, the posted 2008–2015 while rates of CIN2+ were declining significantly materials are not copyedited and are the sole responsibility of the authors, and substantially. Though reported rates of C. trachomatis so questions or comments should be addressed to the corresponding may be influenced not only by sexual risk behaviors but also author. by changes in diagnostic, screening, and reporting practices [23], and screening practices may vary by age, these data do not Notes provide compelling evidence of substantial reductions in the Acknowledgments. The authors thank Lauri Markowitz, Julia Gargano, likelihood of acquiring HPV during the study period. We also and Susan Hariri at the Centers for Disease Control and Prevention (CDC) for contributions to the surveillance project. examined sexual behaviors among high school students that Financial support. This work was supported in part by the National revealed declines in risk, but not of a comparable magnitude to Center for Emerging and Zoonotic Infectious Diseases at CDC through the changes in rates of CIN2+. These data are somewhat limited for Emerging Infections Program. D. M. W. is partially supported by a grant from the Bill & Melinda Gates Foundation (OPP1114733) for this work. comparison due to the different age groups. Potential conflicts of interest. All authors: No reported conflicts of During the study period, declines in rates of CIN2+ among interest. All authors have submitted the ICMJE Form for Disclosure of young women were accompanied by substantial increases in Potential Conflicts of Interest. Conflicts that the editors consider relevant to HPV vaccination coverage, among both adolescent and young the content of the manuscript have been disclosed. adult females. Furthermore, the pattern and timing of the References declines we observed are consistent with HPV vaccine impact 1. Markowitz LE, Hariri S, Unger ER, Saraiya M, Datta SD, Dunne EF. Post-licensure as evidenced by the changes observed for individual years of monitoring of HPV vaccine in the United States. Vaccine 2010; 28:4731–7. 2. Drolet M, Bénard É, Boily MC, et al. Population-level impact and herd effects age and birth cohorts. Each year, as more women who had the following human papillomavirus vaccination programmes: a systematic review opportunity to be vaccinated as young adolescents (prior to and meta-analysis. Lancet Infect Dis 2015; 15:565–80.

888 • CID 2017:65 (15 September) • Niccolai et al 3. Hariri S, Markowitz LE, Dunne EF, Unger ER. Population impact of HPV vac- 14. Centers for Disease Control and Prevention. National, regional, state, and selected cines: summary of early evidence. J Adolesc Health 2013; 53:679–82. local area vaccination coverage among adolescents aged 13–17 years —United 4. Mariani L, Vici P, Suligoi B, Checcucci-Lisi G, Drury R. Early direct and indi- States, 2014. MMWR Morb Mortal Wkly Rep 2015; 64:784–92. rect impact of quadrivalent HPV (4HPV) vaccine on genital warts: a systematic 15. Niccolai LM, Julian PJ, Bilinski A, et al. Area-based poverty, racial, and ethnic review. Adv Ther 2015; 32:10–30. disparities in cervical cancer precursor rates in Connecticut, 2008–2009. Am J 5. Insinga RP, Glass AG, Rush BB. The health care costs of cervical human papillo- Public Health 2013; 103:156–63. mavirus–related disease. Am J Obstet Gynecol 2004; 191:114–20. 16. Niccolai LM, Julian P, Meek J, McBride V, Hadler J, Sosa L. Declining rates of 6. Rogstad KE. The psychological impact of abnormal cytology and colposcopy. high-grade cervical lesions in young women in Connecticut, 2008–2011. Cancer BJOG 2002; 109:364–8. Epidemiol Biomarker Prev 2013; 22:1446–50. 7. Baldur-Felskov B, Dehlendorff C, Munk C, Kjaer S. Early impact of human pap- 17. Pingali SC, Warren JL, Mead AM, Sharova N, Petit S, Weinberger DM. Association illomavirus vaccination on cervical neoplasia—nationwide follow-up of young between local pediatric vaccination rates and patterns of pneumococcal disease in Danish women. J Natl Cancer Inst 2014; 106:djt460. adults. J Infect Dis 2015. doi:10.1093/infdis/jiv431. 18. Flagg EW, Torrone EA, Weinstock H. Ecological association of human papilloma- 8. Crowe E, Pandeya N, Brotherton JM, et al. Effectiveness of quadrivalent human virus vaccination with cervical dysplasia prevalence in the United States, 2007–

papillomavirus vaccine for the prevention of cervical abnormalities: case-control Downloaded from https://academic.oup.com/cid/article-abstract/65/6/884/3829989 by Yale University Law School user on 12 March 2019 2014. Am J Public Health 2016; 106: 2211–8. study nested within a population based screening programme in Australia. BMJ 19. Benard VB, Castle PE, Jenison SA, et al. Population-based incidence rates of cer- 2014; 348:g1458. vical intraepithelial neoplasia in the human papillomavirus vaccine era. JAMA 9. Gertig DM, Brotherton JM, Budd AC, Drennan K, Chappell G, Saville AM. Oncol 2016. doi:10.1001/jamaoncol.2016.3609. Impact of a population-based HPV vaccination program on cervical abnormali- 20. Baldur-Felskov B, Dehlendorff C, Junge J, Munk C, Kjaer SK. Incidence of cervi- ties: a data linkage study. BMC Med 2013; 11:227. cal lesions in Danish women before and after implementation of a national HPV 10. Mahmud SM, Kliewer EV, Lambert P, Bozat-Emre S, Demers AA. Effectiveness vaccination program. Cancer Causes Control 2014; 25:915–22. of the quadrivalent human papillomavirus vaccine against cervical dysplasia in 21. Baldur-Felskov B, Munk C, Nielsen TSS, et al. Trends in the incidence of cervical Manitoba, Canada. J Clin Oncol 2014. doi:10.1200/JCO.2013.52.4645. cancer and severe precancerous lesions in Denmark, 1997–2012. Cancer Causes 11. Pollock KG, Kavanagh K, Potts A, et al. Reduction of low- and high-grade cer- Control 2015. doi:10.1007/s10552-015-0603-7. vical abnormalities associated with high uptake of the HPV bivalent vaccine in 22. Brotherton JM, Fridman M, May CL, Chappell G, Saville AM, Gertig DM. Early Scotland. Br J Cancer 2014; 111:1824–30. effect of the HPV vaccination programme on cervical abnormalities in Victoria, 12. Lipsitch M, Jha A, Simonsen L. Observational studies and the difficult quest for Australia: an ecological study. Lancet 2011; 377:2085–92. causality: lessons from vaccine effectiveness and impact studies. Int J Epidemiol 23. Centers for Disease Control and Prevention. Sexually transmitted disease surveil- 2016. doi:10.1093/ije/dyw124. lance 2014. Atlanta, GA: CDC, 2015. 13. Centers for Disease Control and Prevention. Cervical cancer screening guidelines 24. Hariri S, Johnson ML, Bennett NM, et al; HPV-IMPACT Working Group. for average-risk women. Available at: http://www.cdc.gov/cancer/cervical/pdf/ Population-based trends in high-grade cervical lesions in the early human papil- guidelines.pdf. Accessed 30 November 2015. lomavirus vaccine era in the United States. Cancer 2015; 121:2775–81.

Declines in HPV-Associated Lesions • CID 2017:65 (15 September) • 889 Articles

Population-level impact, herd immunity, and elimination after human papillomavirus vaccination: a systematic review and meta-analysis of predictions from transmission-dynamic models

Marc Brisson*, Élodie Bénard, Mélanie Drolet, Johannes A Bogaards, Iacopo Baussano, Simopekka Vänskä, Mark Jit, Marie-Claude Boily, Megan A Smith, Johannes Berkhof, Karen Canfell, Harrell W Chesson, Emily A Burger, Yoon H Choi, Birgitte Freiesleben De Blasio, Sake J De Vlas, Giorgio Guzzetta, Jan A C Hontelez, Johannes Horn, Martin R Jepsen, Jane J Kim, Fulvio Lazzarato, Suzette M Matthijsse, Rafael Mikolajczyk, Andrew Pavelyev, Matthew Pillsbury, Leigh Anne Shafer, Stephen P Tully, Hugo C Turner, Cara Usher, Cathal Walsh

Summary Background Modelling studies have been widely used to inform human papillomavirus (HPV) Lancet Public Health 2016; decisions; however, many models exist and it is not known whether they produce consistent predictions of 1: e8–17 population-level eff ectiveness and herd eff ects. We did a systematic review and meta-analysis of model predictions Published Online of the long-term population-level eff ectiveness of vaccination against HPV 16, 18, 6, and 11 infection in women September 27, 2016 http://dx.doi.org/10.1016/ and men, to examine the variability in predicted herd eff ects, incremental benefi t of vaccinating boys, and potential S2468-2667(16)30001-9 for HPV-vaccine-type elimination. See Comment page e2 *First 12 authors are listed in Methods We searched MEDLINE and Embase for transmission-dynamic modelling studies published between order of contribution; all other Jan 1, 2009, and April 28, 2015, that predicted the population-level impact of vaccination on HPV 6, 11, 16, and authors are in alphabetical order. 18 infections in high-income countries. We contacted authors to determine whether they were willing to produce new Centre de recherche du CHU de predictions for standardised scenarios. Strategies investigated were girls-only vaccination and girls and boys Québec—Université Laval, vaccination at age 12 years. Base-case vaccine characteristics were 100% effi cacy and lifetime protection. We did Quebec City, QC, Canada (Prof M Brisson PhD, sensitivity analyses by varying vaccination coverage, vaccine effi cacy, and duration of protection. For all scenarios we É Bénard MSc, M Drolet PhD); pooled model predictions of relative reductions in HPV prevalence (RRprev) over time after vaccination and summarised Département de médecine results using the median and 10th and 90th percentiles (80% uncertainty intervals [UI]). sociale et préventive, Université Laval, Quebec City, QC, Canada (Prof M Brisson, Findings 16 of 19 eligible models from ten high-income countries provided predictions. Under base-case assumptions, É Bénard, Prof M-C Boily PhD);

40% vaccination coverage and girls-only vaccination, the RRprev of HPV 16 among women and men was 0·53 (80% UI Department of Infectious Disease Epidemiology, Imperial 0·46–0·68) and 0·36 (0·28–0·61), respectively, after 70 years. With 80% girls-only vaccination coverage, the RRprev of HPV 16 among women and men was 0·93 (0·90–1·00) and 0·83 (0·75–1·00), respectively. Vaccinating boys in College, London, UK (Prof M Brisson, Prof M-C Boily, addition to girls increased the RRprev of HPV 16 among women and men by 0·18 (0·13–0·32) and 0·35 (0·27–0·39) for H C Turner PhD); Centre for

40% coverage, and 0·07 (0·00–0·10) and 0·16 (0·01–0·25) for 80% coverage, respectively. The RRprev were greater for Infectious Disease Control, HPV 6, 11, and 18 than for HPV 16 for all scenarios investigated. Finally at 80% coverage, most models predicted that National Institute of Public Health and the Environment, girls and boys vaccination would eliminate HPV 6, 11, 16, and 18, with a median RRprev of 1·00 for women and men for Bilthoven, Netherlands all four HPV types. Variability in pooled fi ndings was low, but increased with lower vaccination coverage and shorter (J A Bogaards PhD); Infection vaccine protection (from lifetime to 20 years). and Cancer Epidemiology Group, International Agency for Research on Cancer, Lyon, Interpretation Although HPV models diff er in structure, data used for calibration, and settings, our population-level France (I Baussano MD, predictions were generally concordant and suggest that strong herd eff ects are expected from vaccinating girls only, F Lazzarato MSc, S P Tully PhD); even with coverage as low as 20%. Elimination of HPV 16, 18, 6, and 11 is possible if 80% coverage in girls and boys Unit of Cancer Epidemiology, is reached and if high vaccine effi cacy is maintained over time. Department of Medical Sciences, University of Turin, Turin, Italy (F Lazzarato); Funding Canadian Institutes of Health Research. Vaccination Programme Unit, National Institute for Health Copyright © The Author(s). Published by Elsevier Ltd. This is an Open Access article under the CC BY-NC-ND license. and Welfare, Helsinki, Finland (S Vänskä PhD); Modelling and Economics Unit, Public Health Introduction also targets HPV 6 and 11, which are associated with England, London, UK Since 2006, two prophylactic human papillomavirus 80–90% of anogenital wart cases.6 Large randomised (M Jit PhD); Department of (HPV) vaccines have been widely used worldwide: the controlled clinical trials have shown that both vaccines Infectious Disease Epidemiology, London School bivalent and quadrivalent vaccines. Both vaccines target are highly eff ective in protecting against vaccine-type of Hygiene & Tropical Medicine, HPV 16 and 18, which cause about 50% of high-grade persistent HPV infection and precancerous lesions in London, UK (M Jit); Cancer cervical lesions, 70% of cervical cancers, and 40–80% of women and men (vaccine effi cacy 93–100%).7,8 More Research Division, Cancer other HPV-related cancers.1–5 The quadrivalent vaccine than 65 countries have introduced HPV vaccination Council NSW, Sydney, NSW, www.thelancet.com/public-health Vol 1 November 2016 e8 Articles

Australia (M A Smith MPH, Prof K Canfell PhD); Department Research in context of Epidemiology and Biostatistics, VU University Evidence before this study Given the substantial herd eff ects of girls-only vaccination Medical Center, Amsterdam, Many models have been developed over the past decade to when coverage is moderate to high, the incremental benefi t of Netherlands (J Berkhof PhD); understand HPV epidemiology and to help guide policy decisions vaccinating boys is predicted to be small. To our knowledge, our Lowy Cancer Research Centre, concerning HPV vaccination. However, it is unknown whether study is the fi rst to suggest that elimination of vaccine-targeted Prince of Wales Clinical School, University of New South Wales, these models are giving consistent results in terms of HPV HPV types is possible if vaccination coverage of girls and boys Sydney, NSW, Australia vaccination population-level eff ectiveness, herd eff ects, reaches 80%. Finally, our study includes a greater number of (Prof K Canfell); Division of STD incremental benefi t of vaccinating boys in addition to girls, models than any published comparison of infectious disease Prevention, Centers for Disease Control and Prevention, and elimination. To examine this question, we conducted models, refl ecting the infl uential role modelling has played in Atlanta, GA, USA a systematic review and meta-analysis of HPV HPV vaccination policy decisions. (H W Chesson PhD); Center for transmission-dynamic model predictions of the long-term Implications of available evidence Health Decision Science population-level eff ectiveness of HPV vaccination against HPV 6, (E A Burger PhD, Prof J J Kim PhD) HPV vaccination is likely to have a strong direct and indirect 11, 16, and 18 infection in women and men. To our knowledge, and Department of Global impact across diff erent countries and coverage levels. The case this is the fi rst meta-analysis of HPV model predictions, and is the Health and Population for vaccinating boys could mainly depend on other issues (J A C Hontelez PhD), fi rst to examine the potential for elimination. Harvard T H Chan School of besides the predicted additional benefi t for heterosexual males, Public Health, Boston, MA, Added value of the study such as cost of the vaccine for boys, feasibility of increasing USA; Department of Health Our study shows that the HPV models give generally consistent coverage in girls, and equity for men who have sex with men. Management and Health results, which can be reassuring for decision makers using these Although early post-HPV vaccination surveillance data do not Economics, University of Oslo, Oslo, Norway (E A Burger); models for vaccination policy decisions. Results suggest that show herd eff ects in settings with low coverage, our study National Infection Service, HPV vaccination will produce strong herd eff ects leading to suggests that this is probably because herd eff ects take longer Public Health England, London, substantial long-term reductions in HPV infection and related to become evident when coverage is lower. Policy makers might UK (Y H Choi PhD); Oslo Centre diseases in unimmunised women and men. Herd eff ects are want to examine whether to set objectives to eliminate HPV for Biostatistics and Epidemiology, Division of predicted even with vaccination coverage as low as 20%. vaccine types in their jurisdictions. Infectious Disease Control, Norwegian Institute of Public 9,10 Health and Oslo Centre for programmes. Most programmes target girls only, with A systematic review of population-level HPV Statistics and Epidemiology, only a handful of countries vaccinating both girls and post-vaccination surveillance data has shown signifi cant Department of Biostatistics, boys (eg, the USA, Australia, Switzerland, Austria, reductions in anogenital warts in young men in the Institute of Basic Medical and Canada).11–17 These decisions have been made with fi rst 4 years after girls-only quadrivalent vaccination Sciences, University of Oslo, 11,18–22 37 Oslo, Norway substantial input from mathematical models. programmes with high coverage (≥50%). In countries (Prof B Freiesleben De Blasio PhD); Recently, a nonavalent vaccine, which targets HPV 31, 33, with low vaccination coverage (<50%) there was no Department of Public Health, 45, 52, and 58 also, has been licensed and recommended indication of herd eff ects.37 In Australia, where Erasmus MC, University for use in the USA23 after reports of strong effi cacy and quadrivalent vaccination coverage among girls has Medical Center Rotterdam, 24 Rotterdam, Netherlands immunogenicity from large trials. been consistently higher than 70%, anogenital warts (S J De Vlas PhD, J A C Hontelez, Mathematical models have consistently predicted that consultations among heterosexual men declined by more S M Matthijsse MSc); vaccinating girls against HPV is highly cost-eff ective,25–27 than 80% within the fi rst 5 years of the programme Fondazione Bruno Kessler, however, the picture is less clear for vaccinating boys.28–35 (before vaccination of boys).38 These results suggest that Trento, Italy (G Guzzetta PhD); Epidemiological and Statistical This is because the cost-eff ectiveness of vaccinating girls HPV vaccination can produce important herd eff ects for Methods Research Group, is mainly driven by the direct benefi t of HPV vaccination the anogenital warts associated types (HPV 6 and 11), and Helmholtz Centre for Infection among vaccinated women, which depends on well that elimination of these types might be achievable. Research, Braunschweig, Germany (J Horn Dipl Biomath, quantifi ed parameters such as vaccine effi cacy and the Better understanding the potential long-term population- 27 R Mikolajczyk MD); Section for proportion of cervical cancer due to the vaccine types. level eff ectiveness of HPV vaccination, including herd Geography, Department of The incremental eff ectiveness and cost-eff ectiveness of eff ects, is crucial to help inform future vaccine policy Geosciences and Natural vaccinating boys is infl uenced largely by the magnitude decisions such as the inclusion of boys in vaccination Resource Management, University of Copenhagen, of indirect protection conferred to men by vaccinating programmes, incremental impact of increasing vaccin- Copenhagen, Denmark girls (herd immunity), which depends on a complex ation coverage and optimal combin ations of HPV (M R Jepsen PhD); Merck combination of factors (eg, vaccination coverage and vaccination and cervical cancer screening strategies. Research Laboratories, Rahway, sexual behaviour).28,35,36 Modelling studies have shown Mathematical models provide a formal framework to NJ, USA (A Pavelyev PhD, M Pillsbury PhD); Department that if vaccinating girls signifi cantly reduces the burden examine these questions, which cannot be answered in of Internal Medicine, of HPV-related diseases in men through herd immunity, trial settings. However, models require many assumptions, University of Manitoba, then vaccinating boys will produce limited additional which might lead to questions about the validity of Winnipeg, MB, Canada population-level benefi ts for heterosexual men and predictions and uncertainty for decision makers. A large (L A Shafer PhD); National Centre for Pharmacoeconomics women and thus will not be cost-eff ective at the same number of HPV models have been developed over the past (NCPE Ireland), Dublin, Ireland vaccine price.28,30,35 However, it is unclear what vaccination decade, but it is still unclear whether the models are giving (C Usher PhD); and Department coverage is necessary to achieve substantial herd eff ects, consistent results and whether we can draw general of Mathematics and Statistics, and whether models are consistent in their predictions. principles from them. We conducted a systematic review e9 www.thelancet.com/public-health Vol 1 November 2016 Articles

and meta-analysis of model predictions of the long-term Additionally, MB and ÉB provided the modelling teams University of Limerick, population-level eff ectiveness of HPV vaccination against with 19 predetermined vaccination scenarios and asked Limerick, Ireland HPV 6, 11, 16, and 18 infection in women and men, to them to transfer these new results using a standardised (Prof C Walsh PhD) examine the robustness and variability of predicted herd format (appendix p 3). Our primary outcome was the Correspondence to: Dr Marc Brisson, Centre de eff ects, incremental benefi t of vaccinating boys, and relative reduction in the overall prevalence (RRprev) of recherche du CHU de Québec— potential for HPV vaccine-type elimination. HPV 16, 18, 6, and 11 among heterosexual women and Université Laval, Axe Santé des men after 70 years of vaccination (vs no vaccination). We populations et pratiques Methods decided, a priori, to stratify outcomes by HPV type and optimales en santé, Québec City, QC, G1S 4L8, Canada Search strategy and selection criteria sex. Results were stratifi ed by sex to investigate the herd marc.brisson@crchudequebec.

In this systematic review and meta-analysis, we used a eff ects of vaccinating girls only on RRprev among ulaval.ca three-step systematic process to identify independently heterosexual men, and were stratifi ed by type to examine developed HPV transmission-dynamic mathematical if model predictions of herd immunity or elimination models from the published literature, which we report in diff ered between HPV vaccine types. Base-case vaccine accordance with the PRISMA guidelines.39 First, we characteristics used by all groups were 100% vaccine systematically reviewed the literature. Modelling studies effi cacy and lifetime duration of protection. In the were eligible for inclusion if the model: (1) was an HPV sensitivity analyses, we varied vaccine effi cacy (to 90%) See Online for appendix transmission-dynamic model; (2) produced predictions of the population-level impact of vaccination on 612 articles identified from 16 meeting abstracts identified from HPV 16, 18, 6, or 11 infections among women or men or MEDLINE and Embase database search search of EUROGIN 2013 and IPV 2014 both; and (3) was developed to examine the impact of HPV vaccination in a high-income country. We searched MEDLINE and Embase databases using a combination of 549 articles excluded 452 not mathematical modelling 16 meeting abstracts excluded the following MeSH terms, title or abstract words, without studies because models already restriction on the language of the articles: (“models, 67 not a dynamic model published 22 model not representative of theoretical” or “mathematical model” or “models, HIC country statistical” or “cost-benefi t analysis” or “cost-eff ectiveness” 8 model did not present or “risk-benefi t analysis”) and (“papillomavirus vaccines” population-level impact of 0 meeting abstracts included HPV vaccination or “papillomavirus vaccination” or “human papillomavirus vaccine” or “HPV vaccine” or “HPV vaccination”); the exact search for both databases is presented in the appendix p 2. We included models published between Jan 1, 2009, and 63 full-text articles reviewed April 28, 2015, because the authors were likely to use their model to inform future policy decisions. We identifi ed 12 excluded eligible studies through review of titles and abstracts. To 8 not a dynamic model identify additional studies, we reviewed the references of 1 model not representative of HIC 3 model developed by another team already selected articles and hand searched the abstracts from identified in database search the main HPV conferences (Eurogin Congress 2013; Florence, Italy; Nov 3–5; and International Papillomavirus Conference 2014; Seattle, WA, USA; Aug 20–25). MB, ÉB, 51 eligible and MD assessed the eligibility of all studies.

Second, we decided a priori to group the selected Articles grouped according to research studies by research team. Teams were not selected if teams and 19 contacted to participate they used a model previously developed by another team. MB contacted the senior or corresponding authors of all teams to determine whether they were willing to 2 teams were unable to participate produce new predictions using a standardised-input dataset. The ability to provide new standardised 17 participating teams predictions was a prerequisite for inclusion of a model in this meta-analysis, to adequately compare and pool 1 model excluded because it was built for model results. Third, when modelling teams had short-term predictions and had unstable diff erent versions of their models, we included only their long-term predictions most recent published model in the meta-analysis to ensure independence between model predictions. 16 independent models Data collection and quality assessment Figure 1: Model selection All eligible and participating teams were asked to fi ll a HPV=human papillomavirus. HIC=high-income country. EUROGIN=Eurogin International Multidisciplinary standardised form to describe their models in detail. Congress. IPV=International Papillomavirus Conference. www.thelancet.com/public-health Vol 1 November 2016 e10 Articles

A B Men Women Men Women Tully 98% (97–99) 98% (97–99) Jepsen 100% (100–100) 100% (100–100) Mikolajczyk 61% 68% Tully 100% (100–100) 100% (100–100) Chesson 56% 67% Mikolajczyk 100% 100% Jepsen 61% (57–66) 67% (63–71) Guzzetta 98% (97–98) 99% (98–99) Elbasha 48% 58% Jit 96% (56–100) 98% (84–100) Matthijsse 41% 57% Chesson 85% 94% Brisson 45% (32–53) 55% (47–66) Brisson 86% (83–89) 93% (92–95) Jit 35% (22–56) 53% (43–68) De Blasio 82% 93% Bogaards 37% 52% Elbasha 88% 93% De Blasio 34% 52% Matthijsse 80% 92% Guzzetta 30% (29–30) 50% (49–50) Bogaards 82% 92% Canfell 31% 48% Turner 80% 91% Baussano 31% 48% Baussano 75% 91% Vanska 27% 47% Vanska 75% 90% Turner 28% 45% Canfell 74% 89% Burger 28% (27–32) 44% (44–46) Burger 68% (67–74) 85% (84–87) Pooled 36% (28–61) 53% (46–68) Pooled 83% (75–100) 93% (90–100)

Relative reduction of HPV prevalence (%) Relative reduction of HPV prevalence (%) C D Women Men Women Men Tully NA NA Jepsen 0% 0% Mikolajczyk 28% 35% Tully NA NA Chesson 15% 26% Mikolajczyk 0% 0% Jepsen 33% 39% Guzzetta 1% 2% Elbasha 7% 13% Jit 2% 4% Matthijsse 16% 30% Chesson 3% 13% Brisson 20% 31% Brisson 7% 14% Jit 24% 39% De Blasio 7% 18% Bogaards 22% 36% Elbasha 4% 8% De Blasio 19% 36% Matthijsse 7% 19% Guzzetta 36% 55% Bogaards 8% 18% Canfell 18% 35% Turner NA NA Baussano 17% 34% Baussano 9% 25% Vanska 17% 36% Vanska 10% 25% Turner NA NA Canfell 9% 24% Burger 12% 36% Burger 11% 29% Pooled 18% (13–32) 35% (27–39) Pooled 7% (0–10) 16% (1–25)

0 10 20 30 40 50 60 70 80 90 100 0 10 20 30 40 50 60 70 80 90 100 Incremental relative reduction of HPV prevalence (%) Incremental relative reduction of HPV prevalence (%)

Figure 2: Population-level impact of HPV vaccination of girls only (A, B) and boys and girls (C, D)

Predicted relative reduction in the prevalence (RRprev) of HPV 16 among women and men after 70 years of girls-only vaccination, assuming 40% (A) and 80% (B) vaccination coverage; and predicted incremental relative reduction in the prevalence of HPV 16 among women and men after 70 years by vaccinating boys in addition to girls only, assuming 40% (C) and 80% (D) vaccination coverage. The pooled estimates are medians and 80% uncertainty intervals (10% and 90% percentile) of predictions. Models with error bars provided uncertainty intervals (10th and 90th percentile) around their median model predictions. When a model’s results includes a median estimate and uncertainty range, the pooled results used the median value. HPV=human papillomavirus. NA=not available.

and duration of protection (to 20 years) for fi xed Data analysis vaccination coverage of 40% and 80%. As a secondary We derived pooled predictions of HPV vaccination

outcome, we present RRprev over time since the population-level impact by calculating the median and introduction of vaccination. 10th, 25th, 75th, and 90th percentiles of the predictions

Before contacting modelling teams, MB and ÉB of the models, and present RRprev with 80% uncertainty assessed if the studies were of suffi cient methodological intervals (80% UI; the 10th and 90th percentiles). These quality to be included in the pooled analysis. Following pooled results illustrate the central tendency and recommendations from the International Society for variability or robustness of model predictions, with all Pharmacoeconomics and Outcomes Research and Society models having equal weight. These pooled results for Medical Decision Making (ISPOR-SMDM) modelling are diff erent to summary estimates from classical good research practices task force,40 the main quality meta-analysis of empirical studies, which are pooled with criteria were that the models had to be transmission- weights based on sample size or variance. Incremental

dynamic mathematical models and be calibrated to RRprev were obtained by subtracting the RRprev of girls-only

epidemiological data. Given that an objective of the vaccination from the RRprev of girls and boys vaccination. meta-analysis was to examine herd eff ects and HPV We used univariate linear meta-regressions to identify elimination, the transmission-dynamic model criterion potential sources of heterogeneity among the diff erent was included as an eligibility criterion. Finally, once models’ predictions for HPV 16, giving equal weight model results were obtained from the participating to all models. We looked at model characteristics modelling teams, MB and ÉB assessed the construct in univariate analysis and examined the potential validity of predictions. interactions between each characteristic and vaccination e11 www.thelancet.com/public-health Vol 1 November 2016 Articles

coverage (40% and 80%), and between each characteristic Post-vaccination reduction in HPV prevalence over and vaccination strategy (girls only and girls and boys). time after girls-only vaccination was faster for women Given the number of models, it was not possible to than men, for types HPV 6 and 11 compared with HPV 16 perform multivariate analysis. We verifi ed that our and 18, and at higher coverage than at lower coverage results were similar when using HPV 18. We used SAS (appendix pp 15–16). version 9.4 for all analyses. Vaccinating 40% of boys in addition to 40% of girls (girls and boys strategy) resulted in incremental reductions of Role of the funding source HPV 16 prevalence of 18% among women (incremental

The funders of the study had no role in the study design, RRprev 0·18 [80% UI 0·13–0·32]) and 35% among men data collection, analysis and interpretation, or writing of (0·35 [0·27–0·39]) after 70 years (fi gure 2C). Given the the report. MB, ÉB, and MD had full access to all the data important herd eff ects following girls-only vaccination in the study and MB had fi nal responsibility for the with 80% coverage, vaccinating 80% of boys in addition to decision to submit for publication. girls produced small incremental reductions in HPV 16

Results A Our search led to the identifi cation of 51 articles 40% coverage 80% coverage published by 19 diff erent research teams (fi gure 1). Of 100 the 19 teams contacted to participate in the meta-analysis, 90 17 provided new standardised model predictions. All models met the main methodological quality criteria. 80 41 However, one model was excluded because it was built 70 for short-term predictions, and demographic change assumptions produced unstable long-term predictions. 60 The 16 models included in the meta-analysis vary in 50 terms of type (deterministic or stochastic), structure (assumptions about sexual activity, partnership 40 formation and dissolution, transmission, and natural 30 immunity), and baseline HPV prevalence, and were developed in ten countries (Australia, Canada, Finland, of HPV prevalence (%) reduction Relative 20 Germany, Ireland, Italy, Netherlands, Norway, UK, and 10 Men USA; appendix pp 4–5). HPV 16, 18, 6, and 11 were Women included in 16, 13, fi ve, and three models, respectively. 0 Among women after 70 years of girls-only vaccination, B the overall predicted prevalence of HPV 16 decreased 100 by 53% (RRprev 0·53 [80% UI 0·46–0·68]) assuming 90 40% coverage, and by 93% (RRprev 0·93 [80% UI 0·90–1·00]) assuming 80% coverage (fi gure 2). 80 At 40% coverage, the corresponding RRprev for HPV 6, 11, and 18 were 7 to 28 percentage points greater than HPV 16, 70 and at 80% coverage these types were eliminated (HPV 11) 60 or close to elimination (HPV 18 and 6; fi gure 3A, appendix pp 6–13). Given that population-level eff ectiveness (RRprev) 50 was substantially greater than vaccination coverage for all 40 HPV vaccine-targeted types, these results indicate that girls-only vaccination is expected to produce substantial 30 herd eff ects for unvaccinated women even at low coverage, 20 with greater herd immunity eff ects for HPV 18, 6, and 11 than for HPV 16 (fi gure 3A, appendix p 14). of HPV prevalence (%) Incremental relative reduction 10 Important herd eff ects were also predicted in men after 0 girls-only vaccination for both low and high coverage 16 18 6 11 16 18 6 11 scenarios (fi gures 2, 3). Among men, the overall HPV type HPV type prevalence of HPV 16 decreased by 36% after 70 years of Figure 3: Pooled predictions of the vaccine-type-specifi c population-level impact of HPV vaccination girls-only vaccination assuming 40% coverage (RRprev Relative reduction of HPV prevalence among women and men after 70 years of girls-only vaccination (A), and 0·36 [80% UI 0·28–0·61]), and by 83% assuming 80% incremental relative reduction in HPV prevalence among women and men after 70 years by vaccinating boys in coverage (0·83 [0·75–1·00]; fi gure 2). The RR of addition to girls only (B). Shown here are median (line) and 25th and 75th percentiles (box) and 10th and 90th prev percentiles (whiskers) of the predictions of the models. HPV 11 results have a diff erent presentation due to the few HPV 18, 6, and 11 were again greater than that of HPV 16 models that include this outcome. See appendix pp 6–9 for forest plots of model predictions for types HPV 16, 18, 6, (fi gure 3A, appendix p 14). and 11; and appendix pp 10–13 for values of pooled estimates and uncertainty intervals. HPV=human papillomavirus. www.thelancet.com/public-health Vol 1 November 2016 e12 Articles

prevalence (incremental RRprev of 0·07 [80% 0·00–0·10] for vaccination coverage by 40% (from 40% to 80%) reduced women and 0·16 [80% 0·01–0·25] for men), after 70 years HPV 16, 18, 6, and 11 prevalence in women by an (fi gure 2D). The incremental population-level eff ectiveness additional 40%, 38%, 35%, and 19%, respectively; higher of vaccinating boys in addition to girls was similar between than the incremental benefi ts of vaccinating 40% of boys the HPV vaccine types when assuming 40% coverage, but in addition to 40% of girls (fi gure 4; appendix pp 10–13, was substantially higher for HPV 16 (vs HPV 18, 6, 11) 15–16). The same increase in girls-only vaccination when assuming 80% coverage (fi gure 3B). Vaccinating coverage from 40% to 80% was also more eff ective in boys in addition to girls produced a slightly faster decline reducing HPV 16, 18, and 6 prevalence in men than the in vaccine-type-specifi c prevalence among women and incremental benefi t of vaccinating 40% of boys in men (appendix pp 15–16). addition to 40% of girls, and the strategies were equally For the same number of additional vaccinated eff ective for HPV 11 (fi gure 4, appendix pp 10–13). individuals, increasing coverage in a girls-only strategy Increasing coverage considerably improved population- was predicted to provide greater population-level benefi ts level eff ectiveness up to 80% for girls-only vaccination than was including boys in a vaccination programme. and 60% for girls and boys vaccination, after which, The models predicted that increasing girls-only increasing coverage had very little marginal benefi t

A HPV 16 HPV 18 HPV 6 HPV 11 100

90

80

70

60

50

40

30

Relative reduction of HPV prevalence (%) reduction Relative 20

10 Men Women 0 B 100

90

80

70

60

50

40

30

Relative reduction of HPV prevalence (%) reduction Relative 20

10

0 20 40 60 80 100 20 40 60 80 100 20 40 60 80 100 20 40 60 80 100 Coverage (%) Coverage (%) Coverage (%) Coverage (%)

Figure 4: Pooled predictions according to vaccination coverage and vaccine type Relative reduction of HPV prevalence among women and men after girls-only vaccination (A) and after vaccination of boys in addition to girls (B). Shown here are median (line) and 25th and 75th percentiles (box) and 10th and 90th percentiles (whiskers) of the predictions of the models. HPV 11 results have a diff erent presentation due to the limited number of models that include this outcome. See appendix pp 6–9 for forest plots of model predictions for types HPV 16, 18, 6, and 11; and appendix pp 10–13 for values of pooled estimates and uncertainty intervals. HPV=human papillomavirus. e13 www.thelancet.com/public-health Vol 1 November 2016 Articles

(fi gure 4). Substantial herd eff ects were predicted with 80% coverage is achieved in both sexes and the vaccine girls-only vaccination coverage as low as 20% (fi gure 4, confers long-term protection. appendix pp 10–13). Interestingly, 19% (three of 16), 46% Our fi ndings have important policy implications. First, (six of 13), 60% (three of fi ve), and 100% (three of three) because of the important herd eff ects from girls-only of models predicted that HPV 16, 18, 6, and 11, vaccination, our models predict that increasing HPV respectively, would be eliminated among heterosexual vaccination coverage among girls has greater incremental populations if girls-only vaccination reaches 80% benefi t for both male and female individuals than adding coverage (fi gure 4, appendix p 14). The girls and boys boys to a vaccination programme. The herd eff ects for strategy substantially increased the predicted potential heterosexual men are predicted to be about the same for elimination; 64%, 92%, 80%, and 100% of models magnitude as the level of girls-only vaccination coverage. predicted that HPV 16, 18, 6, and 11 would be eliminated Given this, the epidemiological and economic con- with 80% coverage, and a few models predicted siderations about vaccinating boys should focus on the elimination of types HPV 18, 6, and 11 with 60% coverage following issues: the feasibility and incremental marginal (fi gure 4, appendix p 14). costs of increasing vaccination coverage among girls Predicted population-level eff ectiveness and herd versus introducing a gender-neutral programme;42 eff ects were much lower and more variable across the whether the price of the vaccine can be reduced for boys models when assuming that vaccine duration of for the strategy to be cost-eff ective; and the importance protection was 20 years compared with lifelong protection placed on achieving equal protection for men who have (appendix p 17). However, reducing vaccine effi cacy from sex with men (MSM). 100% to 90% had little impact on the pooled predictions Second, our models suggest that elimination of (appendix p 17). HPV 6, 11, and 18 is likely if vaccination coverage of In the meta-regression analysis, vaccination coverage girls and boys reaches 80% and the vaccine provides and vaccination strategies were the main sources of long-term protection. However, vaccination coverage heterogeneity between the predictions of the models might have to be slightly higher than 80% to eliminate (data not shown). However, the meta-regression results HPV 16, the type responsible for most of the HPV-related are presented separately by vaccination coverage and burden. These results also have implications for the vaccination strategy, given that signifi cant interactions recently licensed nonavalent vaccine. Given their were observed between these variables and the other prevalence and biological characteristics, the additional model characteristics (appendix pp 18–19). Across all types are expected to behave similarly to type HPV 18 vaccination scenarios, the main source of heterogeneity and thus be easier to eliminate than HPV 16. It should in predictions for women and men was whether the also be noted that the time to elimination might be transmission-dynamic models included (or not) shorter than was predicted in the meta-analysis diff erent sexual activity risk groups (appendix pp 18–19). (appendix pp 15–16), because we did not include Models that did not include risk groups for sexual scenarios with catch-up programmes. Policy makers activity predicted signifi cantly higher population-level might want to examine whether to set objectives to HPV vaccination eff ectiveness and herd eff ects. eliminate HPV vaccine types in their jurisdictions and Population-level eff ectiveness was also signifi cantly set vaccination coverage targets. higher among models that assume lower natural Third, post-vaccination reduction of HPV 6 and 11 immunity among women (mostly when vaccine infections was predicted to be steeper than for HPV 16 protection is assumed lifelong), or include the natural and 18 infections, which is due to the shorter durations history of cervical cancer (when vaccine protection is of infectiousness and lower transmissibility of these low 20 years). These results were similar when examining oncogenic risk types.43 Hence, HPV 6 and 11-related model predictions for HPV 18. disease (eg, anogenital warts) trends are a poor proxy of change in HPV 16 and 18 and related diseases, and Discussion should not be directly extrapolated to inform policy The fi ndings from our systematic review and meta-analysis decisions regarding prevention of HPV-related cancers suggest that HPV vaccination will produce strong herd and its precursors. eff ects leading to substantial long-term reductions in HPV Fourth, the models predicted substantial herd eff ects infection and related diseases in unimmunised women even at relatively lower coverage (40%). However, a recent and men. Herd eff ects are predicted even with vaccination meta-analysis found evidence of signifi cant herd eff ects coverage as low as 20%, and to be greater for only in higher-coverage settings.37 Our study fi ndings HPV 18, 6, and 11 types than for HPV 16. Given the suggest this diff erence is likely to be because herd eff ects substantial herd eff ects of girls-only vaccination when take longer to become evident where coverage is lower. coverage is moderate to high, the incremental benefi t of Early programme impact data should not be used to rule vaccinating boys is predicted to be small. However, most out herd eff ects in settings where coverage is low. models predict that vaccinating boys in addition to girls This study addresses a key source of concern when could eliminate HPV 16, 18, 6, and 11, as long as using mathematical model results for decision making: www.thelancet.com/public-health Vol 1 November 2016 e14 Articles

the robustness and validity of predictions. By system- Our main fi ndings are likely generalisable to most atically identifying published HPV transmission-dynamic high-income countries given that they are based on models that were independently developed from one models from ten of these countries and very little another and presenting the distribution of predictions, heterogeneity remains when results are stratifi ed by our analysis provides decision makers with a measure vaccination coverage, vaccination strategy, and key of uncertainty and robustness surrounding model model characteristics. In addition, there are similarities predictions. Reassuringly, the key conclusions of this in sexual behaviour,49 HPV type distribution,50,51 and age systematic review were consistent among the models, profi le of HPV prevalence52 among high-income even though the models were built to represent diff erent countries. However, our predicted herd eff ects and countries and varied in structure and assumptions. potential for elimination should be extrapolated to According to good modelling practice guidelines, low-income and middle-income countries (LMIC) with model validation should include between-model caution, because there are important diff erences corroboration using independent models and, if possible, between high-income countries and LMICs in sexual external or predictive validation.44 Our systematic review behaviour and potential cofactors of HPV such as high shows high consistency in HPV model predictions of HIV prevalence.49,53,54 population-level reductions in HPV vaccine types among The study has two main limitations that should women and men when vaccination coverage is 80% or be considered, in addition to those inherent in higher for girls only and 60% or higher for girls and boys meta-analysis.55,56 First, although the models were built vaccination. The models also consistently predict that independently from one another and showed variability in post-vaccination reductions will be smaller for HPV 16 their structures, similarities remain in key structural than the other vaccine types. However, there is substantial assumptions, which could contribute to the consistency of variability in predictions when vaccination coverage is fi ndings. None of the models included MSM, incorporated between 40–60%, and when duration of vaccine multisite infection or transmission (transmission between protection is 20 years. Our meta-regression identifi ed oral, anal, and genital sites), or the possibility of two key model choices or parameters that explain this reactivation. Modelling HPV transmission between men variability: inclusion of sexual activity risk groups in the would be unlikely to have an eff ect on predictions about model and the level of natural immunity among women. the herd eff ects of girls-only vaccination, but could slightly Sexual activity has been shown to be very heterogeneous impact predictions about elimination. Including higher in populations (eg, skewed distribution of number of risk groups or MSM, who have greater HPV prevalence

partners), leading most models built for prediction of (and higher R0), would make it more diffi cult to eliminate interventions against sexually transmitted diseases to HPV, and therefore the coverage required for elimination include sexual risk groups. Furthermore, a recent could be higher than estimated by the models in the meta-analysis found that the proportion of women who systematic review. On the other hand, recent modelling develop natural immunity against a subsequent HPV 16 studies suggest that adding multisite transmission or or 18 infection is greater than 35% when controlling for reactivation would probably have very little impact potential confounders or using neutralising assays.45 on long-term predictions about the population-level When considering only the predictions of models that eff ectiveness of HPV vaccination.57,58 Second, we purposely include diff erent sexual activity risk groups and that decided to examine the predicted impact of vaccination on assume that the proportion of women who develop HPV infection rather than disease endpoints, as the main natural immunity is greater than 35%, the variability in focus of the paper is on herd eff ects and elimination of predictions is substantially reduced with little impact on HPV vaccine types. Additional uncertainty in HPV model the median pooled estimates (eg, at 40% vaccination predictions, when examining the impact of vaccination on

coverage, the RRprev of HPV 16 among women after disease endpoints, will be in the absolute reduction of girls-only vaccination is 0·52 [80% UI 0·45–0·58] vs HPV burden of disease (related to country specifi c 0·53 [0·46–0·68] when considering all predictions pre-vaccination burden of disease and proportion due to (appendix pp 20–21). Finally, although external or the vaccine types) and the timing of these benefi ts (related predictive validation was beyond the scope of the paper, to assumptions about progression from infection to the descriptive results of the pooled analysis are in line lesions or cancer). with results from a recent meta-analysis of post-HPV To our knowledge, our study includes a greater vaccination data,37 showing strong herd eff ects shortly number of models than any published comparison of after vaccination in countries with high vaccination infectious disease models, refl ecting the infl uential coverage and very little evidence of herd eff ects in this role that modelling has played in HPV vaccination timeframe where coverage is low. In addition, models policy decisions. Furthermore, this is the fi rst included in the systematic review have previously shown model comparison study in which systematic review that they reproduce the reduction in HPV 16 and 18 methodology was used to identify all potentially eligible infection46,47 and anogenital warts consultations observed mathematical models (reducing possible selection bias); in the fi rst 5 years after HPV vaccination introduction.48 independence of models was controlled for by including e15 www.thelancet.com/public-health Vol 1 November 2016 Articles

only one model per team and producing predictions for References predetermined and standardised scenarios without prior 1 Cliff ord GM, Smith JS, Aguado T, Franceschi S. Comparison of HPV type distribution in high-grade cervical lesions and cervical harmonisation of model structures or natural history cancer: a meta-analysis. Br J Cancer 2003; 89: 101–05. parameters; and meta-analytic techniques were used 2 Munoz N, Bosch FX, de Sanjose S, et al. Epidemiologic classifi cation to describe the central tendency and variability of of human papillomavirus types associated with cervical cancer. N Engl J Med 2003; 348: 518–27. predictions and identify sources of heterogeneity in 3 De Vuyst H, Cliff ord GM, Nascimento MC, Madeleine MM, results. Finally, our study shows that the HPV models Franceschi S. Prevalence and type distribution of human developed over the past decade give consistent results in papillomavirus in carcinoma and intraepithelial neoplasia of the vulva, vagina and anus: a meta-analysis. Int J Cancer 2009; 124: 1626–36. terms of population-level eff ectiveness against infection, 4 Kreimer AR, Cliff ord GM, Boyle P, Franceschi S. herd eff ects, and the possibility of elimination, which Human papillomavirus types in head and neck squamous cell could be reassuring for decision makers using these carcinomas worldwide: a systematic review. Cancer Epidemiol Biomarkers Prev 2005; 14: 467–75. models for vaccination policy decisions. We also 5 Guan P, Howell-Jones R, Li N, et al. Human papillomavirus types identifi ed key HPV modelling choices (ie, inclusion of in 115,789 HPV-positive women: A meta-analysis from cervical sexual risk groups, natural history of cervical cancer, and infection to cancer. Int J Cancer 2012; 131: 2349–59. natural immunity) that have the greatest infl uence on 6 Garland SM, Steben M, Sings HL, et al. Natural history of genital warts: analysis of the placebo arm of 2 randomized phase III trials model predictions, which should guide the future of a quadrivalent human papillomavirus (types 6, 11, 16, and 18) development of models. vaccine. J Infect Dis 2009; 199: 805–14. In conclusion, the results of this study indicate strong 7 Paavonen J, Naud P, Salmeron J, et al. Effi cacy of human papillomavirus (HPV)-16/18 AS04-adjuvanted vaccine against promise for the long-term population-level impact of HPV cervical infection and precancer caused by oncogenic HPV types vaccination programmes. However, it will be important to (PATRICIA): fi nal analysis of a double-blind, randomised study in continue to validate HPV model predictions and to young women. Lancet 2009; 374: 301–14. 8 Garland SM, Hernandez-Avila M, Wheeler CM, et al. compare them to post-vaccination surveillance data, as Quadrivalent vaccine against human papillomavirus to prevent many policy decisions about HPV vaccination (ie, number anogenital diseases. N Engl J Med 2007; 356: 1928–43. of doses, which vaccine to use, and vaccination of boys) 9 Brotherton J, Bloem P. HPV Vaccination: Current Global Status. and cervical cancer screening remain. Curr Obstet Gynecol Rep 2015; 4: 220–33. 10 World Health Organization. Countries with HPV vaccine in the Contributors national immunization programme and planned introductions. MB conceived of the study through discussion with MD, JB, IB, SV, MJ, World Health Organization/IVB Database. January, 2014. M-CB, JB, KC, EAB, JJK, MAS, and HWC. ÉB did the literature search http://www.who.int/immunization/diseases/hpv/decision_ and pooling of the data. MB drafted the fi rst version of the Article. implementation/en/ (accessed Sept 15, 2014). MD codrafted the methods section. MB, ÉB, and MD independently 11 Centers for Disease Control and Prevention. Recommendations on assessed eligibility of studies. All authors provided new predictions for the use of quadrivalent human papillomavirus vaccine in males. Advisory Committee on Immunization Practices (ACIP), 2011. the purposes of this meta-analysis or codeveloped a model included in MMWR Morb Mortal Wkly Rep 2011; 60: 1705–08. the analysis, and interpreted the results and critically revised the 12 Georgousakis M, Jayasinghe S, Brotherton J, Gilroy N, Chiu C, manuscript for scientifi c content. All authors approved the fi nal version Macartney K. Population-wide vaccination against human of the Article. papillomavirus in adolescent boys: Australia as a case study. Declaration of interests Lancet Infect Dis 2012; 12: 627–34. MB has received an unrestricted grant from Merck (for herpes zoster; 13 Switzerland Federal Offi ce of Public Health. Human papillomavirus. project is fi nished). MD has consulted for GlaxoSmithKline (herpes http://www.bag.admin.ch/themen/medizin/00682/00684/03853/ zoster vaccine). JAB has received consultancy fees from index.html?lang=en (accessed Dec 18, 2015). GlaxoSmithKline and Merck (fees were collected by his employer). 14 Department of Health and Wellness. Prince Edward Island. AP and MP are employees of Merck. All other authors declare no Immunization schedule. http://www.gov.pe.ca/health/ competing interests. immunizationschedule (accessed Dec 18, 2015). 15 Alberta Health Services. Alberta prevents cancer. Acknowledgments http://albertapreventscancer.ca/reduce-your-risk/prevent-hpv- The main funder of this study was the Canadian Institutes of Health infections/ (accessed Dec 18, 2015). Research. We received support from the Canada Research Chairs 16 Ministère de la Santé et des Services Sociaux du Québec (MSSS). programme (support for MB), an operating grant from the Canadian Vaccin contre les infections par les virus du papillome humain Institutes of Health Research (number MOP-119427), and a foundation (VPH). http://sante.gouv.qc.ca/conseils-et-prevention/vaccin- scheme grant from the Canadian Institutes of Health Research (number contre-les-infections-par-les-virus-du-papillome-humain-vph/ FDN-143283). IB is supported by the European Community’s Seventh (accessed Dec 18, 2015). Framework Programme (FP7-HEALTH-2013; grant number 603019) and 17 Austrian Ministry of Health. HPV vaccination. http://www.bmg. the Bill & Melinda Gates Foundation (number OPP1053353), and is an gv.at/home/Timeline/Timeline/?timeline_mode=list&yat-table1- Honorary Research Fellow at the School of Public Health, Imperial fi lters-tags=Pr%C3%A4vention (accessed Feb 26, 2016). College, London, UK. MRJ was supported by the National Institute for 18 Markowitz LE, Dunne EF, Saraiya M, et al. Human papillomavirus Health Research Health Protection Research Unit (NIHR HPRU) in vaccination: recommendations of the Advisory Committee on Immunisation at the London School of Hygiene & Tropical Medicine in Immunization Practices (ACIP). MMWR Recomm Rec 2014; 63: 1–30. partnership with Public Health England. JB is supported from the 19 Comité sur l’immunisation du Québec (CIQ) et Comité scientifi que European Community’s Seventh Framework Programme ad hoc VPH. La Vaccination contre les VPH au Québec: Mise à jour (FP7-HEALTH-2013; grant no. 603019). JJK and EAB are supported by des connaissances et propositions du comité d’experts. Direction the US National Cancer Institute of the National Institutes of Health des risques biologiques et de la santé au travail, Institut national de (number R01CA160744). KC is supported by a NHMRC Career santé publique Québec (INSPQ), 2012: chapter 2. Development Fellowship Grant (number AP1082989). The views 20 Canadian Immunization Committee. Recommendations on a expressed here are those of the authors and not necessarily those of the human papillomavirus immunization program. Ontario: NHS, the NIHR, the Department of Health or Public Health England Canadian Immunization Committee, 2007. nor the Centers for Disease Control and Prevention. www.thelancet.com/public-health Vol 1 November 2016 e16 Articles

21 JCVI. Human papillomavirus vaccines to protect against cervical 40 Pitman R, Fisman D, Zaric GS, et al. Dynamic Transmission cancer. London: Joint Committee on Vaccination and Modeling: A Report of the ISPOR-SMDM Modeling Good Research Immunisation, 2008. Practices Task Force Working Group-5. Med Decis Making 2012; 22 World Health Organization. Evidence based recommendations on 32: 712–21. Human Papilloma Virus (HPV) vaccines schedules: Background 41 Shafer LA, Jeff rey I, Elias B, Shearer B, Canfell K, Kliewer E. paper for SAGE discussions. March 11, 2014. http://www.who.int/ Quantifying the impact of dissimilar HPV vaccination uptake immunization/sage/meetings/2014/april/1_HPV_Evidence_based_ among Manitoban school girls by ethnicity using a transmission recommendationsWHO_with_Appendices2_3.pdf (accessed dynamic model. Vaccine 2013; 31: 4848–55. July 23, 2014). 42 Ryser MD, McGoff K, Herzog DP, Sivakoff DJ, Myers ER. Impact of 23 Advisory Committee on Immunization Practices (ACIP). Meeting coverage-dependent marginal costs on optimal HPV vaccination of the Advisory Committee on Immunization Practices (ACIP) strategies. Epidemics 2015; 11: 32–47. October 29–30, 2014—Summary Report. Atlanta: Centers for 43 Brisson M, Van de Velde N, Boily MC. Diff erent population-level Disease Control and Prevention (CDC), 2015. vaccination eff ectiveness for HPV types 16, 18, 6 and 11. 24 Joura EA, Giuliano AR, Iversen OE, et al. A 9-valent HPV vaccine Sex Transm Infect 2011; 87: 41–43. against infection and intraepithelial neoplasia in women. 44 Weinstein MC, O’Brien B, Hornberger J, et al. Principles of good N Engl J Med 2015; 372: 711–23. practice for decision analytic modeling in health-care evaluation: 25 Brisson M, Van de Velde N, Boily MC. Economic evaluation of report of the ISPOR Task Force on Good Research human papillomavirus vaccination in developed countries. Practices—Modeling Studies. Value Health 2003; 6: 9–17. Public Health Genomics 2009; 12: 343–51. 45 Beachler DC, Jenkins G, Safaeian M, Kreimer AR, Wentzensen N. 26 Fesenfeld M, Hutubessy R, Jit M. Cost-eff ectiveness of human Natural acquired immunity against subsequent genital human papillomavirus vaccination in low and middle income countries: papillomavirus infection: a systematic review and meta-analysis. a systematic review. Vaccine 2013; 31: 3786–804. J Infect Dis 2016; 213: 1444–54. 27 Jit M, Brisson M, Portnoy A, Hutubessy R. Cost-eff ectiveness of 46 Smith MA, Canfell K. Testing previous model predictions against female human papillomavirus vaccination in 179 countries: new data on human papillomavirus vaccination program outcomes. a PRIME modelling study. Lancet Glob Health 2014; 2: e406–14. BMC Res Notes 2014; 7: 109. 28 Brisson M, van de Velde N, Franco EL, Drolet M, Boily MC. 47 Elfstrom KM, Lazzarato F, Franceschi S, Dillner J, Baussano I. Incremental impact of adding boys to current human Human papillomavirus vaccination of boys and extended catch-up papillomavirus vaccination programs: role of herd immunity. vaccination: eff ects on the resilience of programs. J Infect Dis 2016; J Infect Dis 2011; 204: 372–76. 213: 199–205. 29 Choi YH, Jit M, Gay N, Cox A, Garnett GP, Edmunds WJ. 48 Brisson M, Van de Velde N, Drolet M, et al. HPV-advise: Transmission dynamic modelling of the impact of human technical appendices. http://www.marc-brisson.net/HPVadviseCEA. papillomavirus vaccination in the United Kingdom. Vaccine 2010; pdf (accessed Aug 2, 2016). 28: 4091–102. 49 Wellings K, Collumbien M, Slaymaker E, et al. Sexual behaviour in 30 Bogaards JA, Kretzschmar M, Xiridou M, Meijer CJ, Berkhof J, context: a global perspective. Lancet 2006; 368: 1706–28. Wallinga J. Sex-specifi c immunization for sexually transmitted 50 de Sanjose S, Diaz M, Castellsague X, et al. Worldwide prevalence infections such as human papillomavirus: insights from and genotype distribution of cervical human papillomavirus DNA mathematical models. PLoS Med 2011; 8: e1001147. in women with normal cytology: a meta-analysis. Lancet Infect Dis 31 Kim JJ, Goldie SJ. Cost eff ectiveness analysis of including boys in a 2007; 7: 453–59. human papillomavirus vaccination programme in the United 51 Cliff ord GM, Smith JS, Plummer M, Munoz N, Franceschi S. States. BMJ 2009; 339: b3884. Human papillomavirus types in invasive cervical cancer worldwide: 32 Chesson HW, Ekwueme DU, Saraiya M, Dunne EF, Markowitz LE. a meta-analysis. Br J Cancer 2003; 88: 63–73. The cost-eff ectiveness of male HPV vaccination in the 52 Bruni L, Diaz M, Castellsague X, Ferrer E, Bosch FX, de Sanjose S. United States. Vaccine 2011; 29: 8443–50. Cervical human papillomavirus prevalence in 5 continents: 33 Elbasha EH, Dasbach EJ. Impact of vaccinating boys and men meta-analysis of 1 million women with normal cytological fi ndings. against HPV in the United States. Vaccine 2010; 28: 6858–67. J Infect Dis 2010; 202: 1789–99. 34 Ben Hadj Yahia MB, Jouin-Bortolotti A, Dervaux B. Extending the 53 Adler DH, Kakinami L, Modisenyane T, et al. Increased regression human papillomavirus vaccination programme to include males in and decreased incidence of HPV-related cervical lesions among high-income countries: a systematic review of the cost-eff ectiveness HIV-infected women on HAART. AIDS 2012; 26: 1645–52. studiess. Clin Drug Investig 2015; 35: 471–85. 54 Baussano I, Lazzarato F, Brisson M, Franceschi S. 35 Smith MA, Lew JB, Walker RJ, Brotherton JM, Nickson C, Canfell K. Hu man papillomavirus vaccination at a time of changing sexual The predicted impact of HPV vaccination on male infections and behavior. Emerg Infect Dis 2016; 22: 18–23. male HPV-related cancers in Australia. Vaccine 2011; 29: 9122–22. 55 Greenland S. Can meta-analysis be salvaged? Am J Epidemiol 1994; 36 Bogaards JA, Wallinga J, Brakenhoff RH, Meijer CJ, Berkhof J. Direct 140: 783–87. benefi t of vaccinating boys along with girls against oncogenic human 56 Bailar JC 3rd. The promise and problems of meta-analysis. papillomavirus: bayesian evidence synthesis. BMJ 2015; 350: h2016. N Engl J Med 1997; 337: 559–61. 37 Drolet M, Benard E, Boily MC, et al. Population-level impact and 57 Lemieux-Mellouki P, Drolet M, Jit M, Brisson M. Modelling herd eff ects following human papillomavirus vaccination multi-site transmission of HPV and its impact on vaccine programmes: a systematic review and meta-analysis. Lancet Infect Dis eff ectiveness against HPV. 30th International Papillomavirus 2015; 15: 565–80. Conference; Lisbon, Portugal; Sept 17–21, 2015. 38 Ali H, Donovan B, Wand H, et al. Genital warts in young Australians 58 Korostil IA, Regan DG. The potential impact of HPV-16 fi ve years into national human papillomavirus vaccination reactivation on prevalence in older Australians. BMC Infect Dis programme: national surveillance data. BMJ 2013; 346: f2032. 2014; 14: 312. 39 Liberati A, Altman DG, Tetzlaff J, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration. BMJ 2009; 339: b2700.

e17 www.thelancet.com/public-health Vol 1 November 2016 BMJ 2013;347:f5906 doi: 10.1136/bmj.f5906 (Published 9 October 2013) Page 1 of 11

Research BMJ: first published as 10.1136/bmj.f5906 on 9 October 2013. Downloaded from RESEARCH

Autoimmune, neurological, and venous thromboembolic adverse events after immunisation of adolescent girls with quadrivalent human papillomavirus vaccine in Denmark and Sweden: cohort study OPEN ACCESS

1 2 Lisen Arnheim-Dahlström associate professor , Björn Pasternak postdoctoral fellow , Henrik 2 1 2 Svanström statistician , Pär Sparén professor , Anders Hviid senior investigator

1Department of Medical Epidemiology and Biostatistics, Karolinska Institutet, Stockholm 171 77, Sweden; 2Department of Epidemiology Research, Statens Serum Institut, Copenhagen, Denmark

Abstract criteria. Furthermore, the pattern of distribution in time after vaccination http://www.bmj.com/ Objective To assess the risk of serious adverse events after vaccination was random for all three and the rate ratios for these outcomes in the of adolescent girls with quadrivalent human papillomavirus (qHPV) period from day 181 after vaccination were similar to the rate ratios in vaccine. the primary risk period. The rate ratios for five neurological events were not significantly increased and there were inverse associations with Design Register based cohort study. epilepsy (rate ratio 0.66, 95% confidence interval 0.54 to 0.80) and Setting Denmark and Sweden, October 2006 to December 2010. paralysis (0.56, 0.35 to 0.90). There was no association between

Participants 997 585 girls aged 10-17, among whom 296 826 received exposure to qHPV vaccine and venous thromboembolism (0.86, 0.55 on 11 March 2019 by guest. Protected copyright. a total of 696 420 qHPV vaccine doses. to 1.36). Main outcome measures Incident hospital diagnosed autoimmune, Conclusions This large cohort study found no evidence supporting neurological, and venous thromboembolic events (53 different outcomes) associations between exposure to qHPV vaccine and autoimmune, up to 180 days after each qHPV vaccine dose. Only events with at least neurological, and venous thromboembolic adverse events. Although five vaccine exposed cases were considered for further assessment. associations for three autoimmune events were initially observed, on Rate ratios adjusted for age, country, calendar year, and parental country further assessment these were weak and not temporally related to of birth, education, and socioeconomic status were estimated, comparing vaccine exposure. Furthermore, the findings need to be interpreted vaccinated and unvaccinated person time. For outcomes where the rate considering the multiple outcomes assessed. ratio was significantly increased, we regarded three criteria as signal strengthening: analysis based on 20 or more vaccine exposed cases Introduction (reliability), rate ratio 3.0 or more (strength), and significantly increased Since the regulatory approval of first the quadrivalent human rate ratio in country specific analyses (consistency). We additionally papillomavirus (qHPV) vaccine in 2006 and later the bivalent assessed clustering of events in time and estimated rate ratios for a risk HPV vaccine, as of 2011 about 120 million doses have been period that started on day 181. distributed worldwide.1 The introduction of a new vaccine Results Among the 53 outcomes, at least five vaccine exposed cases invariably puts focus not only on its effectiveness but also on occurred in 29 and these were analysed further. Whereas the rate ratios its safety. From experience we know that adverse events, with for 20 of 23 autoimmune events were not significantly increased, onset shortly after the receipt of a vaccine, especially if these exposure to qHPV vaccine was significantly associated with Behcet’s events are serious (for example, chronic immune mediated and syndrome, Raynaud’s disease, and type 1 diabetes. Each of these three neurological diseases), tend to be attributed to this exposure by outcomes fulfilled only one of three predefined signal strengthening pure temporal association.2 3 The likelihood of adverse events

Correspondence to: L Arnheim-Dahlström [email protected] Extra material supplied by the author (see http://www.bmj.com/content/347/bmj.f5906?tab=related#webextra) Supplementary tables 1 to 4

No commercial reuse: See rights and reprints http://www.bmj.com/permissions Subscribe: http://www.bmj.com/subscribe BMJ 2013;347:f5906 doi: 10.1136/bmj.f5906 (Published 9 October 2013) Page 2 of 11

RESEARCH

occurring in temporal association with exposure to vaccine will Sweden, between 1 October 2006 and 31 December 2010. Every increase proportionally with vaccine coverage as a new vaccine resident in both Denmark and Sweden has a unique personal is introduced. Concern about vaccine related adverse events has identification number enabling individual level linkage between BMJ: first published as 10.1136/bmj.f5906 on 9 October 2013. Downloaded from been identified as an important barrier to vaccination and one multiple registers.18 To define the study cohort in Sweden, we of the reasons for low HPV vaccine uptake in some settings.4 5 obtained information on birth date and date of death from the Concerns about autoimmune and neurological conditions being total population register and death register, respectively, from triggered by HPV vaccination may be fueled further by findings Statistics Sweden. To define the study cohort in Denmark, we related to other vaccines, such as the reported association used the Civil Registration System, which contains daily updated between adjuvanted influenza A(H1N1)pdm09 vaccine and information on vital and demographic variables, such as birth narcolepsy in Sweden and Finland6 7 as well as a small but date and place, and loss to follow-up due to emigration or significantly increased risk of Guillain-Barré syndrome after disappearance from the registers, and death.19 influenza A(H1N1)pdm09 vaccination, according to a recent meta-analysis.8 Several potential mechanisms by which vaccines Vaccination could induce or stimulate autoimmunity have been suggested, The qHPV vaccine (Gardasil; Sanofi Pasteur MSD SNC, Lyon, including molecular mimicry and bystander activation.2 France (in the United States: Merck, Whitehouse Station, NJ)) Consequently, to acquire the best possible evidence of safety, was marketed in Europe on 20 September 2006. In Sweden, the adverse events of newly introduced vaccines such as the HPV qHPV vaccine has been subsidised for 13-17 year old adolescent vaccines need to be monitored continuously. girls since May 2007 (inclusion of HPV vaccination in the Most adverse events occurring after HPV vaccination of national vaccination programme for 10-12 year old girls was adolescent girls have been mild and temporary, such as fever, initially planned for January 2010 but was postponed to January 9 10 headache, and injection site reactions. Pooled analyses of 2012 and then coupled with catch-up vaccination of 13-17 year clinical trials have involved almost 12 000 participants exposed old adolescent girls). In Denmark, the qHPV vaccine has been 11 to the qHPV vaccine and more than 16 000 participants included in the national vaccination programme since January 12 exposed to the bivalent HPV vaccine. Although these studies 2009 for 12 year old girls, with catch-up vaccination of 13-15 did not identify an increased risk of chronic or autoimmune year olds from October 2008. diseases overall, they were not large enough to study individual We obtained information on exposure to HPV vaccine in conditions. An analysis of 12 424 reports to the Vaccine Adverse Denmark from the childhood vaccination database at Statens Event Reporting System, among which 772 described serious 20 Serum Institut. This database is continually updated from events, identified disproportionate reporting of syncope and National Health Insurance data obtained from the National Board venous thromboembolic events but not other events, such as of Health. General practitioners in Denmark carry out all autoimmune conditions.13 However, analysis of data reported immunisations in the childhood vaccination programme and are to passive surveillance can only identify potential risk signals http://www.bmj.com/ reimbursed for reporting each instance to the National Health and can neither estimate the risk relative to an unexposed Insurance. Reimbursement takes place only after the general population nor exclude risks with certainty. Sequential analyses practitioner has submitted a form that details the vaccine, the of observational data within seven date it was administered, and the personal identification number managed care organisations in the United States (600 558 qHPV of the recipient. Therefore the database is thought to be close vaccine doses) found no increased risk of eight prespecified to complete for vaccines administered through the national outcomes, although a non-significantly increased relative risk 14 programme. Because HPV vaccination was also available

of venous thromboembolism was observed. A cohort study of on 11 March 2019 by guest. Protected copyright. outside the national programme in the study period, we retrieved 189 629 women in two managed care organisations in California additional vaccination data from the national prescription explored the risk of 16 autoimmune events and found a register, which contains individual level information on all significantly increased rate ratio for Hashimoto’s thyroiditis, prescriptions filled at all Danish pharmacies. This includes the although further assessment revealed no temporal relation or personal identification number of the recipient, the date the biological plausibility to support a true association.15 In this prescription was dispensed, and the Anatomic Therapeutic same cohort, no new safety concerns emerged when the risk of 21 Chemical (ATC) code. The ATC code used to identify the visits to an emergency department or admissions to hospital qHPV vaccine was J07BM01. In Sweden, correspondingly, we were evaluated for a wide range of health outcomes.16 obtained information on vaccination status from Svevac and Sweden and Denmark keep population based healthcare registers the drug prescription register. Svevac is a national HPV and thereby have unique opportunities to address the safety of vaccination register, established in 2006 and held by the Swedish HPV vaccination. In Denmark, we have previously described Institute for Communicable Disease. Healthcare staff who the incidence rates of anticipated immune mediated adverse administer the vaccines report to Svevac on a voluntary basis, events among adolescent girls in the period before the and the register has an estimated completeness of about 80%.22 17 introduction of HPV vaccination. In the present study we The drug prescription register contains all prescription drugs identified potential safety signals after the introduction of qHPV dispensed at pharmacies in Sweden since 1 July 2005.23 vaccination in Denmark and Sweden by comparing incidence Adolescent girls aged between 13 and 17 years received rates of several autoimmune, neurological, and venous subsidised HPV vaccination and vaccination had to be prescribed thromboembolic adverse events between adolescent girls and expedited at a pharmacy, thereby generating a register entry exposed and not exposed to the qHPV vaccine. in the drug prescription register. The register is held by the National Board of Health and Welfare. For adolescent girls aged Methods 13-17 years it is assumed that almost 100% of administered HPV vaccine doses are registered in the drug prescription Study population register. This register based cohort study of serious adverse events associated with the qHPV vaccine was based on individual level data from all 10 to 17 year old adolescent girls in Denmark and

No commercial reuse: See rights and reprints http://www.bmj.com/permissions Subscribe: http://www.bmj.com/subscribe BMJ 2013;347:f5906 doi: 10.1136/bmj.f5906 (Published 9 October 2013) Page 3 of 11

RESEARCH

Outcomes vaccination, the median time between first symptoms and 15 26 We identified data on predefined adverse events from the diagnosis was 23 days (interquartile range 2-59 days). For national patient registers in both countries using ICD-10 codes venous thromboembolism, the period at risk was 90 days after BMJ: first published as 10.1136/bmj.f5906 on 9 October 2013. Downloaded from (international classification of diseases, 10th revision). The vaccination. This was regarded as the maximal period where patient registers include nationwide individual level information an acute event could be plausibly related to vaccination; on dates of hospital contact and doctor assigned diagnoses furthermore, the mean time between vaccination and diagnosis 24 of thromboembolism among cases reported to the Vaccine according to the international classification of diseases. We 13 did not have information on outcomes from primary healthcare. Adverse Event Reporting System was 42 days. The Danish patient register was established in 1977, has included Because we acquired data on vaccine exposure from two sources both inpatients and outpatients since 1994, and has been using that were partly overlapping—that is, some of the girls had both ICD-10 codes since 1995, whereas the Swedish patient register filled prescriptions and were registered in one of the vaccination has had nationwide coverage since 1987, has included both databases—we applied an algorithm to harmonise the data and inpatients and outpatients since 2001, and has used ICD-10 define dates of vaccination. Essentially this algorithm removed codes since 1997.25 We predefined several serious adverse double data entries and doses appearing beyond the three dose outcome events, as identified from records of inpatient schedule. Furthermore, on the basis of data from girls with both admissions and hospital outpatient and emergency department filled prescriptions and register entries in vaccination databases, visits, based on our earlier study of autoimmune events,17 and it was established that the median lag between the dispensing we added several neurological events. We also included venous of a prescription and the date of vaccination as registered in the thromboembolism because it represents a potential adverse vaccination databases was two days. Consequently, for vaccine event.13 14 The included outcomes are all well defined diseases. doses that were defined by prescriptions alone, the date of In total, we assessed 53 outcomes (see supplementary table 1 exposure was defined as the date of filling the prescription plus for all included outcome events, with ICD-10 codes). two days. As the recommended qHPV vaccine schedule includes three Covariate information doses given at 0, 2, and 6 months, any girl could contribute up From Statistics Denmark and Statistics Sweden we obtained to three doses in the analyses; we counted exposed person time data on parental educational level, country of birth, and from the date the vaccine was administered, and each dose socioeconomic status. We identified the parents from the Danish contributed up to 180 days (autoimmune and neurological Civil Registration System and Swedish multigeneration register. events) of follow-up or up to 90 days (venous thromboembolism) of follow-up (fig 1⇓). We used SAS Statistical analyses statistical software version 9.3 (SAS Institute, Cary, NC).

Adolescent girls were followed from age 10 years or 1 October Analytical strategy http://www.bmj.com/ 2006, whichever came latest, and until either the occurrence of an adverse event, receipt of bivalent HPV vaccine (Cervarix, Given that a large number of serious adverse events were GlaxoSmithKline Biologicals; Rixensart, Belgium; ATC code assessed and consequently there was a high probability of chance J07BM02), death, disappearance from the registers, emigration findings, we used the following predefined criteria for the (this information was only available for Denmark), 18th analysis of data. As the first criterion, and in the interest of birthday, or end of follow-up (31 December 2010). We obtaining relatively reliable rate ratios, for any further assessment to take place we considered only outcomes with at

aggregated the resulting person years of follow-up with counts on 11 March 2019 by guest. Protected copyright. of outcome events according to qHPV vaccine exposure status least five vaccine exposed cases during the predefined period and analysed these using Poisson regression (log-linear at risk. To be regarded as a safety signal, the rate ratio for an regression of the counts using the logarithm of follow-up time outcome with at least five vaccine exposed cases had to be as offset). This produced incidence rate ratios according to significantly increased (lower bound of 95% confidence interval qHPV exposure status. Exposure to the qHPV vaccine was a >1.0). We regarded three criteria as signal strengthening: time varying variable; thus adolescent girls could contribute analysis based on 20 or more vaccine exposed cases (reliability person time to the study first as unvaccinated and later as of analysis); a rate ratio of 3.0 or more (strength of association); vaccinated, but once vaccinated the girls could not be put into and significantly increased rate ratios in both countries when the unvaccinated category again. All individual outcomes were analysed separately (consistency). For outcomes where the rate treated as separate analyses, and for each specific analysis girls ratio was significantly increased, we additionally assessed were eligible only if free from the outcome event before entry clustering of events in time by plotting cases according to time to the cohort. Estimates were adjusted for country, age in two since exposure to the vaccine and estimated rate ratios for a risk year categories, calendar year, parental educational level (highest period that started on day 181. attained level of either parent classified as: primary school (nine years) or shorter; secondary school (12 years); short tertiary Results education; and medium or long tertiary education), parental Cohort country of birth (categories: both parents, one parent, or no parent born in Scandinavia), and paternal socioeconomic status The study cohort included 997 585 girls of whom 296 826 (categories: employment with basic, unknown, or no (29.8%) received at least one dose of the qHPV vaccine (table qualification; employment with medium level or high level 1⇓). Among the vaccinated girls, 238 608 (80.4% of vaccinated qualifications; self employed; and not in labour market). girls and 23.9% of total study cohort) received the second dose For all autoimmune and neurological outcomes, we defined the and 160 986 (54.2% of vaccinated girls and 16.1% of total study period at risk as 180 days after exposure to vaccine. This period cohort) received the third. Overall, 696 420 qHPV vaccine doses was chosen to allow for the insidious onset of the diseases were administered. During follow-up, 1322 girls received the studied and because diagnostic investigations may take time; bivalent HPV vaccine and hence were censored. in a recent study of autoimmune outcomes after qHPV

No commercial reuse: See rights and reprints http://www.bmj.com/permissions Subscribe: http://www.bmj.com/subscribe BMJ 2013;347:f5906 doi: 10.1136/bmj.f5906 (Published 9 October 2013) Page 4 of 11

RESEARCH

Outcome events Among 40 predefined autoimmune outcomes, five or more Of the 53 assessed outcomes, 29 fulfilled the criterion for further vaccine exposed cases occurred in 23, and these were analysed analysis (≥5 vaccine exposed cases within the predefined risk further. Exposure to qHPV vaccine was not significantly BMJ: first published as 10.1136/bmj.f5906 on 9 October 2013. Downloaded from periods after vaccination) whereas 24 did not (see supplementary associated with 20 of these autoimmune outcome events. table 2). Table 2⇓ and figure 2⇓ show crude incidence rates and However, the rate ratios were significantly increased for adjusted rate ratios, respectively, according to exposure status Behcet’s syndrome, Raynaud’s disease, and type 1 diabetes. of the qHPV vaccine for the 29 analysed outcomes. According to the predefined analytical strategy, we assessed the strength of the observed safety signal for these outcome Autoimmune events events; for each of the three outcomes, only one of three predefined signal strengthening criteria was met. Furthermore, The rate ratios for 20 of the 23 analysed autoimmune outcomes the rate ratios for the association between exposure to qHPV were not significantly increased. Exposure to qHPV vaccine vaccine and each of these three outcome events in the period was significantly associated with Behcet’s syndrome (rate ratio starting on day 181 after vaccination and beyond were similar 3.37, 95% confidence interval 1.05 to10.80), Raynaud’s disease to the rate ratios in the primary period at risk (180 days). (1.67, 1.14 to 2.44), and type 1 diabetes (1.29, 1.03 to 1.62). Additionally, on visual inspection, the distribution of cases exhibited a random pattern. The initially observed associations Neurological events thus need to be interpreted with several factors considered: the The rate ratios were not significantly increased for any of the overall number of outcomes included in the study and hence five analysed neurological outcomes. For two of these outcomes, the possibility of chance findings; the relative weakness of these epilepsy and paralysis, the rate ratios were significantly signals, as per our evaluation of predefined signal strengthening decreased. criteria; and the biological implausibility of events occurring without a clear temporal pattern in relation to the exposure. On Venous thromboembolism this basis, although significantly increased rate ratios were initially observed for three outcomes, it can be concluded that The rate ratio for the association between exposure to qHPV after further assessment no consistent evidence for a causal vaccine and venous thromboembolism was 0.86 (0.55 to 1.36). association was found.

Evaluation of safety signals Relation to other studies Each of the three autoimmune outcome events where the rate These findings corroborate those from a cohort study of 189 ratios were significantly increased was assessed using the 629 women in two managed care organisations in California, predefined signal strengthening criteria. For each of these three which found no safety signal when investigating the risk of 16 outcomes, one of the three signal strengthening criteria was autoimmune events.15 That study did find an inverse association http://www.bmj.com/ fulfilled (table 3⇓). For Raynaud’s disease and type 1 diabetes, between exposure to qHPV vaccine and type 1 diabetes (rate the fulfilled criterion was that the analysis was based on 20 or ratio 0.57, 95% confidence interval 0.47 to 0.73), lending further more vaccine exposed cases. For Behcet’s syndrome, the support to the conclusion that the initial signal for type 1 criterion was that the rate ratio was 3.0 or above. diabetes observed in our study might be a false positive. Subsequently, for each of the three outcome events with a Of the 12 neurological outcomes assessed in our study, five significantly increased rate ratio within 180 days after qHPV fulfilled the criterion for further analyses. The rate ratio was not vaccination, we estimated rate ratios for a later period. Starting significantly increased for any of these five outcomes, and both on 11 March 2019 by guest. Protected copyright. from day 181 after vaccination and onwards, the rate ratios were epilepsy and paralysis were inversely associated with exposure 3.18 (95% confidence interval 0.83 to 12.19) for Behcet’s to qHPV vaccine. Not at least given that the Vaccine Safety syndrome, 1.50 (0.95 to 2.37) for Raynaud’s disease, and 1.18 Datalink study of qHPV vaccine related adverse events reported (0.91 to 1.55) for type 1 diabetes (see supplementary table 3). a relative risk of 1.02 for seizures,14 the observed inverse Visual inspection of the temporal distribution of cases revealed association for this outcome in our study is most likely due to no distinct pattern in time for Raynaud’s disease and type 1 chance. This underlines the need to interpret the findings from diabetes (fig 3⇓). For Behcet’s syndrome, this analysis was our study taking into account the large number of comparisons inconclusive owing to the low number of cases. made and the need for ongoing monitoring in independent populations. Sensitivity analysis Analysis of data reported to the Vaccine Adverse Event Because of the possible delay between disease onset and Reporting System revealed disproportionate reporting of venous 13 diagnosis, the selected risk period of 180 days might not have thromboembolism. A study by the Vaccine Safety Datalink, captured narcolepsy events adequately. Therefore we did a which involved eight outcomes, identified a non-significantly sensitivity analysis with the risk period starting from day 181; increased relative risk (1.98) of venous thromboembolism; the rate ratio was 0.64 (95% confidence interval 0.26 to 1.57; medical record review could confirm five of the eight cases see supplementary table 4). identified from databases using international classification of diseases codes, and all five had known risk factors for venous 14 Discussion thromboembolism. In our analysis, based on 21 vaccine exposed cases, there was no significant association with venous This population based study included all adolescent girls aged thromboembolism within 90 days after exposure to qHPV between 10 and 17 years in Denmark and Sweden who received vaccine. These results are corroborated by a study in two quadrivalent human papillomavirus (qHPV) vaccine in the first managed care organisations in California that did not find four years after its licensure. Overall, the findings of this study, evidence of an association between qHPV vaccination and which were based on nearly one million girls and 700 000 venous thromboembolism when assessing all events in vaccine doses, were reassuring for autoimmune, neurological, emergency department and admissions to hospital using and venous thromboembolic events after qHPV vaccination. international classification of diseases codes.16

No commercial reuse: See rights and reprints http://www.bmj.com/permissions Subscribe: http://www.bmj.com/subscribe BMJ 2013;347:f5906 doi: 10.1136/bmj.f5906 (Published 9 October 2013) Page 5 of 11

RESEARCH

Strengths and weaknesses of this study provides an opportunity to evaluate symptoms that may not Our study has strengths and limitations. We analysed nationwide have been evaluated otherwise and, hence, that those vaccinated data from Denmark and Sweden, which permitted a large sample may be more likely to have certain disorders diagnosed; this BMJ: first published as 10.1136/bmj.f5906 on 9 October 2013. Downloaded from size; this was advantageous because many of the outcomes would bias results towards increased risk attributed to studied are rare and therefore not possible to study in a clinical vaccination. This phenomenon is unlikely for diseases with trial. Furthermore, the use of individually linked data allowed relatively prominent and well recognised symptoms, such as the identification of truly incident cases. Additionally, the type 1 diabetes, but is plausible for diseases that may present inclusion of entire populations in the cohort allowed the rates with obscure symptoms or that initially may be interpreted by among those exposed to vaccine to be compared directly with the patient as normal variation, such as Raynaud’s disease. the true national background rates.27 Although Denmark and Although the results from this study are possibly generalisable Sweden are alike in many respects, including having similar to comparable populations that are of similar age, they cannot healthcare systems and universal access to healthcare, be directly extrapolated to adults. Furthermore, the results should differences in diagnostic coding practices might have influenced not be inferred to the bivalent HPV vaccine, because the the ascertainment of cases. This is, however, unlikely for constituents of the vaccine differ. relatively well defined diseases. Our case definition was based on hospital diagnoses, which likely captured the majority of Conclusion cases because most adolescent girls with the severe conditions This cohort study of about one million adolescent girls aged 10 studied are under specialised paediatric care (this type of care to 17 years utilised routine healthcare data from two is only available from hospitals in Denmark and Sweden), at Scandinavian countries to identify potential serious adverse the very least during the diagnostic phase. events during the first four years after the qHPV vaccine was Dates of onset of symptoms or disease were not available for marketed. While the study expands on the current safety this study, which instead relied on dates of diagnoses to define information of the qHPV vaccine by systematically assessing the index dates of outcome events. It is therefore possible that a range of serious adverse outcomes, the results need to be a proportion of events attributed to vaccine exposure in the interpreted cautiously considering the large number of statistical analyses had symptom onset before the vaccine was tests performed; as well as the chance of false positive findings, administered, and similarly, that a proportion of events had true associations may have been missed. Although significantly symptom onset within the risk period but were diagnosed later increased rate ratios were initially observed for three outcomes, on and were thus not attributed to vaccine exposure in the further assessment showed no consistent evidence for a plausible analyses. To accommodate the time delay between first symptom association; firstly, these risk signals were relatively weak, as and diagnosis we used a 180 day risk period for autoimmune assessed by prespecified criteria, and, secondly, no temporal and neurological outcomes, which is longer than in some studies relation between vaccine exposure and outcome was evident. (for example, studies of Guillain-Barré syndrome after Thus, this study identified no safety signals with respect to http://www.bmj.com/ vaccination typically have a risk window of 42 days), but autoimmune, neurological, and venous thromboembolic events consistent with other studies of autoimmune disease after HPV after the qHPV vaccine had been administered. Nevertheless, vaccination.15 However, it may have been too short to capture these findings need to be confirmed in studies with longer diseases with a more insidious onset. Because the latter may be follow-up time, validation of outcomes, and data on time of the case for narcolepsy, we conducted a sensitivity analysis with onset of disease. Further monitoring of HPV vaccine safety is a risk window starting on day 181 after vaccination; no warranted in other populations when use and coverage has significant association was observed with narcolepsy. increased. on 11 March 2019 by guest. Protected copyright. The vaccine coverage was 49% in Denmark and 18% in Sweden. This difference between the countries may reflect the fact that We thank Jonas Hällgren for data administration, Karin Sundström for during the study period the qHPV vaccine had been introduced valuable discussion on the findings in this study, and the Swedish in the national childhood vaccination programme in Denmark Institute for Communicable Diseases for contributing with HPV but not in Sweden. The statistical models included adjustment vaccination data from the Svevac register. for age, country, calendar year, and parental educational level, Contributors: BP, HS, and AH created the database and performed the country of birth, and socioeconomic status. However, we could statistical analyses. LAD and BP drafted the manuscript. All authors not obtain information on other potential risk factors such as actively participated in study design, interpretation and discussion of smoking as this information is not recorded in any nationwide the results, revision of the manuscript, and approval of the final version register. This may represent a source of residual confounding. of the manuscript. AH is the guarantor. In particular, given an overall 30% vaccine coverage, adolescent Funding: This study was supported by a grant from the Swedish girls who did receive the vaccine may be selected individuals Foundation for Strategic Research and the Danish Medical Research and could be at differential risk of certain outcomes or be more Council. The funding bodies had no role in the study design; the or less likely to consult healthcare for health problems. Self collection, analysis, and interpretation of the data; the writing of the controlled case series analysis28 could have tackled some of the article; and the decision to submit it for publication. All authors are potential problems from confounding. However, as the aim of independent from the funding agencies. this study was signal detection and strengthening and the method Competing interests: All authors have completed the ICMJE uniform chosen is computationally efficient for analysing a large number disclosure form at www.icmje.org/coi_disclosure.pdf (available on of outcomes, the self controlled case series method is data request from the corresponding author) and declare: no support from intensive and would be cumbersome when studying a large any organisation for the submitted work; LAD and PS are and have number of outcomes. Furthermore, the self controlled case series been involved in other studies with unconditional grants from becomes problematic when onset of disease is not known to GlaxoSmithKline, Sanofi Pasteur MSD, and Merck; and no other any great extent (which is the case, for example, for narcolepsy, relationships or activities that could appear to have influenced the which normally has a long incubation period). submitted work. An unmasking phenomenon has been described in vaccine safety research.26 This refers to the fact that the vaccination visit No commercial reuse: See rights and reprints http://www.bmj.com/permissions Subscribe: http://www.bmj.com/subscribe BMJ 2013;347:f5906 doi: 10.1136/bmj.f5906 (Published 9 October 2013) Page 6 of 11

RESEARCH

What is already known on this topic

Vaccines against human papillomavirus (HPV) have been available since 2006 BMJ: first published as 10.1136/bmj.f5906 on 9 October 2013. Downloaded from Clinical trials and post-licensure studies from the United States have not identified any increase in the risk of serious adverse events after vaccination

What this study adds

This European cohort study found no evidence supporting associations between exposure to qHPV vaccine and autoimmune, neurological, and venous thromboembolic adverse events in almost one million adolescent girls

Ethical approval: This study was approved by the regional ethical review 13 Slade BA, Leidel L, Vellozzi C, Woo EJ, Hua W, Sutherland A, et al. Postlicensure safety surveillance for quadrivalent human papillomavirus recombinant vaccine. JAMA committee in Stockholm, Sweden, and by the Danish Data Protection 2009;302:750-7. Agency. Ethical approval is not required for register based research in 14 Gee J, Naleway A, Shui I, Baggs J, Yin R, Li R, et al. Monitoring the safety of quadrivalent human papillomavirus vaccine: findings from the Vaccine Safety Datalink. Vaccine Denmark. 2011;29:8279-84. Declaration of transparency: The lead author affirms that this manuscript 15 Chao C, Klein NP, Velicer CM, Sy LS, Slezak JM, Takhar H, et al. Surveillance of autoimmune conditions following routine use of quadrivalent human papillomavirus vaccine. is an honest, accurate, and transparent account of the study being J Intern Med 2012;271:193-203. reported; that no important aspects of the study have been omitted; and 16 Klein NP, Hansen J, Chao C, Velicer C, Emery M, Slezak J, et al. Safety of quadrivalent human papillomavirus vaccine administered routinely to females. Arch Pediatr Adolesc that any discrepancies from the study as planned (and, if relevant, Med 2012:166:1140-8. registered) have been explained. 17 Callreus T, Svanstrom H, Nielsen NM, Poulsen S, Valentiner-Branth P, Hviid A. Human papillomavirus immunisation of adolescent girls and anticipated reporting of Data sharing: No additional data available. immune-mediated adverse events. Vaccine 2009;27:2954-8. 18 Ludvigsson JF, Otterblad-Olausson P, Pettersson BU, Ekbom A. The Swedish personal identity number: possibilities and pitfalls in healthcare and medical research. Eur J 1 Medical Products Agency. Vaccination against human papillomavirus (HPV) vaccine Epidemiol 2009;24:659-67. Gardasil and Cervarix. 2012. [In Swedish.] www.lakemedelsverket.se/OVRIGA-SIDOR/ 19 Pedersen CB, Gotzsche H, Moller JO, Mortensen PB. The Danish Civil Registration HPV-vaccinering/. System. A cohort of eight million persons. Dan Med Bull 2006;53:441-9. 2 Wraith DC, Goldman M, Lambert PH. Vaccination and autoimmune disease: what is the 20 Hviid A. Postlicensure epidemiology of childhood vaccination: the Danish experience. evidence? Lancet 2003;362:1659-66. Expert Rev Vaccines 2006;5:641-9. 3 Siegrist CA. Autoimmune diseases after adolescent or adult immunization: what should 21 Kildemoes HW, Sorensen HT, Hallas J. The Danish National Prescription Registry. Scand we expect? CMAJ 2007;177:1352-4. J Public Health 2011;39:38-41. 4 Trim K, Nagji N, Elit L, Roy K. Parental knowledge, attitudes, and behaviours towards 22 Leval A, Herweijer E, Ploner A, Eloranta S, Fridman Simard S, Dillner J,et al. Quadrivalent human papillomavirus vaccination for their children: a systematic review from 2001 to HPV-vaccine effectiveness on genital warts: a population-based study in over 2.2 million 2011. Obstet Gynecol Int 2012;2012:921236. women including dose evaluation. J Natl Cancer Inst 2013 (in press). 5 Laz TH, Rahman M, Berenson AB. An update on human papillomavirus vaccine uptake 23 Wettermark B, Hammar N, Fored CM, Leimanis A, Otterblad Olausson P, Bergman U, et among 11-17 year old girls in the United States: national health interview survey, 2010. al. The new Swedish Prescribed Drug Register—opportunities for pharmacoepidemiological Vaccine 2012;30:3534-40. research and experience from the first six months. Pharmacoepidemiol Drug Safety 6 Nohynek H, Jokinen J, Partinen M, Vaarala O, Kirjavainen T, Sundman J, et al. AS03 2007;16:726-35. adjuvanted AH1N1 vaccine associated with an abrupt increase in the incidence of childhood

24 Lynge E, Sandegaard JL, Rebolj M. The Danish National Patient Register. Scand J Public http://www.bmj.com/ narcolepsy in Finland. PLoS One 2012;7:e33536. Health 2011;39:30-3. 7 Medical Products Agency. Occurrence of narcolepsy with cataplexy among children and 25 Ludvigsson JF, Andersson E, Ekbom A, Feychting M, Kim JL, Reuterwall C, et al. External adolescents in relation to the H1N1 pandemic and Pandemrix vaccinations. Secondary review and validation of the Swedish national inpatient register. BMC Public Health occurrence of narcolepsy with cataplexy among children and adolescents in relation to 2011;11:450. the H1N1 pandemic and Pandemrix vaccinations 2011. www.lakemedelsverket.se/upload/ 26 Chao C, Jacobsen SJ. Evaluation of autoimmune safety signal in observational vaccine nyheter/2011/Fallinventeringsrapport_pandermrix_110630.pdf. safety studies. Hum Vaccin Immunother 2012;8:1302-4. 8 Salmon DA, Proschan M, Forshee R, Gargiullo P, Bleser W, Burwen DR, et al. Association 27 Rasmussen TA, Jorgensen MR, Bjerrum S, Jensen-Fangel S, Stovring H, Ostergaard L, between Guillain-Barre syndrome and influenza A (H1N1) 2009 monovalent inactivated et al. Use of population based background rates of disease to assess vaccine safety in vaccines in the USA: a meta-analysis. Lancet 2013;381:1461-8. childhood and mass immunisation in Denmark: nationwide population based cohort study. 9 Lu B, Kumar A, Castellsague X, Giuliano AR. Efficacy and safety of prophylactic vaccines BMJ 2012;345:e5823.

against cervical HPV infection and diseases among women: a systematic review and on 11 March 2019 by guest. Protected copyright. 28 Farrington CP, Nash J, Miller E. Case series analysis of adverse reactions to vaccines: meta-analysis. BMC Infect Dis 2011;11:13. a comparative evaluation. Am J Epidemiol 1996;143:1165-73. 10 Van Klooster TM, Kemmeren JM, van der Maas NA, de Melker HE. Reported adverse events in girls aged 13-16 years after vaccination with the human papillomavirus Accepted: 28 August 2013 (HPV)-16/18 vaccine in the Netherlands. Vaccine 2011;29:4601-7. 11 Block SL, Brown DR, Chatterjee A, Gold MA, Sings HL, Meibohm A, et al. Clinical trial and post-licensure safety profile of a prophylactic human papillomavirus (types 6, 11, 16, Cite this as: BMJ 2013;347:f5906 and 18) l1 virus-like particle vaccine. Pediatr Infect Disease J 2010;29:95-101. 12 Descamps D, Hardt K, Spiessens B, Izurieta P, Verstraeten T, Breuer T, et al. Safety of This is an Open Access article distributed in accordance with the Creative Commons human papillomavirus (HPV)-16/18 AS04-adjuvanted vaccine for cervical cancer Attribution Non Commercial (CC BY-NC 3.0) license, which permits others to distribute, prevention: a pooled analysis of 11 clinical trials. Hum Vaccin 2009;5:332-40. remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/3.0/.

No commercial reuse: See rights and reprints http://www.bmj.com/permissions Subscribe: http://www.bmj.com/subscribe BMJ 2013;347:f5906 doi: 10.1136/bmj.f5906 (Published 9 October 2013) Page 7 of 11

RESEARCH

Tables BMJ: first published as 10.1136/bmj.f5906 on 9 October 2013. Downloaded from

Table 1| Descriptive characteristics of adolescent girls aged 10-17 years included in cohort, Denmark and Sweden, October 2006-December 2010. Values are numbers (percentages) unless stated otherwise

Characteristics Overall (n=997 585) Denmark (n=387 294) Sweden (n=610 291)

Person years of follow-up 2 797 701 1 090 515 1 707 186 Mean (SD) age at study entry (years) 12.8 (2.7) 12.5 (2.6) 12.9 (2.7) Year of study entry: 2006 700 156 (70.2) 260 849 (67.4) 439 307 (72.0) 2007 74 809 (7.5) 32 044 (8.3) 42 765 (7.0) 2008 73 653 (7.4) 31 307 (8.1) 42 346 (6.9) 2009 73 909 (7.4) 31 439 (8.1) 42 470 (7.0) 2010 75 058 (7.5) 31 655 (8.2) 43 403 (7.1) Exposed to quadrivalent human papillomavirus vaccine

Mean (SD) age at vaccination (years) 14.6 (1.7) 14.0 (1.6) 15.7 (1.4) Total vaccine doses, No (% of total No in cohort): 696 420 409 724 286 696 Dose 1 296 826 (29.8) 188 053 (48.6) 108 773 (17.8) Dose 2 238 608 (23.9) 139 861 (36.1) 98 747 (16.2) Dose 3 160 986 (16.1) 81 810 (21.1) 79 176 (13.0) Year of first vaccine dose, No (% of vaccinated): 2006 426 (0.1) 248 (0.1) 178 (0.2) 2007 22 943 (7.7) 6280 (3.3) 16 663 (15.3) 2008 41 799 (14.1) 12 314 (6.5) 29 485 (27.1) 2009 170 830 (57.6) 133 571 (71.0) 37 259 (34.3) 2010 60 828 (20.5) 35 640 (19.0) 25 188 (23.2)

Data from vaccination registers 572 696 (82.2) 351 804 (85.9) 220 892 (77.0) http://www.bmj.com/ Data from prescription registers 123 724 (17.8) 57 920 (14.1) 65 804 (23.0)

Because of rounding, percentages may not total 100. on 11 March 2019 by guest. Protected copyright.

No commercial reuse: See rights and reprints http://www.bmj.com/permissions Subscribe: http://www.bmj.com/subscribe BMJ 2013;347:f5906 doi: 10.1136/bmj.f5906 (Published 9 October 2013) Page 8 of 11

RESEARCH

Table 2| Rates of adverse events according to quadrivalent human papillomavirus (qHPV) vaccination status, cohort of adolescent girls aged 10-17 years in Denmark and Sweden, October 2006-December 2010 BMJ: first published as 10.1136/bmj.f5906 on 9 October 2013. Downloaded from Unvaccinated Within 180 days after qHPV vaccine exposure Adverse events Person years No of events Incidence rate* (95% CI) Person years No of events Incidence rate* (95% CI)

Autoimmune Thyroid: Graves’ disease 2 373 554 237 9.99 (8.79 to 11.34) 229 914 27 11.74 (8.05 to 17.12) Hashimoto’s thyroiditis 2 371 866 560 23.61 (21.73 to 25.65) 229 751 50 21.76 (16.49 to 28.71) Other hyperthyroidism 2 373 629 250 10.53 (9.30 to 11.92) 229 946 23 10.00 (6.65 to 15.05) Hypothyroidism 2 368 919 1018 42.97 (40.41 to 45.70) 229 563 79 34.41 (27.60 to 42.90) Gastrointestinal: Coeliac disease 2 358 918 1413 59.90 (56.86 to 63.11) 228 820 107 46.76 (38.69 to 56.52) Crohn’s disease 2 372 337 539 22.72 (20.88 to 24.72) 229 825 47 20.45 (15.37 to 27.22) Ulcerative colitis 2 373 288 350 14.75 (13.28 to 16.38) 229 889 35 15.22 (10.93 to 21.20) Pancreatitis 2 374 129 103 4.34 (3.58 to 5.26) 230 004 10 4.35 (2.34 to 8.08) Muscoloskeletal or systemic: Ankylosing spondylitis 2 374 065 93 3.92 (3.20 to 4.80) 230 001 8 3.48 (1.74 to 6.96) Behcet’s syndrome 2 374 464 13 0.55 (0.32 to 0.94) 230 025 5 2.17 (0.90 to 5.22) Henoch-Schönlein’s purpura 2 369 280 203 8.57 (7.47 to 9.83) 229 365 17 7.41 (4.61 to 11.92) Juvenile arthritis 2 366 484 861 36.38 (34.03 to 38.90) 229 202 86 37.52 (30.37 to 46.35) Myositis 2 373 974 84 3.54 (2.86 to 4.38) 229 988 8 3.48 (1.74 to 6.96) Rheumatoid arthritis 2 373 763 216 9.10 (7.96 to 10.40) 229 943 27 11.74 (8.05 to 17.12) Systemic lupus erythematosus 2 374 231 74 3.12 (2.48 to 3.91) 230 005 11 4.78 (2.65 to 8.64) Vasculitis, unspecified 2 373 826 89 3.75 (3.05 to 4.61) 229 959 14 6.09 (3.61 to 10.28) Haematological: Idiopathic thrombocytopenic 2 373 040 107 4.51 (3.73 to 5.45) 229 896 14 6.09 (3.61 to 10.28) purpura http://www.bmj.com/ Dermatological: Erythema nodosum 2 373 608 163 6.87 (5.89 to 8.01) 229 935 19 8.26 (5.27 to 12.95) Localised scleroderma 2 374 016 88 3.71 (3.01 to 4.57) 229 976 6 2.61 (1.17 to 5.81) Psoriasis 2 368 423 1091 46.06 (43.41 to 48.88) 229 540 80 34.85 (27.99 to 43.39) Vitiligo 2 372 765 310 13.06 (11.69 to 14.60) 229 886 24 10.44 (7.00 to 15.58)

Miscellaneous on 11 March 2019 by guest. Protected copyright. Raynaud’s disease 2 373 798 218 9.18 (8.04 to 10.49) 229 939 37 16.09 (11.66 to 22.21) Type 1 diabetes 2 363 153 975 41.26 (38.75 to 43.93) 228 965 99 43.24 (35.51 to 52.65) Neurological

Bell’s palsy 2 370 195 480 20.25 (18.52 to 22.15) 229 675 41 17.85 (13.14 to 24.24) Epilepsy 2 351 894 1701 72.32 (68.97 to 75.84) 227 897 116 50.90 (42.43 to 61.06) Narcolepsy 2 374 402 43 1.81 (1.34 to 2.44) 230 018 6 2.61 (1.17 to 5.81) Optical neuritis 2 374 273 61 2.57 (2.00 to 3.30) 230 013 6 2.61 (1.17 to 5.81) Paralysis 2 367 206 302 12.76 (11.40 to 14.28) 229 574 20 8.71 (5.62 to 13.50) Venous thromboembolism† 2 373 786 297 12.51 (11.17 to 14.02) 149 817 21 14.02 (9.14 to 21.50)

Table shows outcomes with five or more vaccine exposed cases. *Events per 100 000 person years. †Risk window for venous thromboembolism was within 90 days after vaccine exposure.

No commercial reuse: See rights and reprints http://www.bmj.com/permissions Subscribe: http://www.bmj.com/subscribe BMJ 2013;347:f5906 doi: 10.1136/bmj.f5906 (Published 9 October 2013) Page 9 of 11

RESEARCH

Table 3| Evaluation of signal strengthening criteria among outcomes where rate ratios were significantly increased

Criterion Behcet’s syndrome Raynaud’s disease Type 1 diabetes BMJ: first published as 10.1136/bmj.f5906 on 9 October 2013. Downloaded from

Analysis based on 20 or more vaccine No (n=5) Yes (n=37) Yes (n=99) exposed cases Rate ratio ≥3.0 Yes (3.37) No (1.67) No (1.29) Significantly increased rate ratios in No (3.38*, 95% CI 0.83 to 13.84 for No (1.86*, 95% CI 1.19 to 2.89 for No (1.47*, 95% CI 1.08 to 2.01 for both countries when analysed Sweden (3 exposed cases); 4.63†, 95% Sweden (25 exposed cases); 1.46*, 95% Sweden (47 exposed cases); 1.09*, 95% separately CI 0.64 to 33.66 for Denmark (2 exposed CI 0.64 to 3.33 for Denmark (12 exposed CI 0.76 to 1.57 for Denmark (52 exposed cases)) cases)) cases))

*Adjusted for age in two year intervals, calendar year, and parental country of birth, parental education, and paternal socioeconomic status. †Adjusted for age in two year intervals (model with full adjustment did not converge). http://www.bmj.com/ on 11 March 2019 by guest. Protected copyright.

No commercial reuse: See rights and reprints http://www.bmj.com/permissions Subscribe: http://www.bmj.com/subscribe BMJ 2013;347:f5906 doi: 10.1136/bmj.f5906 (Published 9 October 2013) Page 10 of 11

RESEARCH

Figures BMJ: first published as 10.1136/bmj.f5906 on 9 October 2013. Downloaded from

Fig 1 Periods at risk for autoimmune and neurological events in adolescent girls after exposure to quadrivalent human papillomavirus (qHPV) vaccine. For venous thromboembolism, each period at risk was up to 90 days http://www.bmj.com/ on 11 March 2019 by guest. Protected copyright.

Fig 2 Association between exposure to quadrivalent human papillomavirus (qHPV) vaccine and adverse events in adolescent girls in Denmark and Sweden, October 2006-December 2010. Rate ratios are adjusted for country, age in two year intervals, calendar year, and parental country of birth, parental education, and paternal socioeconomic status

No commercial reuse: See rights and reprints http://www.bmj.com/permissions Subscribe: http://www.bmj.com/subscribe BMJ 2013;347:f5906 doi: 10.1136/bmj.f5906 (Published 9 October 2013) Page 11 of 11

RESEARCH BMJ: first published as 10.1136/bmj.f5906 on 9 October 2013. Downloaded from http://www.bmj.com/ on 11 March 2019 by guest. Protected copyright.

Fig 3 Distribution of cases according to days since first dose of quadrivalent human papillomavirus (qHPV) vaccine. For type 1 diabetes case 1, vaccine doses 2 and 3 and event are not displayed (dose 2 was administered on day 425, dose 3 on day 609, and the event was on day 570). For type 1 diabetes case 2, event is not displayed (event was on day 460). For type 1 diabetes case 82, vaccine dose 3 is not displayed (dose was administered on day 455)

No commercial reuse: See rights and reprints http://www.bmj.com/permissions Subscribe: http://www.bmj.com/subscribe Articles

Early eff ect of the HPV vaccination programme on cervical abnormalities in Victoria, Australia: an ecological study

Julia M L Brotherton, Masha Fridman, Cathryn L May, Genevieve Chappell, A Marion Saville, Dorota M Gertig

Summary Background Australia introduced a human papillomavirus (HPV) vaccination programme with the quadrivalent HPV Lancet 2011; 377: 2085–92 vaccine for all women aged 12–26 years between 2007 and 2009. We analysed trends in cervical abnormalities in See Comment page 2057 women in Victoria, Australia, before and after introduction of the vaccination programme. Victorian Cytology Service Registries, Victorian Cytology Methods With data from the Victorian Cervical Cytology Registry between 2003 and 2009, we compared the incidence Service, East Melbourne, VIC, Australia of histopathologically defi ned high-grade cervical abnormalities (HGAs, lesions coded as cervical intraepithelial (J M L Brotherton B Med, neoplasia of grade 2 or worse or adenocarcinoma in situ; primary outcome) and low-grade cytological abnormalities C L May BAppSc, (LGAs) in fi ve age groups before (Jan 1, 2003, to March 31, 2007) and after (April 1, 2007, to Dec 31, 2009) the G Chappell BAppSc, vaccination programme began. Binary comparisons between the two periods were done with Fisher’s exact test. A M Saville MBChB, D M Gertig ScD); and Poisson piecewiseregression analysis was used to compare incident rate trends. Biosciences Research Division, Department of Primary Findings After the introduction of the vaccination programme, we recorded a decrease in the incidence of HGAs Industries, Bundoora, VIC, by 0·38% (95% CI 0·61–0·16) in girls younger than 18 years. This decrease was progressive and signifi cantly diff erent Australia (M Fridman PhD) to the linear trend in incidence before introduction of the vaccination (incident rate ratio 1·14, 1·00–1·30, p=0·05). Correspondence to: Dr Julia M L Brotherton, Victorian No similar temporal decline was recorded for LGAs or in older age groups. Cytology Service Registries, PO Box 310, East Melbourne, Interpretation This is the fi rst report of a decrease in incidence of HGAs within 3 years after the implementation of a VIC 8002, Australia population-wide HPV vaccination programme. Linkage between vaccination and screening registers is needed to [email protected] confi rm that this ecological observation is attributable to vaccination and to monitor participation in screening among vaccinated women.

Funding None.

Introduction Station, NJ, USA) in April, 2007, within the context of Since the fi rst prophylactic vaccine against human an already intensive and successful national cervical papillomavirus (HPV) was licensed in mid-2006, the screening programme. The vaccination programme quadrivalent vaccine (which provides protection against consists of a continuing component that targets high-risk HPV types 16 and 18, and low-risk types 6 and 11, 12–13-year-old girls in schools and two catch-up which cause 90% of genital warts) or bivalent vaccine programmes, one for 13–17-year-old school girls, and (targeting HPV types 16 and 18) have been implemented one for 18–26-year-old women through general practice in more than 28 countries as part of their national and community settings delivered between July, 2007, immunisation programmes and implemented at a sub- and December, 2009. In Victoria, the second most national level through donations in at least 17 developing populous Australian state, the HPV vaccine programme countries.1 Persistent infection with high-risk genital HPV in secondary schools began on April 16, 2007. Girls in types is needed for the development of cervical cancer, and school years 7 (ages 12–13 years), 10, 11, and 12 (ages HPV types 16 and 18 are detected in 70% of cervical 15–18 years) were off ered vaccination in 2007, with the cancers, half of high-grade cervical abnormalities (HGAs), remaining two catch-up cohorts (aged 13–14 and and a quarter of low-grade cervical abnormalities (LGAs) 14–15 years in 2007) off ered vaccine in 2008.4 Vaccine worldwide.2 Although the target age groups vary in coverage estimates from the National HPV Vaccination diff erent countries, the vaccine is aimed mainly at girls Program Register for the school programme in Victoria between the ages of 9 and 12 years because it is most show a three-dose coverage of 79% in fi rst-year high- eff ective when given before the onset of sexual activity, school students and 71% in fi nal-year high-school because it has no eff ect against HPV infection—which is students.4 A population-based telephone survey done in transmitted sexually—once it has been acquired. Various Victoria in early 2009 noted self-reported coverage rates countries have also chosen to implement short-term catch- of 74% for one dose, 69% for two doses, and 56% for up programmes aimed at older age groups, ranging from three doses in young women aged 18–28 years.5 These 13–18 years to 26 years.3 data indicate that the programme probably achieved Australia was the fi rst country to roll out an extensive, high coverage. funded national HPV vaccination programme with the Australia’s HPV vaccination programme includes the quadrivalent vaccine GARDASIL (Merck, Whitehouse broadest funded catch-up age range in the world3 and www.thelancet.com Vol 377 June 18, 2011 2085 Articles

overlaps with the age cohort presently eligible for cervical programme, is especially important because the eff ect screening in Australia. The National Cervical Screening on these abnormalities is more proximal than, but Program policy recommends one cervical cytology test closely related to, the development of cervical cancer, every 2 years, beginning at age 18 years (or 2 years after and the treatment of such lesions is associated with onset of sexual activity, whichever is later) until age morbidity and cost. 69 years. The National Cervical Screening Program was Here we present data from Victoria, reporting cervical established in 1991, and since that time both cervical cancer abnormality rates in young women for the fi rst 3 years incidence and mortality have halved.6 Participation rates in (2007–09) after the introduction of a widely targeted the programme are 61·2% of women every 2 years, 73·9% population-based HPV vaccination programme. every 3 years, and 86·3% every 5 years.7 Monitoring of the early eff ect of the vaccine in Australia is helped by the Methods existence of state and territory Papanicolaou (Pap) test Data collection registers that record nearly all cervical cytology and The Victorian Cervical Cytology Registry (VCCR) is one histology results and the National HPV Vaccination of eight Pap test registries in Australia and promotes Program Register, which was established to support and regular participation of women in the National Cervical monitor the HPV vaccination programme.8 Screening Program by sending reminder letters and A rapid eff ect on infection with vaccine-targeted HPV enables the follow-up of women with abnormal Pap types is predicted after the implementation of tests. In brief, follow-up of cervical abnormalities population-based HPV vaccination programmes.9 detected by screening programmes in Australia is guided Indeed, early data from sexual health clinics in Australia by national recommendations,11 with incident LGAs suggest that the incidence of genital warts in Victoria generally followed up with another smear test after began to decrease in the fi rst year of the vaccination 12 months to establish whether the abnormality has programme.10 However, because of the long lead-time resolved or whether colposcopy is needed. Patients with between infection and development of malignant HGAs or possible HGAs are immediately referred for disease, the programme’s eff ect on cancer incidence colposcopy. The VCCR compiles statistics for the purpose will take decades to assess. Hence monitoring of of monitoring and research. cervical abnormality rates in a country such as Australia, The VCCR receives timely data for almost all cervical See Online for webappendix with a longstanding high-quality cervical screening cytology and cervical histopathology taken in Victoria, with a population of more than 2·7 million girls and women. Less than 1% of women request for their test Before vaccination After vaccination Diff erence in proportions p value results not to be held on the VCCR.12 Cervical cytology (Jan 1, 2003, to (April 1, 2007, to (95% CI) results, coded by reporting laboratories with the March 31, 2007) Dec 31, 2009) Australian Standard Modifi ed Bethesda coding schedule, Number of women screened are forwarded to the VCCR. Copies of relevant <18 years 13 620 5538 NA NA histopathology results are received from reporting 18–20 years 86 356 50 644 NA NA laboratories and coded according to an in-house coding 21–25 years 237 599 152 531 NA NA schedule, with most coding checked by a second staff 26–30 years 281 767 177 776 NA NA member for quality assurance purposes. ≥31 years 1 798 842 1 178 351 NA NA De-identifi ed data were extracted from the VCCR for all LGA incidence screening-related episodes between Jan 1, 2001, and <18 years 1658 (12·2%) 691 (12·5%) 0·3% (–0·8 to 1·4) 0·6 Dec 31, 2009. To minimise the prevalent pool eff ect,13,14 18–20 years 9465 (11·0%) 5506 (10·9%) –0·1% (–0·5 to 0·3) 0·6 which would result in prevalent lesions being regarded 21–25 years 18 671 (7·9%) 11 067 (7·3%) –0·6% (–0·8 to –0·4) <0·0001 as incident because of an absence of preceding data, a 26–30 years 14 049 (5·0%) 7810 (4·4%) –0·6% (–0·7 to –0·5) <0·0001 clearance period of 2 years was applied to the data. We ≥31 years 44 408 (2·5%) 23 106 (2·0%) –0·5% (–0·47 to –0·54) <0·0001 therefore analysed LGA and HGA incidence rates HGA incidence between 2003 and 2009. <18 years 109 (0·80%) 23 (0·42%) –0·38% (–0·61 to –0·16) 0·003 The process of data exclusion from the analytical 18–20 years 1035 (1·20%) 593 (1·17%) –0·03% (–0·15 to 0·09) 0·7 dataset is shown in the webappendix (p 1). Episodes that 21–25 years 3639 (1·53%) 2609 (1·71%) 0·18% (0·10 to 0·26) <0·0001 were not related to cervical diagnoses (eg, vaginal and 26–30 years 3561 (1·26%) 2542 (1·43%) 0·17% (0·10 to 0·24) <0·0001 non-cervical diagnoses) were excluded. Other exclusions ≥31 years 6320 (0·35%) 4397 (0·37%) 0·02% (0·01 to 0·04) 0·002 included HPV DNA tests, non-diagnostic episodes (describing clinical procedures or treatment), and Data are number and percentage of women screened, unless otherwise stated. HGA=high-grade abnormality. diagnoses obtained through colposcopy alone. LGA=low-grade abnormality. NA=not applicable. Table 1: Number of individuals screened and incidence of low-grade cervical cytological abnormalities Data analysis and high-grade cervical histopathological abnormalities before and after introduction of the national We aimed to fi nd out whether the incidence of cervical human papillomavirus vaccination programme, by age group abnormalities detected by screening has changed since

2086 www.thelancet.com Vol 377 June 18, 2011 Articles

<18 years 18–20 years 20·0

15·0

10·0

LGA incidence (%) 5·0

0

21–25 years 26–30 years 10·0

8·0

6·0

4·0

LGA incidence (%) 2·0

0 2003 2004 2005 2006 2007 2008 2009 2010 ≥31 years Year 3·0

2·0

1·0 LGA incidence (%)

0 2003 2004 2005 2006 2007 2008 2009 2010 Year

Figure 1: Incidence of low-grade cervical abnormalities, by age group Incidence of low-grade cervical abnormalities (LGA; green dots) is the number of new diagnoses within a 3-month period per 100 women tested. Lowess smoothing trends are shown with red lines. The vertical lines, at the start of the second quarter in 2007, signify the introduction of human papillomavirus vaccination. the introduction of the HPV vaccination programme in this paper because Victorian Cancer Registry data are April, 2007, compared with the 4 years before its not available for 2008 and 2009. introduction. The incidence of histopathologically An LGA or HGA outcome was regarded as incident if it defi ned HGAs was our primary outcome measure, and was a woman’s fi rst LGA or HGA diagnosis, or a woman’s the incidence of cytologically defi ned LGAs was our fi rst abnormality that occurred at least 2 years (730 days) secondary outcome measure. after a previous abnormality, with at least two negative An LGA was defi ned according to the results of Pap tests in the intervening period. tests, coded with the Bethesda system. Low-grade A woman’s fi rst HGA diagnosis was also regarded as squamous intraepithelial lesions and atypical squamous incident if it occurred after an LGA diagnosis, irrespective cells of undetermined signifi cance were classifi ed as of test results in the intervening period. No event was cases of LGA. defi ned as incident if it occurred after a cancer diagnosis, Histopathology results were used to defi ne cases of meaning that these records were excluded from the HGA and cancer. HGA included all lesions coded as analysis. Incidence rates were defi ned as the number of cervical intraepithelial neoplasia of grade 2 or worse or incident events per 100 women tested within 3 months. adenocarcinoma in situ, with invasive cancers grouped separately, and according to the national data dictionary Statistical analysis and Australian Institute of Health and Welfare HGA and LGA incidence rates were estimated for classifi cation system.15 Cancer data are not presented in 3-month periods and stratifi ed by fi ve age groups, www.thelancet.com Vol 377 June 18, 2011 2087 Articles

<18 years 18–20 years 1·60

1·20

0·80

HGA incidence (%) 0·40

0

21–25 years 26–30 years 2·00

1·60

1·20

0·80

HGA incidence (%) 0·40

0 2003 2004 2005 2006 2007 2008 2009 2010 ≥31 years Year 0·50

0·40

0·30

0·20

HGA incidence (%) 0·10

0 2003 2004 2005 2006 2007 2008 2009 2010 Year

Figure 2: Incidence of high-grade cervical abnormalities, by age group Incidence of high-grade cervical abnormalities (HGA; green dots) is the number of new diagnoses within a 3-month period per 100 women tested. Lowess smoothing trends are shown with red lines. The vertical lines, at the start of the second quarter in 2007, signify the introduction of human papillomavirus vaccination.

March 31, 2007) and after vaccination (April 1, 2007, to Aged <18 years Aged 18–20 years Dec 31, 2009). Binary comparisons between the two Incidence 95% CI p value Incidence 95% CI p value periods for each age group were done with Fisher’s rate ratio rate ratio exact test. Before HPV vaccination 0·99 0·96–1·02 0·5 0·99 0·98–1·00 0·1 Temporal trend analysis was used to test the hypothesis (per 3-month interval) that HGA would decrease more in younger age groups After HPV vaccination 0·87 0·78–0·97 0·01 1·00 0·98–1·02 0·8 (per 3-month interval) than in older age groups after the introduction of HPV Before vs after HPV vaccination 1·14 1·00–1·30 0·05 0·99 0·97–1·02 0·7 vaccination in April, 2007, and that this decrease would be detected at a population level as a progressive HPV=human papillomavirus. decrease (negative slope) in HGA incidence. Lowess Table 2: Comparison of trends in incident high-grade cervical abnormalities in the two youngest age smoothing (bandwidth 0·5) was used to show incidence groups, before and after introduction of the HPV vaccination programme trends over time. A quantitative comparison of HGA temporal trends before and after vaccination was done with piecewise Poisson regression analysis.16–18 In the which had diff erent exposures to the vaccination context of a constant trend, the incidence rate ratio programme (individuals aged ≤17 years, 18–20 years, (IRR) was used as a measure of proportional change in 21–25 years, 26–30 years, and ≥31 years) and two incidence rate within a 3-month period. In the piecewise periods: before vaccination (Jan 1, 2003, to comparison of trends, IRR was used to estimate the

2088 www.thelancet.com Vol 377 June 18, 2011 Articles

ratios of slopes for temporal trends before and after A <18 years vaccination. StataSE (version 10) was used to do all 1·50 Observed HGA (%) statistical analyses. Predicted HGA (%)

Role of the funding source 1·00 There was no funding source for this study. MF, JMLB, and DMG had full access to data and JMLB had fi nal responsibility for the decision to submit for publication. 0·50 HGA incidence (%) Results Table 1 shows the number of individuals included in 0 the analysis and incidence rates for LGA and HGA diagnoses before and after introduction of the B 18–20 years vaccination programme. Although a decrease in LGA 1·60 incidence was recorded in age groups 21–25 years, 26–30 years, and 31 years and older, analysis of temporal 1·20 trends suggests that these changes are a continuation of long-term trends that began before vaccination 0·80 (fi gure 1). Figure 1 also indicates no decrease in LGA incidence in individuals aged younger than 18 years or

HGA incidence (%) 0·40 those aged 18–20 years after the introduction of the HPV vaccination programme. 0 We recorded a signifi cant decrease of 0·38% (95% CI 2003 2004 2005 2006 2007 2008 2009 2010 0·61–0·16; p=0·003) in HGA incidence in women Year younger than 18 years, beginning shortly after Figure 3: High-grade cervical abnormalities in individuals aged younger than introduction of the HPV vaccination programme 18 years (A) and 18–20 years (B) (fi gure 2), with a reduction from 0·85% in 2006 (the Incidence of high-grade cervical abnormalities (HGA; green dots) is the number of year before vaccination) to 0·22% in 2009 (p=0·003). new diagnoses within a 3-month period per 100 women tested. Predicted incidences are shown with a red line. The vertical lines, at the start of the second We recorded no signifi cant change in incidence in quarter in 2007, signify the introduction of human papillomavirus vaccination. women aged 18–20 years although fi gure 2 shows a non-linear decline in incidence. Small increases in incidence were recorded in women aged 21–30 years decrease began soon after the introduction of the (0·17–0·18%, 95% CI 0·10–0·26; p<0·0001) and in vaccination programme. In women aged 18–20 years, a those aged 31 years or older (0·02%, 0·01–0·04; decrease in incidence seems to have begun about p=0·002; fi gure 2). Trends in the prevalence of LGA 1·5 years after vaccine introduction. Our fi nding that and HGA by age group and time are shown in the the decrease in HGA incidence occurred in the webappendix (pp 2–3), and accord with trends shown youngest vaccination cohort before it occurred in the in fi gure 2. older, catch-up cohorts (who were more likely to have A quantitative comparison of linear trends also showed been previously sexually experienced) reinforces the a signifi cant decrease in HGA incidence after introduction appropriateness of the targeting of prophylactic HPV of the vaccination programme in individuals aged 17 years vaccines to pre-adolescent girls. or younger but no signifi cant decrease in those aged The strengths of our analyses are that we have almost 18–20 years (table 2). Figure 3 shows predicted HGA complete population-based data about cervical-screening- incidence trends from piecewise regression models for related outcomes on the VCCR. Coding of histo- the two youngest age groups. In girls aged younger than pathological abnormalities was done with the national 18 years, there is a progressive linear decrease in the standard classifi cation, and a 6-month period was allowed HGA incidence rate after the introduction of the for reporting of histology to the register and checking of vaccination programme; in those aged 18–20 years, the data. Our defi nition of incident abnormalities was HGA incidence trend after introduction of vaccination is conservative, requiring both an extended time interval non-linear, and the decline is smaller and seems delayed and two negative tests after a previous abnormality for (fi gure 3). new lesions to be defi ned as incident. Prevalence trends in our study were similar to the incidence trends and Discussion support the robustness of the fi ndings. Our defi nition of This ecological analysis reports a decrease in the the period after vaccination was also conservative because incidence of high-grade cervical lesions in girls aged we defi ned this phase as starting at the introduction of younger than 18 years in the 3 years after the start of the vaccination programme, rather than after the fi rst the HPV vaccination programme in Victoria. This date (4 months after its introduction) when women could www.thelancet.com Vol 377 June 18, 2011 2089 Articles

screened—on average, 2000–3000 per year between 2003 Panel: Research in context and 2009. These women could have been screened Systematic review because of a misinterpretation of the screening policy or We systematically searched Medline and PreMedline on they could have been at higher-than-average risk for HPV Nov 9, 2010, with the search terms (“HPV vaccination” OR infection and cervical intraepithelial neoplasia. Some the subject heading “Papillomavirus vaccines”) AND ([the individuals could have been screened too early because subject heading “cervical intraepithelial neoplasia” OR “CIN”] they were sexually active early in mid-adolescence, OR ‘impact’) with a publication date from Jan 1, 2006, meaning they would have received vaccination after they onwards (the fi rst HPV vaccine was licensed in 2006). We had become sexually active. However, this possibility identifi ed 418 articles but identifi ed no population-based could not explain our fi ndings because the vaccine would post vaccination reports on cervical intraepithelial neoplasia. be less eff ective for such individuals. Similarly, if they were screened early because they were deemed at high Interpretation risk, we would expect the lesion prevalence to be higher Our study is the fi rst to report the eff ect of a national human not lower in those women. One scenario that could papillomavirus vaccination programme on cervical contribute to a decrease in incidence is if young women abnormalities at a population level. With data from a at high risk are preferentially no longer being screened. state-based cervical screening register, we have shown a We believe such a scenario is unlikely for the following decrease in high-grade cervical abnormalities in young women reasons: no signifi cant decrease was apparent in the after the implementation of the vaccination programme. older catch-up cohorts; all vaccinated cohorts were targeted with the same information about the need for screening after vaccination; and the decrease in screening have completed the three-dose course. This starting point rates in younger women occurred before the introduction allows for some vaccine eff ectiveness after receipt of one of the vaccination programme. to two doses of prophylactic HPV vaccines, which is Understanding of the possible eff ect of vaccination on biologically plausible.19 screening behaviour is important to exclude diff erential The main limitation of our analysis is that it is ecological screening in vaccinated and unvaccinated women as an in nature, and therefore a causal link between the explanation for recorded changes in lesion prevalence. recorded decrease in incidence and the vaccination Widespread publicity that accompanied the vaccination programme cannot necessarily be ascribed. To roll-out emphasised the importance of continued substantiate these fi ndings, cervical cytology data should screening, and a Victorian population-based telephone be linked to HPV vaccination register data to enable survey in 2009 found that 96% of women aged analysis of cervical abnormality rates and participation 18–28 years knew that Pap tests were still needed after rates by vaccination status. Monitoring of the eff ect of the vaccination.5 In Victoria, as in the rest of Australia, vaccine is complex and needs data from several sources overall cervical screening participation by the target regarding cancer and abnormality rates, participation in group of women aged 20–69 years has been stable for screening, adverse events, and HPV typing of cancers about a decade. However, in women younger than and abnormalities.20 However, we believe that our 35 years, a gradual decrease in participation has been fi ndings have strong biological plausibility and that the recorded in the past decade.7 In Victoria, 58% of women specifi c temporal association, diff erential by age (which aged 20–24 years and 70% of women aged 25–29 years is related to both coverage and likelihood of sexual activity had a Pap test between 2007 and 2009, compared with and therefore HPV exposure before vaccination), suggests 62% of women aged 20–24 and 74% of women aged that the vaccination programme caused the decrease. 25–29 years between 2004 and 2006.12 Reasons for this Data from cohort studies and HPV vaccine trials indicate decrease are unclear but reported barriers to screening that the time from incident infection with HPV types 16 for young women include young women having a low or 18 to development of cervical intraepithelial neoplasia awareness of the purpose of cervical screening, of grade 2 or worse is often less than 12 months.21,22 perceiving that the test would be embarrassing or New guidelines for the management of abnormalities painful, and reporting a lack of time or not even having detected by screening were adopted in Australia in 2006.11 thought of having a Pap test.23 There has also been an These new guidelines were more conservative than the increase in the population of eligible women in Victoria, previous guidelines in the management of women with and a delay in health-service use in young women newly LGAs and are unlikely to have had an eff ect on the migrated to Victoria could be a contributing factor. A reported incidence of HGAs specifi cally in younger gradual decrease in the number of women who were women; neither guidelines have specifi c recom- screened too early (before 18 years of age) is evident in mendations targeting women aged 20 years or younger. Victoria, perhaps as a result of increased eff orts in Little is known about the characteristics of women who education for practitioners; this improvement in attend Pap screening before the recommended starting compliance with screening recommendations is not age in Victoria. However, few young women were temporally related to the introduction of the vaccination

2090 www.thelancet.com Vol 377 June 18, 2011 Articles

programme and is unlikely to explain our fi ndings, these ecological results, our fi ndings are a timely because the denominator is screened women. As the reminder that cervical screening programmes will need cohorts vaccinated before becoming sexual active enter to adapt and respond to a post-vaccination environment screening, data linkage between the vaccine and Pap in which lesion prevalence will decrease, accelerating registers will provide information about screening the need to defi ne workable screening algorithms, participation in both vaccinated and unvaccinated especially in vaccinated populations.30 women, and will be crucial to confi rm the emerging Contributions trends in the incidence of cervical abnormalities JMLB, DMG, and MF designed and were principal investigators of the reported in this study. study, with assistance from MS, GC, and CLM. MF prepared and We recorded no signifi cant decrease in incidence of analysed incidence data. Data interpretation was led by DMG, JMLB, and MS, with statistical interpretation by MF. JMLB and DMG wrote the LGAs, which are a subset of acute HPV infections. fi rst draft and all authors contributed to the fi nal report. Although HGAs are strongly associated with the detection Confl icts of interest of HPV types 16 and 18 (detected in >50% of all patients, JMLB, DMG, and MS are investigators on an Australian Research probably more in young women),24 LGAs are associated Council Linkage Grant, for which CSL Biotherapies is a partner less strongly with detection of HPV types 16 and 18 organisation. JMLB was an investigator on a national HPV prevalence study that received partial, equal, and unrestricted funding from CSL (about 25%; HPV types 6 and 11 are detected in Biotherapies and GlaxoSmithKline. GC, MF, and CLM declare that they 25 about 10%). All 40 genital HPV types can lead to low- have no confl icts of interest. grade Pap test abnormalities, and most young women Acknowledgments 26 have concurrent infections with more than one type. We thank Grace Zampogna (VCCR) for her assistance with variable Furthermore, physiological changes such as infl ammation defi nitions and appreciate the methodological advice received from and atrophy can closely mimic the appearance of LGAs.27 Ian Gordon (Statistical Consulting Centre, University of Melbourne, Victoria, Australia), Karen Canfell (NSW Cancer Council, NSW, Therefore, a reduction in infection with HPV types 16 Australia), and Carolyn Nickson (University of Melbourne, Victoria, or 18 might not result in a demonstrable decrease in the Australia). The VCCR is fully funded by the Victorian Government and detection of LGAs on Pap tests. An intention-to-treat operated by the Victorian Cytology Service. analysis from the phase 3 quadrivalent vaccine trials of References more than 17 000 women aged 15–26 years recorded a 1 Sellors J. HPV vaccination in the developing world. http://www. thelancetconferences.com/hpv-and-cancer/presentations/sellors_ statistically signifi cant 19% reduction of any HGAs (with vaccine_presentation.pdf (accessed April 7, 2011). an average follow-up of 3·6 years), but a non-signifi cant 2 Smith JS, Lindsay L, Hoots B, et al. Human papillomavirus type reduction in any Pap abnormality (11·3% reduction; distribution in invasive cervical cancer and high-grade cervical lesions: a meta-analysis update. Int J Cancer 2007: 21: 621–32. diff erence 1·32 per 100 person-years at risk, 95% CI 3 Levy-Bruhl D, Bousquet V, King LA, et al. The current state of 28 0·74–1·90). Although an eventual decrease in LGAs introduction of HPV vaccination into national immunisation because of vaccination in HPV-naive cohorts is predicted,29 schedules in Europe: results of the VENICE 2008 survey. these data emphasise that cervical abnormalities will Eur J Cancer 2009; 45: 2709–13. 4 Immunise Australia Program. Human papillomavirus (HPV). http:// 26 continue to occur in vaccinated women in the future. www.immunise.health.gov.au/internet/immunise/publishing.nsf/ We are aware of no other study to document the Content/immunise-hpv (accessed May 19, 2011). possible eff ect of a national HPV vaccination 5 Brotherton JML, Mullins R. Will young women have a free HPV vaccine and will it impact on cervical screening? An analysis from programme on cervical abnormalities at a population Victoria, Australia. http://www.hpv2010.org/main/index. level (panel). We have shown a decrease in the incidence php?option=com_conference&view=presentation&id=868&confere of HGA in young women after the implementation of nce=1&Itemid=103 (accessed May 19, 2011). 6 Australian Institute of Health and Welfare and Australasian the vaccination programme, and that this decrease Association of Cancer Registries. Cancer in Australia: an overview, occurred soon after vaccination. This fi nding suggests 2008. Cancer series no 46 (CAN 42). Canberra: AIHW, 2008. an urgent need to review the age at which cervical 7 Australian Institute of Health and Welfare 2010. Cervical screening in Australia 2007–2008: data report. Cancer series no 54 (CAN 50). screening is begun in Australia and in other countries Canberra: AIHW. with national vaccination programmes that begin 8 Gertig DM, Brotherton JML, Saville M. Methods for measuring screening of women at a young age, because cost- HPV vaccination coverage and the role of the National HPV Vaccination Program Register, Australia. Sex Health 2011; eff ectiveness of screening will decrease for the youngest 8: 171–78. age groups screened. In countries that screen women at 9 Smith M , Canfell K, Brotherton J, Lew J-B, Barnabas R, on behalf an older age, the eff ect of the vaccination will take of the Modellers Sans Frontières working group. The predicted impact of vaccination on human papillomavirus infections in longer to be seen. During the study period, we recorded Australia. Int J Cancer 2008; 123: 1854–63. no decrease in incidence of LGAs in women younger 10 Fairley CK, Hocking JS, Gurrin LC, Chen MY, Donovan B, than 21 years (in whom LGA incidence was greater than Bradshaw CS. Rapid decline in presentations of genital warts after 10%), which was to be expected because of the lower the implementation of a national quadrivalent human papillomavirus vaccination programme for young women. proportion of abnormalities that are due to vaccine Sex Transm Infect 2009; 85: 499–502. preventable types. Long-term gradual decreases in LGA 11 NHMRC. Screening to prevent cervical cancer: guidelines for the rates were, however, noted in women older than management of asymptomatic women with screen detected abnormalities, 2005. http://www.nhmrc.gov.au/publications/ 21 years. Although more time and linked data analyses synopses/wh39syn.htm (accessed Dec 7, 2010). by vaccination status are now needed to substantiate www.thelancet.com Vol 377 June 18, 2011 2091 Articles

12 Victorian Cervical Cytology Registry. Victorian cervical cytology 22 Mao C, Koutsky LA, Ault KA, et al. Effi cacy of human registry statistical Report 2009. http://www.vccr.org/stats.html papillomavirus-16 vaccine to prevent cervical intraepithelial neoplasia: (accessed Dec 7, 2010). a randomized controlled trial. Obstet Gynecol 2006; 107: 1425. 13 Asghari S, Courteau J, Carpentier A, et al. Optimal strategy to 23 PapScreen Victoria. Young women increasingly shunning Pap tests identify incidence of diagnostic of diabetes using administrative [media release]. http://www.cancervic.org.au/media/media- data. BMC Med Res Methodol 2009; 9: 62. releases/2010-media-releases/august-2010/pap-test-rate-lowest-in- 14 Brameld KJ, Holman CJD, Lawrence DM, Hobbs MST. Improved decade.html (accessed Dec 31, 2010). methods for estimating incidence from linked hospital morbidity 24 Porras C, Rodriguez AC, Hildesheim A, et al. Human data. Int J Epidemiol 2003; 32: 617–24. papillomavirus types by age in cervical cancer precursors: 15 Australian Institute of Health and Welfare. National Cervical predominance of human papillomavirus 16 in young women. Screening Program Dataset, Version 5. Canberra: AIHW, 2008. Cancer Epidemiol Biomarkers Prev 2009; 18: 863–65. 16 UCLA: Academic Technology Services SCG. Stata FAQ: how can I 25 Cliff ord GM, Rana RK, Franceschi S, Smith JS, Gough G, run piecewise regression in Stata? http://www.ats.ucla.edu/stat/ Pimenta JM. Human papillomavirus genotype distribution in stata/faq/piecewise.htm (accessed July 30, 2010). low-grade cervical lesions: comparison by geographic region and 17 Constantine R, Tandon R. Changing trends in pediatric antipsychotic with cervical cancer. Cancer Epidemiol Biomarkers Prev 2005; use in Florida’s Medicaid program. Psychiatr Serv 2008; 59: 1162–68. 14: 1157–64. 18 Vito MRM. Estimating regression models with unknown break- 26 Heley S, Brotherton J. Abnormal Pap tests after the HPV vaccine. points. Stat Med 2003; 22: 3055–71. Aust Family Physician 2009; 38: 977–79. 19 Kreimer AR, Rodriguez AC, Hildesheim A, et al. Proof-of-principle: 27 Kurman RJ, Solomon D. The Bethesda System for reporting effi cacy of fewer than 3-doses of a bivalent HPV 16/18 vaccine cervical/vaginal cytologic diagnoses: Defi nitions, criteria, and against incident persistent HPV infection in Guanacaste, Costa explanatory notes for terminology and specimen adequacy. Rica. http://www.hpv2010.org/main/index.php?option=com_confer New York; Springer-Verlag, 1994. ence&view=presentation&id=1754&conference=1&Itemid=103 28 Muñoz N, Kjaer SK, Sigurdsson K, et al. Impact of human (accessed May 19, 2011). papillomavirus (HPV)-6/11/16/18 vaccine on all HPV-associated 20 WHO. Report of the meeting on HPV Vaccine Coverage and Impact genital diseases in young women. J Natl Cancer Inst 2010; Monitoring 16-17 November 2009 Geneva, Switzerland. Department 102: 325–39. of Immunization, Vaccines and Biologicals, World Health 29 Cuzick J, Castanon A, Sasieni P. Predicted impact of vaccination Organization (915 WHO/IVB/10.05 May 2010). http://whqlibdoc. against human papillomavirus 16/18 on cancer incidence and who.int/hq/2010/WHO_IVB_10.05_eng.pdf. (accessed cervical abnormalities in women aged 20–29 in the UK. Br J Cancer April 1, 2011). 2010; 102: 933–39. 21 Winer RL, Kiviat NB, Hughes JP, et al. Development and duration 30 Tota J, Mahmud SM, Ferenczy A, Coutlée F, Franco EL. Promising of human papillomavirus lesions, after initial infection. J Infect Dis strategies for cervical cancer screening in the post-human 2005; 191: 731–38. papillomavirus vaccination era. Sexual Health 2010; 7: 376–82.

2092 www.thelancet.com Vol 377 June 18, 2011 The Journal of Infectious Diseases MAJOR ARTICLE

Very Low Prevalence of Vaccine Human Papillomavirus Types Among 18- to 35-Year Old Australian Women 9 Years Following Implementation of Vaccination Dorothy A. Machalek,1,2,3 Suzanne M. Garland,2,4 Julia M. L. Brotherton,3,5 Deborah Bateson,6,7 Kathleen McNamee,8,9 Mary Stewart,6 S. Rachel Skinner,10 Bette Liu,11 Alyssa M. Cornall,1,2,4 John M. Kaldor,12 and Sepehr N. Tabrizi1,2,4 217 1Department of Microbiology and Infectious Diseases, The Royal Women’s Hospital, Melbourne; 2Murdoch Children’s Research Institute, Melbourne, Victoria; 3School of Downloaded from https://academic.oup.com/jid/article-abstract/217/10/1590/4841780 by Yale University user on 12 March 2019 Population and Global Health, University of Melbourne, Victoria; 4Department of Obstetrics and Gynaecology, University of Melbourne, Victoria; 5National HPV Vaccination 15 Program Register, Victorian Cytology Service, East Melbourne, Victoria; 6Family Planning New South Wales, Sydney; 7Discipline of Obstetrics, Gynaecology and Neonatology, University of Sydney; 8Family Planning Victoria, Melbourne; 9Department of Obstetrics and Gynaecology, Monash University, Clayton, Victoria; 10Sydney University Discipline of Paediatrics and Child Health, Children’s Hospital Westmead; 11School of Public Health and Community Medicine, and 12The Kirby Institute for Infection and Immunity in Society, May University of New South Wales, Sydney, Australia

Introduction. A quadrivalent human papillomavirus vaccination program targeting females aged 12–13 years commenced in Australia in 2007, with catch-up vaccination of 14–26 year olds through 2009. We evaluated the program’s impact on HPV preva- lence among women aged 18–35 in 2015. Methods. HPV prevalence among women aged 18–24 and 25–35 was compared with prevalence in these age groups in 2005– 2007. For women aged 18–24, we also compared prevalence with that in a postvaccine study conducted in 2010–2012. Results. For the 2015 sample, Vaccination Register-confirmed 3-dose coverage was 53.3% (65.0% and 40.3% aged 18–24 and 25–35, respectively). Prevalence of vaccine HPV types decreased from 22.7% (2005–2007) and 7.3% (2010–2012), to 1.5% (2015) (P trend < .001) among women aged 18–24, and from 11.8% (2005–2007) to 1.1% (2015) (P = .001) among those aged 25–35. Conclusions. This study, reporting the longest surveillance follow-up to date, shows prevalence of vaccine-targeted HPV types has continued to decline among young women. A substantial fall also occurred in women aged 25–35, despite lower coverage. Strong herd protection and effectiveness of less than 3 vaccine doses likely contributed to these reductions. Keywords. human papillomavirus; HPV infection; prevalence; vaccine impact; surveillance; women.

Infection with human papillomavirus (HPV) is the underlying high coverage each year, with over 70% of school-based cohorts cause of cervical cancer and other types of anogenital cancers in receiving all 3 doses [6]. All female residents of Australia born both males and females [1, 2]. A government funded national on or after 30 June 1980 were eligible for free vaccination in the program using the quadrivalent HPV vaccine (protection catch-up program. The result of this coverage includes striking STANDARD against types 6, 11, 16, and 18) was introduced in Australia in reductions in HPV infections, genital warts, and cervical high- 2007 for females, and in 2013 extended to males. The vaccine grade lesions, with the greatest impact observed among the has high protective efficacy when administered to people with- youngest cohorts [7–10]. out existing infection [3]. The catch-up program provided the opportunity for individ- Between 2007 and 2009, all females aged 12–26 years were uals to be vaccinated at ages older than routinely recommended. eligible to receive 3 doses of the vaccine free of charge. The pro- Indeed, a decline in both genital warts and cervical lesions in gram was delivered through schools and community providers. females aged up to 30 years has recently been observed, as vac- In that period, an estimated 83% of adolescent girls and 55% cinated cohorts age and new infections are prevented in these of women received at least 1 vaccine dose; with 70% and 32%, groups [11, 12]. These findings suggest that the population-level respectively, having all 3 doses [4, 5]. Vaccination of children prevalence of vaccine-targeted HPV types has significantly aged 12–13 years through schools continued and has achieved declined across all cohorts who have been offered vaccination. This is despite the lower vaccine coverage among older groups,

Received 24 November 2017; editorial decision 31 January 2018; accepted 6 February 2018; as well as the likelihood that a substantial proportion of these published online February 7, 2018. older women would have been previously exposed to vaccine Presented in part: 15th National Immunisation Conference, 7–9 June 2016, Brisbane, Australia. HPV types at the time of vaccination. Correspondence: D. Machalek, PhD, Department of Microbiology and Infectious Diseases, The Royal Women’s Hospital, Bio 21 Institute, Level 1 Building 404, 30 Flemington Road, We previously reported on the substantial decline in vac- Parkville, Victoria 3052, Australia ([email protected]). cine-targeted HPV types in a repeat cross-sectional study The Journal of Infectious Diseases® 2018;217:1590–600 that compared cervical HPV prevalence among females aged © The Author(s) 2018. Published by Oxford University Press for the Infectious Diseases Society of America. All rights reserved. For permissions, e-mail: [email protected]. 18–24 years before the introduction of vaccination with preva- DOI: 10.1093/infdis/jiy075 lence among females of the same age group recruited 4–5 years

1590 • JID 2018:217 (15 May) • Machalek et al after program implementation [13, 14]. These data provided the the time of Papanicolaou test, exfoliated cervical cells were col- first evidence of the population-level benefit of the program, lected in PreservCyt (Hologic Corporation, Bedford, MA) for both direct and through herd protection [13]. The program’s HPV testing. Information on age, smoking status, and residen- impact on HPV infections among older vaccine-eligible females tial postcode was collected from routine records. Information has not been described to date using this repeat cross-sectional on sexual practices was sought from women recruited in the 2 study methodology. postvaccine samples, including age at first sexual intercourse, In this follow-up repeat cross-sectional study, we compared number of male sexual partners (lifetime and in the previous prevalence of vaccine-targeted HPV types among women in the 12 months), and whether or not they had received HPV vac- age groups 18–24 and 25–35 recruited in 2015, with prevalence cination. Written consent was collected to obtain vaccination among women in the same age groups recruited in 2005–2007. status from the National HPV Vaccination Program Register For women aged 18–24, we also compared prevalence with that (hereafter referred to as the Register) [17]. in a postvaccine study conducted in 2010–2012 [13], extending HPV testing was performed as previously described [13, 16], Downloaded from https://academic.oup.com/jid/article-abstract/217/10/1590/4841780 by Yale University user on 12 March 2019 the results of a previous study. with one modification. For specimens collected in 2015 the cobas HPV test (Roche Diagnostics, Indianapolis, IN) was used METHODS as the initial screening assay, with the results used for clinical Study Population follow-up, replacing the Roche Amplicor HPV (AMP) [13]. For this report, the prevaccine sample consisted of women High concordance has been reported between AMP and cobas aged 18–35 years who attended urban family planning clin- in several studies [18, 19]. Briefly, 1 mL of the PreservCyt speci- ics in Victoria and New South Wales (NSW), Australia, for men was tested for the presence of 14 high-risk HPV types using cervical screening, between November 2005 and April 2007. the cobas HPV test. A second 1-mL aliquot of the original sam- This was a subset of women who participated in the national ple was peleted and resuspended in 200 µL of phosphate buff- Women, Human Papillomavirus Prevalence, Indigenous, Non- ered saline (PBS) [20] to use for DNA extraction (MagNA Pure Indigenous, Urban, Rural Study (WHINURS) [15, 16], previ- 96 DNA and Viral Nucleic Acid Small Volume Kit, Pathogen ously described. There were 2 postvaccine-implementation Universal 200 protocol; Roche Molecular Diagnostics), then samples made up of women who attended family planning clin- eluted in 100 µL. All extracted DNA was assessed for adequacy ics in the same metropolitan areas in Victoria and NSW: the by quantitative polymerase chain reaction (PCR) amplification first consisted of the Victoria and NSW subset of women aged of a 260-bp segment of the human β-globin gene [21]. Extracted 18–24 years, recruited between 2010 and 2012 from clinics in DNA (20 µL of extract) from samples negative or invalid on 3 States (NSW, Victoria, and Western Australia), as described cobas were analyzed for the presence of mucosal HPV type DNA previously [13]; the second consisted of a subset of women aged by an PGMY09-PGMY11-based HPV consensus PCR/enzyme- 18–35 years, recruited between January and October 2015. For linked immunosorbent assay (ELISA) using a set of biotin-la- the 2015 sample, only women born on or after 30 June 1980 beled probes, as previously described [21–24]. Extracted DNA (vaccine catch-up program eligible) were included. As pre- (50 µL of extract) from samples positive for HPV by either test viously described [14], there were some changes in the num- (cobas or PCR/ELISA) were genotyped using the Linear Array ber of participating clinics and in clinic location between the HPV genotyping test (Roche Molecular Diagnostics) according pre- and first postvaccine implementation samples, but no fur- to manufacturer’s instructions, with minor modifications as ther changes prior to the second post vaccine sample. Under previously reported [13, 25]. Due to possible cross-reactivity of Australian guidelines before December 2017, cervical screen- the HPV52 probe with types 33, 35, and 58 amplicons, samples ing using cytology began at age 18 years or 2 years after first positive for the HPV52 probe in the presence of 1 or more of intercourse. There were not changes to this recommendation these 3 probes were further tested for HPV52 using a type-spe- between 2005 and 2015. cific PCR assay [26]. Approval to undertake this study was obtained from ethics Statistical Analyses committees associated with each study site, and all participants Vaccine doses are known to be underreported to the Register, provided written, informed consent. particularly among women vaccinated in the community [4]. Procedures In view of this, classification of vaccination status was based As previously described, the procedures to recruit partici- on a composite measure of self-reported and registry reported pants were identical for the prevaccine and postvaccine sam- doses, as previously described [13]. Women were classified as ples [13, 16]. Clinic staff identified all consecutive age-eligible fully vaccinated if they had 3 doses of the vaccine recorded women attending health services for routine cervical screen- on the Register. Unvaccinated women were those reporting ing. Invitation to participate was dependent on the clinician’s not being vaccinated who also had no Register record of vac- judgment that there was sufficient time to discuss the study. At cination. If consent to access the Register was not obtained,

HPV Vaccine Impact in Australian Women • JID 2018:217 (15 May) • 1591 self-report of nonvaccination was accepted. Women who had sample (n = 688) and the prevaccine sample (n = 88) [13]. They received 1 or 2 doses of vaccine as confirmed by the Register, were also more likely to be fully vaccinated (P < .001), com- or had self-reported doses that could not be verified on the pared with women in the first postvaccine sample; 21 (10.5%) Register, were classified as being partly vaccinated. The latter were unvaccinated, 130 (65.0%) were fully vaccinated, and 49 category included women who reported doses, but did not pro- (24.5%) were partly vaccinated. Women aged 25–35 in 2015 vide consent for access to Register-recorded vaccination status (n = 181) were younger (P = .02) than women recruited in the [13]. Each individual’s residential postcode was used to assign prevaccine sample (n = 187) [16]. Overall, 33 (18.2%) were socioeconomic status (upper or lower 50th centile based on the unvaccinated, 73 (40.3%) were fully vaccinated, and 75 (41.4%) Australian Bureau of Statistics Index of Relative Socioeconomic were partly vaccinated. Disadvantage [12]) and area of residence (major city or The combined prevalence of vaccine-targeted HPV types regional/remote based on the Accessibility/Remoteness Index among women aged 18–35 decreased from 15.3% (42/275) in of Australia classification from the 2011 census [13]). 2005–2007, to 1.3% (5/381) in 2015; aPR = 0.08 (95% CI, 0.03– Downloaded from https://academic.oup.com/jid/article-abstract/217/10/1590/4841780 by Yale University user on 12 March 2019 HPV prevalence estimates and 95% confidence intervals (CIs) 0.20) compared with the prevaccine sample; P < .001 (Table 2). were calculated using the exact binomial method in the following Of the 5 samples positive for vaccine-targeted HPV types in categories: vaccine-targeted HPV types (HPV6/11/16/18); any 2015, 4 were positive for HPV16 and 1 for HPV18. There were of the 5 additional HPV types covered by the 9-valent vaccine no cases of HPV6 or HPV11 detected. In analyses stratified by (HPV31/33/45/52/58) [27]; any high-risk HPV types other than age group (Table 2 and Figure 1), prevalence of vaccine-targeted 16/18 (HPV31/33/35/39/45/51/52/56/58/59/68); any high-risk HPV types among women aged 18–24 decreased from 22.7% in HPV types (HPV16/18/31/33/35/39/45/ 51/52/56/58/59/68); 2005–2007 and 7.3% in 2010–2012, to 1.5% in 2015; aPR = 0.40 any 1 of 37 HPV types identified by Linear Array; and any 1 of (95% CI, 0.25–0.63) and 0.08 (95% CI, 0.02–0.26), respectively, 37 HPV types excluding 6/11/16/18. Chi-square tests were used compared with the prevaccine sample; P trend < .001. A signif- to examine differences in characteristics between study periods, icant decline in vaccine-targeted HPV types was also observed and within subgroups of vaccination status. Binomial log linear among women aged 25–35, from 11.8% in 2005–2007 to 1.1% regression was used to estimate prevalence ratios (PRs) and 95% in 2015; aPR = 0.10 (95% CI, 0.02–0.41); P = .001. No signifi- CIs for each grouping of HPV types between the study peri- cant reductions in detection of nonvaccine-targeted HR-HPV ods, adjusted for sociodemographic characteristics that varied types were observed. However, we noted a significantly lower between the groups. PRs were also estimated for each vaccine prevalence of nonvaccine HPV types (excluding 6/11/16/18) subgroup (unvaccinated, partly vaccinated, and fully vaccinated) overall, among women recruited in 2015 compared with the in the 2015 sample, compared with the prevaccine sample. We prevaccine sample (31.5% [95% CI, 27.0–36.4] vs 43.3% [95% performed logistic regression to investigate the relationship CI, 37.5–49.2], respectively, aPR = 0.70 [95% CI, 0.57–0.85]) between HPV, vaccination status, and a range of sociodemo- (Supplementary Table S1). graphic and behavioral characteristics among women recruited Next, results from 54 unvaccinated, 124 partly vaccinated, in 2015. Variables that were associated with each outcome at and 203 fully vaccinated women recruited in 2015 were com- P < .10, along with vaccination status and age, were included pared with those of 275 women recruited in the prevaccine in the adjusted models. Data analyses were performed using sample [13]. Fully vaccinated women were younger (P < .001), STATA version 14 (Stata Corporation, College Station, TX). and more likely to have been born in Australia (P < .001), but were otherwise similar for other measured characteristics RESULTS (Supplementary Table S2). Prevalence of vaccine-targeted HPV Between January and October 2015, we recruited 381 women types was significantly lower than the prevaccine sample in all aged 18–35 (Table 1). Compared with the prevaccine sample vaccine-eligible subgroups recruited in 2015: aPR = 0.13 (95% (n = 275), women recruited in 2015 were younger (P < .001), CI, 0.02–0.91) for unvaccinated; aPR = 0.10 (95% CI, 0.02–0.41) and less likely to be smokers (P = .04). One in 7 (n = 54; 14.2%) for partly vaccinated; and aPR = 0.06 (95% CI, 0.01–0.24) for were unvaccinated and just over half (n = 203; 53.3%) were fully fully vaccinated women (Table 3). No significant reductions in vaccinated. The remaining 124 (32.6%) were partly vaccinated: detection of nonvaccine-targeted HR-HPV types were observed 21 (5.5%) had documented receipt of 1 or 2 doses, 101 (26.5%) among each vaccine-eligible subgroup, compared with the pre- reported receiving doses or were unsure of their status, but had vaccine sample (Table 3). Again, we noted small but significant no Registry record of being vaccinated, and 2 (0.5%) women reductions in the prevalence of nonvaccine HPV types overall reported that they had been vaccinated but did not provide con- (Supplementary Table S1). sent to verify their status with the Register. Among women recruited in 2015, there was a higher crude In analyses stratified by age group (Table 1), women aged prevalence of nonvaccine-targeted HPV types among fully vac- 18–24 in 2015 (n = 200) were older (P < .001), and less likely to cinated women compared with those who were unvaccinated or be smokers (P = .03), than those recruited in the first postvaccine partly vaccinated. In univariate analyses, detection of any HPV

1592 • JID 2018:217 (15 May) • Machalek et al Table 1. Cohort Characteristics Among Australian Females Attending for Cervical Cytology Screening, According to Study Period and Age Group

Prevaccine Sample Vaccine-Eligible Samples

2005–2007 2010–2012 2015

n (%) n (%) n (%) P valuea 18–35 years old N = 275 … N = 381 Age Median (IQR) 27 (23–30) … 24 (22–27) <.001 Mean (SD) 26.8 (4.5) … 25.2 (4.0) <.001 Current smoker No 25 (74.5) … 309 (81.1) .04 Ye s 70 (25.5) … 72 (18.9) Socioeconomic status Less disadvantaged 235 (85.5) … 308 (80.8) .12

More disadvantaged 40 (14.5) … 73 (19.2) Downloaded from https://academic.oup.com/jid/article-abstract/217/10/1590/4841780 by Yale University user on 12 March 2019 Area of residence Major city 269 (97.8) … 373 (97.9) .94 Regional/remote 6 (2.2) … 8 (2.1) Vaccination statusc Unvaccinated … … 54 (14.2) Partly vaccinated … … 124 (32.6) Fully vaccinated … … 203 (53.3) 18–24 years old n = 88 n = 688b n = 200 Age Median (IQR) 22 (20–23) 21 (20–23) 22 (21–23) <.001 Mean (SD) 21.6 (1.8) 21.3 (1.8) 22.1 (1.5) <.001 Current smoker No 60 (68.2) 470 (69.3) 157 (78.5) .03 Ye s 28 (31.8) 208 (30.7) 43 (21.5) Socioeconomic status Less disadvantaged 78 (88.6) 573 (83.4) 165 (82.5) .40 More disadvantaged 10 (11.4) 114 (16.6) 35 (17.5) Area of residence Major city 87 (98.9) 669 (97.4) 196 (98.0) .64 Regional/remote 1 (1.1) 18 (2.6) 4 (2.0) Completed high school No … 26 (3.8) 11 (5.5) .28 Ye s … 662 (96.2) 189 (94.5) Country of birth Australia … 589 (85.6) 168 (84.0) .572 Other … 99 (14.4) 32 (16.0) Age at first vaginal sex ≤16 years old … 359 (53.4) 100 (50.0) .40 > 16 years old … 313 (46.6) 100 (50.0) Lifetime number of sexual partners 1–2 … 134 (19.5) 39 (19.5) .53 3–4 … 141 (20.5) 34 (17.0) ≥5 … 413 (60.0) 127 (63.5) Number of sexual partners in the previous 0–1 … 327 (47.5) 99 (49.8) .58 12 months ≥2 … 361 (52.5) 100 (50.3) Vaccination statusc Unvaccinated … 86 (12.5) 21 (10.5) <.001 Partly vaccinated … 250 (36.3) 49 (24.5) Fully vaccinated … 352 (51.2) 130 (65.0) 25–35 years old n = 187 … n = 181 Age Median (IQR) 29 (26–32) … 28 (25–31) .02 Mean (SD) 29.3 (3.1) … 28.5 (3.2) .03 Current smoker No 145 (77.5) … 152 (84.0) .12 Ye s 42 (22.5) … 29 (16.0) Socioeconomic status Less disadvantaged 157 (84.0) … 143 (79.0) .22 More disadvantaged 30 (16.0) … 38 (21.0) Area of residence Major city 182 (97.3) … 177 (97.8) .77 Regional/remote 5 (2.7) … 4 (2.2) Vaccination statusc Unvaccinated … … 33 (18.2) Partly vaccinated … … 75 (41.4) Fully vaccinated … … 73 (40.3) aP values presented are score test of homogeneity between the study periods. bSome numbers do not add up to 688 due to missing data. cWomen were classified as fully vaccinated if they had 3 doses of the vaccine recorded on the National HPV Vaccination Program Register. Unvaccinated women were those reporting not being vaccinated who also had no Register record of vaccination. If consent to access the Register was not obtained, self-report of nonvaccination was accepted. Women who had received 1 or 2 doses of vaccine as confirmed by the Register, or had self-reported doses that could not be verified on the Register, were classified as being partly vaccinated. Abbreviations: IQR, interquartile range; SD, standard deviation.

HPV Vaccine Impact in Australian Women • JID 2018:217 (15 May) • 1593 Table 2. Crude Prevalence and Prevalence Ratios for Human Papillomavirus (HPV) Detected among Australian Females Attending for Cervical Cytology Screening, According to Study Period and Age Group, in Unadjusted and Adjusted Analyses

Crude Prevalence Comparison of Vaccine-Eligible Sample With Prevaccine Samples

n (%; 95% CI) PR (95% CI) P value aPRa (95% CI) P value

18–35 years old HPV 6, 11, 16, 18 Prevaccine sample (2005–2007) 42 (15.3; 11.5–20.0) 1.00 (ref) 1.00 (ref) Vaccine-eligible sample (2015) 5 (1.3; 0.5–3.1) 0.09 (0.03–0.21) <.001 0.08 (0.03–0.20) <.001 HPV 31, 33, 45, 52, 58 Prevaccine sample (2005–2007) 30 (10.9; 7.7–15.2) 1.00 (ref) 1.00 (ref) Vaccine-eligible sample (2015) 47 (12.3; 9.4–16.0) 1.13 (0.73–1.74) .58 1.01 (0.65–1.56) .98

HR-HPV types other than 16 and 18 Downloaded from https://academic.oup.com/jid/article-abstract/217/10/1590/4841780 by Yale University user on 12 March 2019 Prevaccine sample (2005–2007) 67 (24.4; 19.6–29.8) 1.00 (ref) 1.00 (ref) Vaccine-eligible sample (2015) 86 (22.6; 18.6–27.1) 0.93 (0.70–1.22) .59 0.83 (0.63–1.10) .20 Any HR-HPV type Prevaccine sample (2005–2007) 85 (30.9; 25.7–36.6) 1.00 (ref) 1.00 (Ref) Vaccine-eligible sample (2015) 89 (23.4; 19.4–27.9) 0.76 (0.59–0.97) .03 0.69 (0.53–0.89) .004 Any HPV type Prevaccine sample (2005–2007) 129 (46.9; 41.1–52.8) 1.00 (ref) 1.00 (ref) Vaccine-eligible sample (2015) 123 (32.9; 27.8–37.2) 0.69 (0.57–0.83) <.001 0.66 (0.54–0.80) <.001 18–24 years old HPV types 6, 11, 16, 18 Prevaccine sample (2005–2007) 20 (22.7; 15.1–32.7) 1.00 (ref) 1.00 (ref) Vaccine-eligible sample (2010–2012) 50 (7. 3 ; 5.5–9.5) 0.32 (0.20–0.51) <.001b 0.40 (0.25–0.63) <.001b Vaccine-eligible sample (2015) 3 (1.5; 0.5–4.6) 0.07 (0.02–0.22) 0.08 (0.02–0.26) High-risk HPV types 31, 33, 45, 52, 58 Prevaccine sample (2005–2007) 13 (14.8; 8.7–23.9) 1.00 (ref) 1.00 (ref) Vaccine-eligible sample (2010–2012) 131 (19.0; 16.3–22.2) 1.29 (0.76–2.18) .75b 1.38 (0.82–2.33) .80b Vaccine-eligible sample (2015) 31 (15.5; 11.1–21.2) 1.05 (0.58–1.91) 1.09 (0.60–1.97) High-risk HPV other than 16 and 18 Prevaccine sample (2005–2007) 29 (33.0; 23.9–43.5) 1.00 (ref) 1.00 (ref) Vaccine-eligible sample (2010–2012) 251 (36.5; 33.0–40.2) 1.11 (0.81–1.52) .27b 1.12 (0.82–1.53) .46b Vaccine-eligible sample (2015) 59 (29.5; 23.6–36.2) 0.90 (0.62–1.29) 0.94 (0.65–1.35) Any HR-HPV type Prevaccine sample (2005–2007) 36 (40.9; 31.1–51.5) 1.00 (ref) 1.00 (ref) Vaccine-eligible sample (2010–2012) 269 (39.1; 35.5–42.8) 0.96 (0.73–1.25) .03b 0.98 (0.75–1.27) .05b Vaccine-eligible sample (2015) 60 (30.0; 24.0–36.7) 0.73 (0.53–1.02) 0.76 (0.55–1.05) Any HPV type Prevaccine sample (2005–2007) 47 (53.4; 42.9–63.6) 1.00 (ref) 1.00 (ref) Vaccine-eligible sample (2010–2012) 355 (51.6; 47.9–55.3) 0.97 (0.78–1.19) .002b 0.98 (0.80–1.20) .005b Vaccine-eligible sample (2015) 76 (38.0; 31.5–44.9) 0.71 (0.55–0.93) 0.73 (0.56–0.95) 25–35 years old HPV types 6, 11, 16, 18 Prevaccine sample (2005–2007) 22 (11.8; 7.9–17.3) 1.00 (ref) 1.00 (ref) Vaccine-eligible sample (2015) 2 (1.1; 0.3–4.3) 0.09 (0.02–0.39) .001 0.10 (0.02–0.41) .001 High-risk HPV types 31, 33, 45, 52, 58 Prevaccine sample (2005–2007) 17 (9.1; 5.7–14.2) 1.00 (ref) 1.00 (ref) Vaccine-eligible sample (2015) 16 (8.8; 5.5–14.0) 0.97 (0.51–1.87) .93 0.94 (0.49–1.81) .85 High-risk HPV other than 16 and 18 Prevaccine sample (2005–2007) 38 (20.3; 15.1–26.7) 1.00 (ref) 1.00 (ref) Vaccine-eligible sample (2015) 27 (14.9; 10.4–20.9) 0.73 (0.47–1.16) .18 0.70 (0.44–1.10) .12 Any HR-HPV type Prevaccine sample (2005–2007) 49 (26.2; 20.4–33.0) 1.00 (ref) 1.00 (ref) Vaccine-eligible sample (2015) 29 (16.0; 11.3–22.1) 0.61 (0.41–0.92) .02 0.58 (0.38–0.88) .01 Any HPV type Prevaccine sample (2005–2007) 82 (43.9; 36.9–51.1) 1.00 (ref) 1.00 (ref) Vaccine-eligible sample (2015) 47 (26.0; 20.1–32.9) 0.59 (0.44–0.80) <.001 0.57 (0.42–0.77) <.001 aAdjusted for age and smoking status. bScore test for trend, else the P values presented are score test of homogeneity between the study periods. Abbreviations: aPR adjusted PR; CI, confidence interval; HPV, human papillomavirus; HR-HPV, high-risk HPV; PR, prevalence ratio.

1594 • JID 2018:217 (15 May) • Machalek et al A 80 2005-2007 (n=88) 70 2010-2012 (n=688) Ptrend = .005 60 2015 (n=200) Ptrend = .05 53.4 51.6 Ptrend = .46 50 38.0 33.0 40.9 39.1 Ptrend < .001 36.5 40 29.5 30.0 22.7 Ptrend = .80 30 14.8 19.0 15.5 20 Prevalence (%) and 95% CI 7.3

10 Downloaded from https://academic.oup.com/jid/article-abstract/217/10/1590/4841780 by Yale University user on 12 March 2019 1.5 0 HPV 6, 11, HPV 31, 33, HR-HPV types Any HR-HPV Any HPV type 16, 18 45, 52, 58 excluding types 16 and 18

B 60 55 2005-2007 (n=275) P < .001 50 2015 (n=181) 43.9 45

40 P = .01 35 26.2 26.0 30 P = .12

25 16.0 P < .001 P = .85 20.3 14.9 20 9.1 8.8 15 11.8 Prevalence (%) and 95% CI 10 5 1.1 0 HPV 6, 11, HPV 31, 33, HR-HPV types Any HR-HPV Any HPV type 16, 18 45, 52, 58 excluding types 16 and 18

Figure 1. Crude human papillomavirus (HPV) prevalence among Australian females aged 18–24 (A) and 25–35 (B) years attending for routine cervical cytology screening according to study period. P values presented are score test for trend (A) or homogeneity (B) across the study periods. Abbreviations: CI, confidence interval; HR-HPV, high-risk HPV. types was associated with younger age (P = .01), being born in the subgroup of older women (aged 25–35), who were aged Australia (P = .05), reporting 5 or more lifetime sexual partners 16–26 when the program began, the prevalence had fallen by (P < .001), and 2 or more partners in the previous 12 months 90% compared with prevalence in the same age group prior to (P < .001) (Supplementary Table S3 and Supplementary Table the program. This is despite register-recorded 3-dose coverage S4). In adjusted analysis, only younger age (P = .03) and report- being only 40% in this group. We also found that prevalence of ing 5 or more lifetime sexual partners (P < .001) remained vaccine-targeted HPV types, which we had already measured independently associated. Similar results were obtained for all 4–5 years after the implementation of the program, has contin- groupings of HPV (Table 4 and Supplementary Table S4), with ued to decline in the younger age groups. High and increasing the exception of vaccine-targeted HPV types, which was not vaccine coverage, strong population-based herd protection, and associated with any covariates tested. the effectiveness of less than 3 doses of the HPV vaccine, are likely to be contributing to these reductions in HPV infections. DISCUSSION Our finding, of large reductions in vaccine-targeted HPV In this repeat cross-sectional study, we showed that prevalence types among adult women who were offered vaccine in the of cervical HPV types targeted by the quadrivalent vaccine catch-up program, is consistent with published ecological data has declined by 92% among women aged 18–35. Even among showing significant downward trends in genital warts diagnoses

HPV Vaccine Impact in Australian Women • JID 2018:217 (15 May) • 1595 Table 3. Crude Prevalence and Prevalence Ratios for HPV Detected among Australian Females Attending for Cervical Cytology Screening, Stratified by Vaccination Status in 2015a, Compared with the Prevaccine Sample, in Unadjusted and Adjusted Analyses

Crude Prevalence Comparison of Vaccine-Eligible Sample with Prevaccine Sample

n (%; 95% CI) PR (95% CI) P value aPRb (95% CI) P value HPV 6, 11, 16, 18 Prevaccine sample (2005–2007) 42 (15.3; 11.5–20.0) 1.00 (ref) 1.00 (ref) Vaccine-eligible sample (2015) Unvaccinated (n = 54) 1 (1.9; 0.3–12.2) 0.12 (0.02–0.86) .04 0.13 (0.02–0.91) .04 Partly vaccinated (n = 124) 2 (1.6; 0.4–6.3) 0.11 (0.03–0.43) .002 0.10 (0.02–0.41) .001 Fully vaccinated (n = 203) 2 (1.0; 0.2–3.9) 0.06 (0.02–0.26) <.001 0.06 (0.01–0.24) <.001 HPV 31, 33, 45, 52, 58 Prevaccine sample (2005–2007) 30 (10.9; 7.7–15.2) 1.00 (ref) 1.00 (ref)

Vaccine-eligible sample (2015) Unvaccinated (n = 54) 7 (13.0; 6.3–25.0) 1.19 (0.55–2.56) .66 1.23 (0.57–2.64) .60 Downloaded from https://academic.oup.com/jid/article-abstract/217/10/1590/4841780 by Yale University user on 12 March 2019 Partly vaccinated (n = 124) 11 (8.9; 5.0–15.4) 0.81 (0.42–1.57) .54 0.77 (0.40–1.48) .43 Fully vaccinated (n = 203) 29 (14.3; 10.1–19.8) 1.31 (0.81–2.11) .27 1.12 (0.68–1.83) .66 HR-HPV types other than 16 and 18 Prevaccine sample (2005–2007) 67 (24.4; 19.6–29.8) 1.00 (ref) 1.00 (ref) Vaccine-eligible sample (2015) Unvaccinated (n = 54) 8 (14.8; 7.5–27.1) 0.61 (0.31–1.19) .15 0.62 (0.32–1.20) .16 Partly vaccinated (n = 124) 22 (1 7. 7 ; 11.9–25.5) 0.73 (0.47–1.12) .15 0.70 (0.45–1.07) .10 Fully vaccinated (n = 203) 56 (27.6; 21.8–34.2) 1.13 (0.83–1.54) .43 0.98 (0.72–1.34) .92 Any HR-HPV type Prevaccine sample (2005–2007) 85 (30.9; 25.7–36.6) 1.00 (ref) 1.00 (ref) Vaccine-eligible sample (2015) Unvaccinated (n = 54) 9 (16.7; 8.8–29.2) 0.54 (0.29–1.00) .05 0.55 (0.30–1.01) .06 Partly vaccinated (n = 124) 23 (18.6; 12.6–26.4) 0.60 (0.40–0.90) .01 0.57 (0.38–0.85) .006 Fully vaccinated (n = 203) 57 (28.1; 22.3–34.7) 0.91 (0.68–1.20) .51 0.78 (0.59–1.04) .09 Any HPV type Prevaccine sample (2005–2007) 129 (46.9; 41.1–52.8) 1.00 (ref) 1.00 (ref) Vaccine-eligible sample (2015) Unvaccinated (n = 54) 15 (27.8; 17.4–41.3) 0.59 (0.38–0.93) .02 0.60 (0.38–0.94) .03 Partly vaccinated (n = 124) 29 (23.4; 16.7–31.7) 0.50 (0.35–0.70) <.001 0.49 (0.35–0.68) <.001 Fully vaccinated (n = 203) 79 (38.9; 32.4–45.8) 0.83 (0.67–1.03) .09 0.77 (0.62–0.95) .02 aWomen were classified as fully vaccinated if they had 3 doses of the vaccine recorded on the National HPV Vaccination Program Register. Unvaccinated women were those reporting not being vaccinated who also had no Register record of vaccination. If consent to access the Register was not obtained, self-report of nonvaccination was accepted. Women who had received 1 or 2 doses of vaccine as confirmed by the Register, or had self-reported doses that could not be verified on the Register, were classified as being partly vaccinated. bAdjusted for age, smoking status, and area of residence. Abbreviations: aPR adjusted PR; CI, confidence interval; HPV, human papillomavirus; HR-HPV, high-risk HPV; PR, prevalence ratio. and occurrence of high-grade cervical abnormalities among of vaccinated cohorts (both male and female) increase over women offered vaccine in the catch-up program [11, 28]. time, the transmission efficiency of vaccine HPV types in the Multiple factors are likely to be contributing to these reductions. population may be reduced to almost undetectable levels. First, high efficacy of the vaccine in preventing incident HPV Mathematical modeling suggests that elimination of HPV 6, infections among fully vaccinated women, including infections 11, 16, and 18 is possible if 80% coverage in girls and boys is newly acquired in the years since vaccination [29]. Additionally, reached, and if high vaccine efficacy is maintained over time evidence exists from both Australian and international data that [33]. In Australia, 3-dose vaccine coverage by age 15 for girls less than 3 doses of the vaccine may provide some protection and boys had in 2016 reached 79% and 73% respectively, with [8, 30–32], therefore suggesting that a proportion of potential data from the Register suggesting an increasing trend over time incident infections may have been prevented among partly vac- [6, 34]. Completion rates may improve further from 2018, when cinated women. Finally, the observed reductions are likely to a 2-dose schedule with a 9-valent vaccine replaces the current have occurred in part due to herd protection whereby trans- 3-dose schedule [35]. mission of the virus is interrupted, as vaccinated women do not We observed a higher crude prevalence of nonvaccine HPV acquire HPV from, or infect, unvaccinated men and these men types among fully vaccinated women, compared with those in turn do not transmit the virus to future unvaccinated female who were unvaccinated or partly vaccinated in the 2015 sam- partners [33]. ple. Type-replacement is unlikely to explain these differences A key result of the current analyses is our finding of the ongo- because both the relative ecological stability of HPV over time, ing reductions in vaccine-targeted HPV types among young and recent evidence from published data, argue against such a women aged 18–24, from 7% in 2010–2012, to less than 2% in development [36, 37]. Unmasking of HPV types in the absence the current sample. The results suggest that as the proportion of HPV16 is another possible explanation [38]. However, the

1596 • JID 2018:217 (15 May) • Machalek et al Table 4. Multivariate Analyses of Factors Associated with Cervical Human Papillomavirus (HPV) Detection among 381 Australian Females Aged 18–35 Years Attending for Routine Cervical Cytology Screening in 2015

aORa (95% CI); P value

HR-HPV Types Other Any HPV Type Any HR-HPV Type Than 16 and 18 HPV 31, 33, 45, 52, 58

Vaccination statusb Unvaccinated 1.00 (ref) 1.00 (ref) 1.00 (ref) 1.00 (ref) Partly vaccinated 0.68 (0.31–1.46); .32 1.07 (0.45–2.56); .87 1.17 (0.47–2.89); .73 0.60 (0.22–1.68); .33 Fully vaccinated 1.29 (0.62–2.64); .50 1.60 (0.71–3.59); .26 1.77 (0.76–4.13); .18 0.91 (0.36–2.29); .83 Age group 25–35 years 1.00 (ref) 1.00 (ref) 1.00 (ref) 1.00 (ref)

18–24 years 1.67 (1.04–2.69); .03 2.22 (1.31–3.76); .003 2.34 (1.37–4.01); .002 1.95 (0.99–3.84); .05 Downloaded from https://academic.oup.com/jid/article-abstract/217/10/1590/4841780 by Yale University user on 12 March 2019 Country of birth Other 1.00 (ref) … … … Australia 1.59 (0.83–3.05); .16 … … … Lifetime number of sexual partners 1–2 1.00 (ref) 1.00 (ref) 1.00 (ref) 1.00 (ref) 3–4 0.99 (0.38–2.56); .99 0.98 (0.33–2.93); .98 1.15 (0.37–3.52); .81 2.35 (0.41–13.40); .33 ≥5 3.84 (1.90–7.79); <.001 3.45 (1.55–7.66); .002 3.89 (1.68–9.01); .002 6.97 (1.63–29.78); .009 aFor each grouping of HPV, variables significant atP < .10 in univariate analyses (see Supplementary Table S2) as well as vaccination status and age group, were included in the multivariate model, except for partners in the previous 12 month, which was not included due to multicollinearity with age and lifetime number of partners. Detection of vaccine-targeted HPV types was not associated with any covariates tested. bWomen were classified as fully vaccinated if they had 3 doses of the vaccine recorded on the National HPV Vaccination Program Register. Unvaccinated women were those reporting not being vaccinated who also had no Register record of vaccination. If consent to access the Register was not obtained, self-report of nonvaccination was accepted. Women who had received 1 or 2 doses of vaccine as confirmed by the Register, or had self-reported doses that could not be verified on the Register, were classified as being partly vaccinated. Abbreviations: aOR, adjusted odds ratio; CI, confidence interval; HR-HPV, high-risk HPV; OR, odds ratio. extent and severity of this diagnostic artifact on detection of Another limitation of our study is that we did not collect data nonvaccine HPV types is not clear. This observed difference on sexual behavior in the prevaccine survey, and therefore could is almost certainly an age effect, reflecting underlying differ- not formally adjust for these risk factors of HPV infection across ences in the risk of exposure to HPV. Fully vaccinated women the study periods. In view of this, a better understanding of the in our study were significantly younger than partly vaccinated observed difference in nonvaccine types between the samples is or unvaccinated women, as younger women are more likely to limited by the lack of data on other factors between the groups have received vaccine in the school program, and therefore have (ie, sexual behavior). Nevertheless, we believe that these differ- completed vaccination doses [6, 34]. Furthermore, substantial ences are unlikely to fully explain the effects on vaccine-targeted research has demonstrated that HPV infection is most common HPV types for several reasons. First, the observed trends were in young women, who have higher numbers of new sexual part- driven by reductions in low-risk HPV types and the prevalence ners [39]. Our finding of the independent association between of nonvaccine high-risk HPV types did not differ significantly nonvaccine HPV types, younger age, and sexual behavior is between the groups. Furthermore, these reductions were rela- entirely consistent with this pattern. Furthermore, our conclu- tively small and of a different magnitude compared with that sions are in line with those of a recent US-based study in which found between samples for vaccine types (aRP = 0.70 [95% CI, similar increases in nonvaccine-type HPV prevalence in unvac- 0.57–0.85] vs aPR = 0.08 [95% CI, 0.03–0.20], respectively). Also, cinated women were noted [40]. the lower prevalence of smokers in this study is consistent with A strength of our study is the repeat cross-sectional design and national data showing declining rates of smoking among this age the assessment of HPV prevalence across 3 time points among group over time [41]. Additionally, where data on risk behavior women in the younger age group, extending the results of a pre- were available (for example among 18 to 24-year-old women vious study [13]. Additionally, a high proportion of participants in the 2 postvaccine samples), no significant differences in the gave us permission to obtain HPV vaccination status through the risk of exposure to HPV were noted. Second, the low prevalence Register. However, the study also has limitations. Findings from rates reported in our study are consistent with those reported in a population-based survey suggest that coverage rates for women 2 recent Australian studies, including the COMPASS trial [42] vaccinated in the community (at age 18–26 years) are underesti- and a surveillance study of young Australian women recruited mated by 5%–15% on the Register [4]. It is therefore likely that through Facebook [43]. Another limitation is that the study sam- that the true coverage rates among women aged 25–35 years, who ple size was limited and was not powered to detect any potential would largely have received their vaccines in the community, is cross-protection of the vaccine on related HPV types. As such, higher than the estimate reported in this study. we could not examine any association between the vaccine and

HPV Vaccine Impact in Australian Women • JID 2018:217 (15 May) • 1597 change in prevalence of HPV 31, 33, and 45 as observed in the honoraria to her institute from Merck Sharp & Dohme (MSD), first repeat cross-sectional study [13]. Recent evidence is com- outside the submitted work. S. M. G. reports grants from Merck, patible with the absence of any sizeable long-lasting effects 44[ ]. GSK, CSL, Commonwealth Department of Health, nonfinancial Finally, given the sentinel clinic-based design, the results may support from Merck, outside the submitted work; and has deliv- not be generalizable to all Australian women. Nevertheless, this ered lectures and received speaking fees from MSD and Sanofi type of study design does not aim to be representative, but rather Pasteur for work performed in her personal time. J. M. B. and S. N. reproducible, to allow for the detection of changes over time in T. have been investigators on investigator-designed unrestricted similar populations. epidemiological research grants partially funded through bioCSL In summary, this repeat cross-sectional study demonstrates (cancer typing study) and Merck (recurrent respiratory papillo- a marked decline in quadrivalent vaccine-targeted HPV types matosis study) but have received no personal financial benefits. among women up to the age of 35 years since program imple- S. R. S.’s institution has received funds for investigator driven HPV mentation. Continued surveillance is needed to determine if attitudinal and educational research from bioCSL. B. L. owns Downloaded from https://academic.oup.com/jid/article-abstract/217/10/1590/4841780 by Yale University user on 12 March 2019 these results are sustained or improved in the future. Ongoing shares in CSL, a vaccine distributor in Australia, outside the sub- monitoring will also serve to assess the impact of 2 doses of the mitted work. A. M. C. has received travel funding from Seqirus. 9-valent vaccine. J. M. K. reports grants from Australian Government Department of Health during the conduct of the study. All authors have sub- Supplementary Data mitted the ICMJE Form for Disclosure of Potential Conflicts of Supplementary materials are available at The Journal of Infectious Interest. Conflicts that the editors consider relevant to the content Diseases online. Consisting of data provided by the authors to of the manuscript have been disclosed. benefit the reader, the posted materials are not copyedited and are the sole responsibility of the authors, so questions or com- References ments should be addressed to the corresponding author. 1. de Sanjose S, Alemany L, Ordi J, et al. Worldwide human papillomavirus genotype attribution in over 2000 cases Notes of intraepithelial and invasive lesions of the vulva. Eur J Author contributions. D. A. M. designed the study, per- Cancer 2013; 49:3450–61. formed the data analyses and drafted the manuscript. J. M. 2. Forman D, de Martel C, Lacey CJ, et al. Global burden of B. and J. M. K. contributed to the study design, data interpre- human papillomavirus and related diseases. Vaccine 2012; tation, and writing of the report. D. B., K. M., and M. S. con- 30(Suppl 5):F12–23. tributed to the study design, participant recruitment, and data 3. Lu B, Kumar A, Castellsagué X, Giuliano AR. Efficacy and interpretation. S. M. G., S. R. S., B. L., A. M. C., and S. N. T. con- safety of prophylactic vaccines against cervical HPV infec- tributed to the study design and data interpretation. All authors tion and diseases among women: a systematic review and reviewed the manuscript for important intellectual content and meta-analysis. BMC Infect Dis 2011; 11:13. approved the final version of the manuscript. 4. Brotherton JM, Liu B, Donovan B, Kaldor JM, Saville M. Acknowledgments. We would like to thank the study Human papillomavirus (HPV) vaccination coverage in research nurses, Lorraine Edney, Alison Beverley, and Mandy young Australian women is higher than previously esti- Johnson, for their pivotal role in data collection, participant mated: independent estimates from a nationally represen- recruitment, and follow-up. We also thank Jennifer Brosi, Lisette tative mobile phone survey. Vaccine 2014; 32:592–7. Bicknell, and Chantal Kim at the National HPV Vaccination 5. Brotherton JM, Murray SL, Hall MA, et al. Human papil- Program Register for assistance with validation of vaccination lomavirus vaccine coverage among female Australian ado- status, and Michael Malloy from the Victorian Cytology Service lescents: success of the school-based approach. Med J Aust for his assistance with the statistical analyses. Finally, we thank 2013; 199:614–7. all the study participants. 6. Brotherton JM, Winch KL, Bicknell L, Chappell G, Saville Disclaimer. The views expressed in this publication are M. HPV vaccine coverage is increasing in Australia. Med J those of the authors and do not necessarily represent the posi- Aust 2017; 206:262. tion of the Australian Government. 7. Chow EP, Danielewski JA, Fehler G, et al. Human papil- Financial support. This work was supported by the lomavirus in young women with Chlamydia trachomatis Australian Government Department of Health HPV Surveillance infection 7 years after the Australian human papillomavi- Fund (grant number H1314G010). J. M. K. and B. L. hold rus vaccination programme: a cross-sectional study. Lancet National Health and Medical Research Council Fellowships. Infect Dis 2015; 15:1314–23. Potential conflicts of interest. D. A. M. reports grants from 8. Crowe E, Pandeya N, Brotherton JM, et al. Effectiveness Australian Government Department during the conduct of the of quadrivalent human papillomavirus vaccine for the study, travel grants from Seqirus (bioCSL), travel funding, and prevention of cervical abnormalities: case-control study

1598 • JID 2018:217 (15 May) • Machalek et al nested within a population based screening programme in cancer specimens by DNA amplification with consensus Australia. BMJ 2014; 348:g1458. primers. J Natl Cancer Inst 1990; 82:1477–84. 9. Gertig DM, Brotherton JM, Budd AC, Drennan K, Chappell 22. Tabrizi SN, Stevens M, Chen S, Rudland E, Kornegay JR, G, Saville AM. Impact of a population-based HPV vacci- Garland SM. Evaluation of a modified reverse line blot assay nation program on cervical abnormalities: a data linkage for detection and typing of human papillomavirus. Am J study. BMC Med 2013; 11:227. Clin Pathol 2005; 123:896–9. 10. Donovan B, Franklin N, Guy R, et al. Quadrivalent human 23. Gravitt PE, Peyton CL, Alessi TQ, et al. Improved amplifi- papillomavirus vaccination and trends in genital warts in cation of genital human papillomaviruses. J Clin Microbiol Australia: analysis of national sentinel surveillance data. 2000; 38:357–61. Lancet Infect Dis 2011; 11:39–44. 24. Layton-Henry J, Scurry J, Planner R, et al. Cervical ade- 11. Brotherton JM, Gertig DM, May C, Chappell G, Saville noid basal carcinoma, five cases and literature review. Int J M. HPV vaccine impact in Australian women: ready for Gynecol Cancer 1996; 6:193–9. Downloaded from https://academic.oup.com/jid/article-abstract/217/10/1590/4841780 by Yale University user on 12 March 2019 an HPV-based screening program. Med J Aust 2016; 25.Tabrizi SN, Brotherton JM, Stevens MP, et al. HPV geno- 204:184–184e1. typing prevalence in Australian women undergoing routine 12. Ali H, Donovan B, Wand H, et al. Genital warts in young cervical screening by cytology status prior to implementation Australians five years into national human papillomavirus of an HPV vaccination program. J Clin Virol 2014; 60:250–6. vaccination programme: national surveillance data. BMJ 26. Stevens MP, Garland SM, Tabrizi SN. Development and 2013; 346:f2032. validation of a real-time PCR assay specifically detecting 13. Tabrizi SN, Brotherton JM, Kaldor JM, et al. Assessment human papillomavirus 52 using the Roche LightCycler 480 of herd immunity and cross-protection after a human pap- system. J Virol Methods 2008; 147:290–6. illomavirus vaccination programme in Australia: a repeat 27. Joura EA, Giuliano AR, Iversen OE, et al.; Broad Spectrum cross-sectional study. Lancet Infect Dis 2014; 14:958–66. HPV Vaccine Study. A 9-valent HPV vaccine against infec- 14. Tabrizi SN, Brotherton JM, Kaldor JM, et al. Fall in human tion and intraepithelial neoplasia in women. N Engl J Med papillomavirus prevalence following a national vaccination 2015; 372:711–23. program. J Infect Dis 2012; 206:1645–51. 28. Smith MA, Liu B, McIntyre P, Menzies R, Dey A, Canfell 15. Brotherton JM, Condon JR, McIntyre PB, et al. Human pap- K. Fall in genital warts diagnoses in the general and indige- illomavirus prevalence to age 60 years among Australian nous Australian population following implementation of a women prevaccination. Sex Health 2015; 12:353–9. national human papillomavirus vaccination program: anal- 16. Garland SM, Brotherton JM, Condon JR, et al.; WHINURS ysis of routinely collected national hospital data. J Infect Dis study group. Human papillomavirus prevalence among 2015; 211:91–9. indigenous and non-indigenous Australian women prior to 29. Castellsagué X, Muñoz N, Pitisuttithum P, et al. End-of- a national HPV vaccination program. BMC Med 2011; 9:104. study safety, immunogenicity, and efficacy of quadrivalent 17. Gertig DM, Brotherton JM, Saville M. Measuring human HPV (types 6, 11, 16, 18) recombinant vaccine in adult papillomavirus (HPV) vaccination coverage and the role of women 24-45 years of age. Br J Cancer 2011; 105:28–37. the National HPV Vaccination Program Register, Australia. 30. Brotherton JML, Malloy M, Budd AC, Saville M, Drennan Sex Health 2011; 8:171–8. KT, Gertig DM. Effectiveness of less than three doses of 18. Martínez SB, Palomares JC, Artura A, et al. Comparison of quadrivalent human papillomavirus vaccine against cer- the Cobas 4800 Human Papillomavirus test against a com- vical intraepithelial neoplasia when administered using bination of the Amplicor Human Papillomavirus and the a standard dose spacing schedule: Observational cohort Linear Array tests for detection of HPV types 16 and 18 in of young women in Australia. Papillomavirus Res 2015; cervical samples. J Virol Methods 2012; 180:7–10. 1(Suppl C): 59–73. 19. Phillips S, Cornall AM, Machalek DA, et al. Comparison 31. Lamb F, Herweijer E, Ploner A, et al. Timing of two versus of the Roche Cobas® 4800 HPV assay to Roche Amplicor three doses of quadrivalent HPV vaccine and associated for detection of high-risk human papillomavirus. Eur J Clin effectiveness against condyloma in Sweden: a nationwide Microbiol Infect Dis 2016; 35:1305–7. cohort study. BMJ Open 2017; 7:e015021. 20. Stevens MP, Rudland E, Garland SM, Tabrizi SN. Assessment 32. Sankaranarayanan R, Prabhu PR, Pawlita M, et al.; Indian of MagNA pure LC extraction system for detection of HPV Vaccine Study Group. Immunogenicity and HPV human papillomavirus (HPV) DNA in PreservCyt samples infection after one, two, and three doses of quadrivalent by the Roche AMPLICOR and LINEAR ARRAY HPV tests. HPV vaccine in girls in India: a multicentre prospective J Clin Microbiol 2006; 44:2428–33. cohort study. Lancet Oncol 2016; 17:67–77. 21. Resnick RM, Cornelissen MT, Wright DK, et al. Detection 33. Brisson M, Bénard É, Drolet M, et al. Population-level and typing of human papillomavirus in archival cervical impact, herd immunity, and elimination after human

HPV Vaccine Impact in Australian Women • JID 2018:217 (15 May) • 1599 papillomavirus vaccination: a systematic review and acquisition and persistence in Guanacaste, Costa Rica. J meta-analysis of predictions from transmission-dynamic Infect Dis 2005; 191:1808–16. models. Lancet Public Health 2016; 1:e8–e17. 40. Ding L, Widdice LE, Kahn JA. Differences between vacci- 34. National HPV Vaccination Program Register. National nated and unvaccinated women explain increase in non-vac- (Australia) HPV three-dose vaccine coverage for adoles- cine-type human papillomavirus in unvaccinated women cents turning 15 years of age. http://www.hpvregister.org. after vaccine introduction. Vaccine2017 ; 35:7217–21. au/research/coverage-data. Accessed 21 November 2017. 41. Greenhalgh EM, Bayly M, Winstanley MH. 1.4 Prevalence 35. Prime Minister of Australia. Announcement of the addi- of smoking—young adults. In: Scollo MM, Winstanley MH, tion of Gardasil-9 to the national immunisation program. eds. Tobacco in Australia: Facts and issues. Melbourne: https://www.pm.gov.au/media/2017-10-08/announce- Cancer Council Victoria; 2015. http://www.tobaccoin- ment-addition-gardasil-9-national-immunisation-pro- australia.org.au/chapter-1-prevalence/1-4-prevalence-of- gram. Accessed 2 November 2017. smoking-young-adults. Accessed 2 November 2017. Downloaded from https://academic.oup.com/jid/article-abstract/217/10/1590/4841780 by Yale University user on 12 March 2019 36. Chen Z, DeSalle R, Schiffman M, Herrero R, Burk RD. 42. Canfell K, Caruana M, Gebski V, et al. Cervical screening Evolutionary dynamics of variant genomes of human papil- with primary HPV testing or cytology in a population of lomavirus types 18, 45, and 97. J Virol 2009; 83:1443–55. women in which those aged 33 years or younger had previ- 37. Tota JE, Struyf F, Merikukka M, et al. Evaluation of type ously been offered HPV vaccination: Results of the Compass replacement following HPV16/18 vaccination: pooled pilot randomised trial. PLoS Med 2017; 14:e1002388. analysis of two randomized trials. J Natl Cancer Inst 2017; 43. Osborne SL, Tabrizi SN, Brotherton JM, et al.; VACCINE 109:djw300. Study Group. Assessing genital human papillomavirus 38. Tota JE, Ramanakumar AV, Villa LL, et al. Evaluation of genoprevalence in young Australian women following the human papillomavirus type replacement postvaccination introduction of a national vaccination program. Vaccine must account for diagnostic artifacts: masking of HPV52 2015; 33:201–8. by HPV16 in anogenital specimens. Cancer Epidemiol 44. Mesher D, Soldan K, Lehtinen M, et al. Population-level Biomarkers Prev 2015; 24:286–90. effects of human papillomavirus vaccination programs on 39. Castle PE, Schiffman M, Herrero R, et al. A prospective infections with nonvaccine genotypes. Emerg Infect Dis study of age trends in cervical human papillomavirus 2016; 22:1732–40.

1600 • JID 2018:217 (15 May) • Machalek et al Vaccine 36 (2018) 1781–1788

Contents lists available at ScienceDirect

Vaccine

journal homepage: www.elsevier.com/locate/vaccine

Post-licensure safety monitoring of quadrivalent human papillomavirus vaccine in the Vaccine Adverse Event Reporting System (VAERS), 2009–2015 ⇑ Jorge E. Arana a, , Theresa Harrington a, Maria Cano a, Paige Lewis a, Adamma Mba-Jonas b, Li Rongxia a, Brock Stewart a,1, Lauri E. Markowitz c, Tom T. Shimabukuro a a Immunization Safety Office, Division of Healthcare Quality Promotion, National Center for Emerging and Zoonotic Infectious Diseases, Centers for Disease Control and Prevention, United States b Division of Epidemiology, Office of Biostatistics and Epidemiology, Center for Biologics Evaluation and Research, US Food and Drug Administration, United States c Division of Viral Diseases, National Center for Immunization and Respiratory Diseases, Centers for Disease Control and Prevention, United States article info abstract

Article history: Background: The Food and Drug Administration (FDA) approved quadrivalent human papillomavirus vac- Received 21 April 2017 cine (4vHPV) for use in females and males aged 9–26 years, since 2006 and 2009 respectively. We char- Received in revised form 16 October 2017 acterized reports to the Vaccine Adverse Event Reporting System (VAERS), a US spontaneous reporting Accepted 5 February 2018 system, in females and males who received 4vHPV vaccination. Available online 21 February 2018 Methods: We searched VAERS for US reports of adverse events (AEs) following 4vHPV from January 2009 through December 2015. Signs and symptoms were coded using Medical Dictionary for Regulatory Keywords: Activities (MedDRA). We calculated reporting rates and conducted empirical Bayesian data mining to Quadrivalent human papillomavirus vaccine identify disproportional reports. Clinicians reviewed available information, including medical records, (4vHPV) Vaccination and reports of selected pre-specified conditions. Immunization Findings: VAERS received 19,760 reports following 4vHPV; 60.2% in females, 17.2% in males, and in 22.6% Vaccine safety sex was missing. Overall, 94.2% of reports were non-serious; dizziness, syncope and injection site reac- Vaccine Adverse Event Reporting System tions were commonly reported in both males and females. Headache, fatigue and nausea were commonly (VAERS) reported serious AEs. More than 60 million 4vHPV doses were distributed during the study period. Crude AE reporting rates were 327 reports per million 4vHPV doses distributed for all reports, and 19 per mil- lion for serious reports. Among 29 verified reports of death, there was no pattern of clustering of deaths by diagnosis, co-morbidities, age, or interval from vaccination to death. Interpretation: No new or unexpected safety concerns or reporting patterns of 4vHPV with clinically important AEs were detected. Safety profile of 4vHPV is consistent with data from pre-licensure trials and postmarketing safety data. Published by Elsevier Ltd.

1. Introduction females 9–26 years old [1]. In 2009, 4vHPV was approved for males 9–26 years old [2,3]. 4vHPV is indicated for prevention of vaccine Human papillomavirus quadrivalent (types 6, 11, 16, 18) type-associated cervical and other anogenital cancers, neoplasias recombinant vaccine, GardasilÒ (4vHPV), was licensed by the US and warts and was recommended as a three dose series over a 6- Food and Drug Administration (FDA) in 2006 and approved for month period [4–6]. In pre-licensure clinical trials, injection site pain and mild systemic reactions occurred most commonly [4]. The Advisory Committee on Immunization Practices recommends ⇑ Corresponding author at: Immunization Safety Office, Centers for Disease routine HPV vaccination beginning at 11 or 12 years of age, since Control and Prevention (CDC), 1600 Clifton Road, MS D-26, Atlanta, GA 30329, 2006 for females and since 2011 for males [5,6]. United States. A review of the first 2.5 years (June 2006 through December E-mail address: [email protected] (J.E. Arana). 2008) of 4vHPV post-licensure safety monitoring in the Vaccine 1 Present address: Division of Tuberculosis Elimination, National Center for HIV/ AIDS, Viral Hepatitis, STD, and TB Prevention, Centers for Disease Control and Adverse Event Reporting System (VAERS), after approximately 23 Prevention, United States. million doses had been distributed, revealed higher than expected https://doi.org/10.1016/j.vaccine.2018.02.034 0264-410X/Published by Elsevier Ltd. 1782 J.E. Arana et al. / Vaccine 36 (2018) 1781–1788 reporting for venous thromboembolism and syncope [7]; but gen- 2.3. Outcome measures and statistical methods erally, no other potential safety signals were observed. Subsequent epidemiologic studies in the United States and Europe did not 2.3.1. Data analysis detect an increased risk of venous thromboembolism following We summarized basic characteristics of reports by severity, 4vHPV [8–10]. Studies assessing autoimmune and neurological type of reporter, patient age and AE onset interval. We stratified conditions after 4vHPV have also been consistent with prelicensure by sex and assessed the most common MedDRA Preferred Terms data [8–13]. A study in France found an apparently increased risk for all 4vHPV reports and for reports where 4vHPV was adminis- of Guillain-Barré syndrome (GBS) following 4vHPV vaccination tered alone. Pre-specified conditions for clinical review were cho- [14]. However, a recent study in the US vaccine safety datalink sen based on information from previous post-licensure studies found no evidence of an increased risk of GBS following 4vHPV and surveillance, and from public concern about specific AEs [15]. Syncope has been previously reported and the National Acad- [7–15]. We calculated crude AE reporting rates for the following emy of Medicine concluded that syncope can result from any pre-specified conditions: syncope, venous thromboembolism, ana- injected vaccine [16]. phylaxis, a composite of selected autoimmune disorders, postural From 2009 through 2015, 4vHPV accounted for most (over 90%) orthostatic tachycardia syndrome (POTS), complex regional pain of the human papillomavirus vaccine distributed in the United syndrome (CRPS), Guillain-Barré syndrome (GBS), and death States [17,18]. We analyzed reports submitted to VAERS following (Appendix). 4vHPV from 2009 through 2015 to provide updated safety infor- mation, assessed selected rare adverse events (AE) that have more 2.3.2. 4vHPV adverse event reporting rates recently emerged as alleged safety concerns, and included data for We calculated crude 4vHPV AE reporting rates for all reports males, which was lacking in the first VAERS review. and serious reports by dividing the number of reports by 4vHPV doses distributed in the United States from 2009 through 2015; 2. Methods we also graphically depicted annual crude reporting rates from 2006 through 2015 to display historical reporting trends. Crude 2.1. Study population reporting rates were calculated for the pre-specified conditions. Because we were not able to determine doses distributed for a par- VAERS is a national spontaneous reporting system for AEs fol- ticular sex, we could not calculate a reporting rate by sex for any lowing US-licensed vaccines [19]. It is co-administered by the Cen- diagnosis or for any sex-specific conditions. ters for Disease Control and Prevention (CDC) and the FDA. VAERS accepts reports from patients, parents, healthcare providers, vac- 2.3.3. Clinical review of reports of selected pre-specified conditions cine manufacturers, and others. The VAERS reporting form collects Physicians reviewed all reports of GBS, anaphylaxis, CRPS, pri- information on the vaccinated individual, vaccines administered mary ovarian insufficiency (POI), POTS, and death (Appendix). and the AE itself. Signs and symptoms of the AE are coded using Reports suggestive of GBS and anaphylaxis were classified accord- the Medical Dictionary for Regulatory Activities (MedDRA), a clin- ing to Brighton Collaboration criteria [22,23]. Cause of death was ically validated, internationally standardized terminology [20].A determined by autopsy report, death certificate or medical records. single VAERS report may be assigned more than one MedDRA Pre- Syncope, venous thromboembolism, and reports in pregnant ferred Term. Preferred terms are not necessarily medically- women and of infants born to mothers who were pregnant at the confirmed diagnoses. Reports are classified as ‘‘serious” based on time of 4vHPV receipt have been previously published, and the US Code of Federal Regulations if any of the following are doc- selected autoimmune disorders have been extensively studied in umented: hospitalization, prolongation of existing hospitalization, databases that allow for calculation of incidence rates [7–13,15,1 permanent disability, life-threatening illness, or death [21]. Except 6,24–27]; therefore, this summary will not focus on those for reports submitted by vaccine manufacturers, medical records conditions. are requested for reports designated as serious; for reports of death, autopsy reports and death certificates are also requested to ascertain cause of death. Manufacturers are responsible for con- 2.3.4. Empirical Bayesian data mining ducting appropriate follow-up on their reports. We used published methods and criteria [28–30] to identify We included VAERS reports following 4vHPV for persons vacci- 4vHPV-AE pairs reported at least twice as frequently as expected nated from 2009 through 2015 and received by January 31, 2016, (i.e., lower bound of the 90% confidence interval surrounding the  to account for reporting and data lags. For reports with missing empirical Bayesian geometric mean [EB05 2]) compared to all vaccination date, we included those received from 2009 through other US-licensed vaccines. After assessing findings for biological 2015 and imputed vaccination date based on historical lag times. plausibility and clinical significance, we reviewed reports with We excluded foreign source (non-US) reports. VAERS is used to AEs that exceeded this data mining threshold and had not previ- conduct routine surveillance as a public health function; therefore, ously been assessed or identified and characterized in the pre- it is not subject to Institutional Review Board review and informed licensure clinical trials or other post-marketing studies. consent requirements. 3. Results 2.2. Study design From 2009 through 2015, VAERS received 19,760 US reports fol- We conducted descriptive analyses of VAERS data, calculated lowing 4vHPV (Table 1). Females accounted for 11,894 (60.2%) crude 4vHPV AE reporting rates based on vaccine doses dis- reports, males 3391 (17.2%), and in 4475 (22.6%) sex was unknown tributed, performed clinical review of reports for selected pre- or not reported. Overall, 94.2% of reports were non-serious; most specified conditions of interest, and conducted empirical Bayesian reports (40.7%) were in the persons aged 11–17 years. Most reports data mining to identify AEs reported more frequently than came from vaccine manufacturers (56.6%) followed by healthcare expected following 4vHPV compared to other US-licensed vac- providers (28.1%). Median time from receipt of 4vHPV to start of cines. Dose number in a vaccination series is often missing or symptoms was on the day of vaccination and ranged up to 5 years. inconsistently reported in VAERS; therefore, we did not analyze The 90th percentile for symptom onset for all reports was within 4vHPV data by dose number. 2 weeks of vaccination. In 28.3% of reports other vaccines were J.E. Arana et al. / Vaccine 36 (2018) 1781–1788 1783

Table 1 Characteristics of human papillomavirus quadrivalent vaccine (4vHPV) reports, Vaccine Adverse Event Reporting System (VAERS), 2009–2015*.

Report characteristics Female Male Unknown sex Total reports N (%) N (%) N (%) N (%) Total reports 11,894 3391 4475 19,760 Serious reportsa,b 911 (7.7) 189 (5.6) 38 (0.8) 1138 (5.8) 4vHPV given alone 8211 (69.0) 1643 (48.5) 4323 (96.6) 14,177 (71.7) Type of reporter Healthcare provider 3793 (31.9) 1735 (51.2) 22 (0.5) 5550 (28.1) Manufacturer 5786 (48.6) 985 (29.0) 4412 (98.6) 11,183 (56.6) Patient/parent 947 (8.0) 215 (6.3) 3 (0.1) 1165 (5.9) Other 1368 (11.5) 456 (13.4) 38 (0.8) 1862 (9.4) Age groups (years) <9c 42 (0.4) 34 (1.0) 7 (0.2) 83 (0.4) 9–10 152 (1.3) 55 (1.6) 11 (0.2) 218 (1.1) 11–17 5384 (45.3) 2236 (65.9) 421 (9.4) 8041 (40.7) 18–26 2439 (20.5) 372 (11.0) 70 (1.6) 2881 (14.6) >26c 173 (1.5) 24 (0.7) 4 (0.1) 201 (1.0) Not reported or unknown 3704 (31.1) 670 (19.8) 3962 (88.5) 8336 (42.2) Adverse event onset (days)d Median 0 0 0 0 Range 0–1649 0–1826 0–720 0–1826 90th percentile 14 4 0 7

a Includes death, life-threatening illness, hospitalization or prolongation of existing hospitalization, or permanent disability as defined in 21CFR600.80 [21]. b Includes 56 reports of death in females, 12 reports of death in males and 24 reports of death in individuals of unknown sex. c 4vHPV is not approved for these age groups. d Onset interval in days from time of vaccination (day 0) to first adverse event symptoms. * Reports received by January 31, 2016, a standard lag time used for 4vHPV surveillance using VAERS data. given concomitantly with 4vHPV. The most frequently 4.1.2. Anaphylaxis co-administered vaccines were meningococcal conjugate (n = 3212 We identified 38 reports of anaphylaxis. Most (n = 33, 86.8%) reports); tetanus and diphtheria (Td) or tetanus, diphtheria and were in females. In 25 (65.8%), 4vHPV was given alone; the remain- pertussis (Tdap) (n = 2440); and hepatitis A (n = 1754). ing reports involved concomitant administration of other vaccines, Dizziness and syncope were the most commonly reported non- including meningococcal conjugate (n = 10), Tdap (n = 6), varicella serious AEs in both males and females (Table 2). Injection site reac- (n = 4), hepatitis A (n = 3), and quadrivalent live attenuated influ- tions (i.e., various combinations of pain, swelling and erythema) enza (n = 1). Seven reports met Brighton Collaboration criteria were noted in 10.9% of female and 14.9% of male reports. Headache Level 1 and three met Level 2 [23]. Five of these cases had history (32.1%), fatigue (24.2%) and nausea (23.9%) were commonly of non-anaphylaxis hypersensitivity reactions to food, medication, reported serious AEs. latex, or environmental allergens. The remaining reports did not meet Brighton Collaboration criteria or did not contain sufficient 4. 4vHPV adverse event reporting rates information to make a determination for anaphylaxis. In 20 of 26 reports where onset interval was documented, symptoms occurred During the study period, 60,461,220 4vHPV doses were dis- on the day of vaccination. tributed in the United States (number provided by manufacturer, Merck, Inc.). Crude reporting rates to VAERS were 327 total 4vHPV 4.1.3. Complex regional pain syndrome (CRPS) reports per million doses distributed, and 19 serious 4vHPV reports We identified 17 reports of CRPS or the synonymous condition per million doses distributed (Table 3). The crude reporting rate for reflex sympathetic dystrophy (RSD). In two reports, the patient syncope following 4vHPV4 was 47 per million. For all other pre- experienced onset of pain or paresthesias in the same arm that specified conditions, rates were <3 per million. Annual reporting 4vHPV was given. In 12 reports, the patient experienced CRPS rates peaked in 2007 and 2008 (dates outside of the current study symptoms in parts of the body unassociated with receipt of a vac- period) at around 600 reports per million doses administered and cine, including legs, ankles, feet, hip, shins, and chest wall; symp- declined through 2010, subsequently leveling off at around 250 toms were often associated with exercise or trauma (e.g., bone to 300 reports per million (Fig. 1). fracture). In three additional reports, timing of onset of CRPS symp- toms preceded vaccination (1 report) or was not documented (2 4.1. Clinical review of reports of pre-specified conditions reports). Twelve CRPS reports (70.6%) were classified as serious; most (n = 15, 88.2%) were in females. Median age was 15 years 4.1.1. Guillain-Barré syndrome (GBS) (range, 12–19 years) and median onset interval was 7 days (range, We identified 59 reports of GBS, 45 of which did not meet 0–56 days). In 13 reports, 4vHPV was given alone. In four reports, Brighton Collaboration criteria for a diagnosis of GBS or did not vaccines concomitantly administered included meningococcal con- contain sufficient information to make a determination. The jugate, Tdap and inactivated influenza. Co-morbid conditions listed remaining 14 (23.7%) reports of GBS met either Level 1 (n = 5; in the medical history of serious CRPS reports included juvenile highest level of diagnostic certainty) or Level 2 (n = 9) Brighton rheumatoid arthritis, POTS, and fibromyalgia. Collaboration criteria for GBS [22]. Three of these 14 reports describe a viral respiratory illness 2 to 4 weeks prior to presenta- tion of GBS symptoms. Most (n = 10, 71.4%) reports were in 4.1.4. Primary ovarian insufficiency (POI) females. The median onset interval from vaccination to start of We identified 17 reports that met search criteria for POI, also neurologic symptoms in these 14 cases was 21 days (range 2– known as premature ovarian failure; however, 15 reports did not 200 days). 4vHPV was given alone in 6 (42.9%) reports. contain sufficient information to confirm a POI diagnosis. Many 1784 J.E. Arana et al. / Vaccine 36 (2018) 1781–1788

Table 2 Most commonly reported adverse eventsa following human papillomavirus quadrivalent vaccine (4vHPV), Vaccine Adverse Event Reporting System (VAERS), 2009–2015.

All 4vHPV reports N (%) N (%) Females Non-serious 10,983 Seriousb 911 Dizziness 1472 (13.4) Headache 292 (32.1) Syncope 1454 (13.2) Fatigue 220 (24.2) Headache 1083 (9.9) Nausea 218 (23.9) Nausea 987 (9.0) Dizziness 208 (22.8) Loss of consciousness 698 (6.4) Pain 147 (16.1) Reports where 4vHPV was given alone N (%) N (%) Non-serious 7503 Seriousb 708 Dizziness 871 (11.6) Headache 206 (29.1) Syncope 807 (10.8) Fatigue 164 (23.2) Headache 784 (10.5) Nausea 153 (21.6) Nausea 647 (8.6) Dizziness 149 (21.1) Inappropriate schedule of drug administration 557 (7.4) Activities of daily living impaired 113 (16.0) Males Non-serious 3202 Seriousb 189 Dizziness 501 (15.7) Headache 58 (30.7) Syncope 457 (14.3) Nausea 48 (25.4) Injection site erythema 328 (10.2) Pyrexia 46 (24.3) Injection site swelling 280 (8.7) Fatigue 39 (20.6) Pallor 253 (7.9) Vomiting 38 (20.1) Reports where 4vHPV was given alone N (%) N (%) Non-serious 1525 Seriousb 118 Dizziness 220 (14.4) Headache 33 (28.0) Syncope 210 (13.8) Nausea 24 (20.3) Inappropriate schedule of drug administration 171 (11.2) Pyrexia 24 (20.3) Headache 133 (8.7) Fatigue 21 (17.8) Pyrexia 87 (5.7) Vomiting 21 (17.8)

a Based on Medical Dictionary for Regulatory Activities (MedDRA) Preferred Terms; a single report may be assigned more than one MedDRA Preferred Term (i.e., not mutually exclusive). b As defined in 21CFR600.80 [21].

Table 3 date was diagnosed with POI 1.5 weeks after vaccination; (2) A Reports of pre-specified conditions and crude reporting rates following human 16-year-old female, also with amenorrhea of unspecified start date, papillomavirus quadrivalent (4vHPV), Vaccine Adverse Event Reporting System (VAERS), 2009–2015. was diagnosed with POI and hypogonadotropic hypogonadism within a month of vaccination. 4vHPV was the only vaccine given a Report type or adverse event N (Reporting rate ) in these two reports. Report type All reports 19,760 (327) Serious reportsb 1138 (19) 4.1.5. Postural orthostatic tachycardia syndrome (POTS) Adverse eventc,d We identified 69 reports of POTS, of which 11 (15.9%) met the Syncope 2845 (47) clinical diagnostic criteria for the condition [31]. Of these, 8 Autoimmune disorders 164 (2.7) (72.7%) were in females; median age was 15 years (range 12 to e Death 92 (1.5) 22 years), and median onset interval was 42 days (range 0 to e Postural orthostatic tachycardia syndrome 69 (1.1) 110 days). Co-morbid conditions commonly reported included Guillain-Barré syndromee 59 (0.98) Anaphylaxise 38 (0.63) asthma, chronic headache, vasovagal syncope, chronic fatigue, Venous thromboembolism 38 (0.63) celiac disease, and CRPS. Of the remaining reports, 27 (39.1%) did Complex regional pain syndrome 17 (0.28) not meet all diagnostic criteria and 31 (44.9%) did not contain suf- e f Primary ovarian insufficiency 17 (N/A) ficient information to confirm a diagnosis of POTS. A detailed anal- a Reports of adverse events per 1 million 4vHPV doses distributed; estimated ysis of VAERS POTS reports is described in a separate review [24]. 60,461,220 4vHPV doses distributed in the United States from 2009 through 2015 (personal communication with Merck Inc.). b As defined in 21CFR600.80 [21]. 4.1.6. Deaths c A single report may contain more than one adverse event (i.e., not mutually We identified 92 reports of death. Sixty-one were hearsay exclusive). reports which included no medical information that could be ver- d See Appendix for Medical Dictionary for Regulatory Activities (MedDRA) Pre- ified. Two additional reports mentioned a cause of death (‘‘pontine ferred Terms used to identify pre-specified conditions. ... e Confirmed cases following clinical review of reports: death (n = 29), postural glioma”; ‘‘died from bone cancer ”), but there was no patient or orthostatic tachycardia syndrome (n = 11), Guillain-Barré syndrome (n = 14), ana- contact information provided. We were able to verify 29 reports phylaxis (n = 10), primary ovarian insufficiency (n = 2). of death from autopsy reports, death certificates, or medical f Not able to calculate reporting rates due to the inability to determine doses records. Of these verified death reports, the median age was 16 distributed for female versus male use. years (range 10–37 years), median time from vaccination was 13 days (range 0–968 days), 20 (69.0%) were in females, and 9 were ‘‘hearsay reports”, i.e., based on indirect information heard by (31.0%) were in males. In 13 (44.8%) verified death reports, 4vHPV one person about another or read on the internet. was given alone. In the remaining 16 reports, two to four total vac- Only two reports had a physician diagnosis of POI: (1) A 23- cines were given, including hepatitis A, meningococcal conjugate, year-old female with history of amenorrhea of unspecified start Tdap, varicella, and live attenuated inactivated influenza. Causes J.E. Arana et al. / Vaccine 36 (2018) 1781–1788 1785

4vHPV reports to VAERS per million vaccine doses distributed in the United States1

Year

Fig. 1. Crude reporting rates following human papillomavirus quadrivalent vaccine (4vHPV) by year, Vaccine Adverse Event Reporting System (VAERS), 2006–2015.

Table 4 Preferred Terms ‘‘infertility female” with EB05 = 2.88, and Characteristics of the 29 reports of verified deaths following human papillomavirus ‘‘Tourette’s disorder” with EB05 = 2.08. quadrivalent (4vHPV), Vaccine Adverse Event Reporting System (VAERS), 2009–2015. There were eight reports of infertility in females. One patient Cause of death based on Medical Dictionary N (%) had a diagnosis of POI (discussed above). Another received a for Regulatory Activities (MedDRA) System Organ Class provisional diagnosis of infertility with the evaluation by a repro- classification ductive endocrinology specialist in progress. In the remaining six Cardiac disorders 9 (31.0) cases, there was insufficient information to confirm a diagnosis Arrhythmia 3 Myocarditis 2 of infertility. Congenital subaortic membrane 1 There were 17 reports of Tourette’s disorder. Two patients Eosinophilic myocarditis 1 developed movement disorders following 4vHPV with symp- Hypertrophic cardiomyopathy 1 toms similar to Tourette’s, but did not have a definitive clinical Chronic epicarditis 1 diagnosis of Tourette’s disorder from a specialist (i.e., a neurol- Nervous system disorders 5 (17.2) Sudden unexpected death in epilepsya 4 ogist or psychiatrist). In three additional reports, patients had a Amyotrophic lateral sclerosis 1 Tourette’s diagnosis or displayed symptoms of Tourette’s prior Infections and infestations 5 (17.2) to vaccination. The remaining 12 reports were submitted by Group A Streptococcal sepsis 2 one physician who read on internet websites about possible Acute peritonitis 1 Pneumonia 1 Tourette disorder occurring after vaccines, but he had no first- HIV cerebral vasculitis 1 hand information on any patient. None of these 12 reports Psychiatric disorders 4 (13.8) could be verified. Suicideb 4 Blood and lymphatic malignancy or disorder 2 (6.9) Leukemia 1 5. Discussion Immune thrombocytopenic purpura 1 Other MedDRA System Organ Class 4 (13.8) Our analysis of 7 years (2009 through 2015) of post-licensure Hepatobiliary disorders 1 4vHPV safety data in VAERS did not detect any unusual or unex- Acute hepatic necrosis Injury, poisoning and procedural complications 1 pected patterns of AE reporting that would suggest new safety Diphenhydramine intoxication concerns in either males or females. The initial VAERS 4vHPV Vascular disorders 1 safety review included the period June 2006 through December Pulmonary embolism 2008, during which approximately 23 million doses had been dis- Neoplasm- malignant 1 tributed [7]. In contrast, during the current study period, more Cerebellar tumor than 60 million doses were distributed, and 4vHPV vaccine was a All four cases had pre-existing diagnoses of seizure disorder prior to 4vHPV also recommended for males [17,18]. Overall, the percentages of vaccination. Other co-morbid conditions included autism, pervasive developmental non-serious (94.2%) and serious reports (5.8%) were similar to disorder, and hydrocephalus. b Four cases: mechanical asphyxia (2), drug overdose (1), gunshot wound (1). those reported in the previous publication (93.8% and 6.2% respectively), and reporting rates for all reports, serious reports, syncope, venous thromboembolism, anaphylaxis, autoimmune of death among the 29 verified death reports are shown in Table 4. disorders, GBS, and death were similar to or lower than previous There was no clustering in the causes of death, pre-existing estimates [7]. Injection site reactions, dizziness and syncope/ co-morbidities, age, or interval from vaccination to death. presyncope were commonly reported AEs following 4vHPV among both females and males. The safety profile of 4vHPV 4.2. Empirical Bayesian data mining developed through our review is consistent with data from pre- licensure clinical trials and other post-licensure surveillance and We identified two 4vHPV-AE pairs that exceeded the empirical epidemiologic studies [4,5,7–12]. Bayesian data mining threshold (EB05  2) and met the require- After several years of 4vHPV use in the United States and other ments for further assessment. These included the MedDRA countries, concern about possible associations with autoimmune 1786 J.E. Arana et al. / Vaccine 36 (2018) 1781–1788 disorders and unusual and rare AEs, including POTS, CRPS and POI VAERS is subject to the limitations of passive surveillance, began to emerge, despite a lack of epidemiologic evidence or such as under-reporting, reporting biases, inconsistency in defined biologically plausible mechanisms [25,32]. The incidence quality and completeness of reports, and lack of an unbiased of many autoimmune disorders peaks in adolescence to early and unvaccinated comparison group [19]. For example, in our adulthood [33,34], a time when individuals are receiving the review sex was unknown or not reported in 22.6% of reports. 4vHPV series. Therefore the temporally-associated onset of Because of these limitations, VAERS data generally do not enable autoimmune disease following vaccination may occur due to us to determine a causal relationship of vaccines and AEs. Fur- chance alone, and assessing a possible causal association is chal- thermore, estimates of crude reporting rates using doses of lenging. However, epidemiologic studies of selected autoimmune 4vHPV vaccine distributed as a denominator should be inter- disorders have not detected any associations [8–12], and our find- preted with caution, since the actual number of doses adminis- ings indicate that individual autoimmune conditions and autoim- tered and to whom those doses were administered are not mune disorders collectively are rarely reported following 4vHPV known. The rate of underreporting of AEs is also unknown. (less than three reports per million doses distributed) (Table 3, Underreporting of common, mild events (e.g., injection site reac- Appendix). tions) is especially important. Events with a long lag time (e.g., The etiologies of POTS and CRPS are unclear, and concern POTS) between vaccination and appearance of symptoms may over these conditions following 4vHPV is largely based on case also be underreported, as the temporal association between vac- reports and case series [25]. POTS is a complex syndrome cination and the event may not be apparent. Despite these lim- with orthostatic intolerance as the main symptom [35]. CRPS itations, VAERS is a valuable monitoring system to detect is a rare chronic pain condition most often affecting a previ- potential vaccine safety problems that might require further ously injured limb [36]. A review conducted by the European investigations using controlled studies. Medicines Agency concluded that available evidence does not support a causal association between human papillomavirus 6. Conclusion vaccines and POTS or CRPS [25]. In our review, we did not observe unusual clustering around onset interval for POTS, We did not identify any new or unexpected safety concerns in but co-morbid chronic medical conditions were frequently our review of 4vHPV reports to VAERS from 2009 through 2015. reported. For CRPS, we also did not observe any unusual or With the exception of female-specific obstetric and gynecological unexpected patterns of reporting, but noted a lack of consis- conditions, the most common types of reported adverse events tency in the diagnostic criteria used by healthcare providers. were similar between females and males. We evaluated several For both of these conditions, data on population incidence diagnoses of interest that have emerged in the public health and rates are sparse. Overall, POTS and CRPS were rarely reported medical communities and in the media since the initial 4vHPV to VAERS, 1.1 and 0.28 reports per million 4vHPV doses dis- VAERS review of the first 2.5 years of use, which included POTS, tributed, respectively. CRPS and POI; we did not detect any safety concerns for these con- POI is the loss of ovarian function, including cessation of ditions or for other reproductive problems in females. The 4vHPV menses, prior to age 40 years; POI has previously been referred data in our review will serve as a reference for safety surveillance to as primary ovarian failure or premature menopause [37]. Causes of the 9-valent human papillomavirus vaccine, which replaced of POI are varied and in many instances remain unknown [38]. 4vHPV in the United States. Case reports and case series of POI following 4vHPV have been published [39]; however, epidemiologic evidence of an association between 4vHPV and POI and a biologically plausible mechanism Statement are lacking [32]. In our review, we identified 17 reports of POI after 4vHPV vaccine, 0.28 reports per million 4vHPV doses distributed, This paper contains original unpublished work and is not being indicating the condition is rarely reported following 4vHPV. Most submitted for publication elsewhere. Some of the 4vHPV safety of these case reports (15 of 17) had insufficient information to con- surveillance data from the study period have been publicly pre- firm a diagnosis of POI. sented at meetings of the Advisory Committee on Immunization These and other pre-specified conditions that were evalu- Practices (ACIP) and described in CDC’s Morbidity and Mortality ated in our clinical reviews, including deaths, did not reveal Weekly Report (MMWR). any concerning patterns that would suggest a causal associa- tion with 4vHPV. Deaths from cardiac disorders (n = 9 reports), Disclaimer suicide (n = 4), neoplasm (n = 1), HIV (n = 1), pneumonia (n = 1) and poisoning-injury (n = 1) (Table 4) are all among the ten The findings and conclusions in this report are those of the leading causes of death in the United States in individuals authors and do not necessarily represent the official position of 10 to 34 years of age [40]. Previous studies have not detected the Centers for Disease Control and Prevention (CDC) or the US an increased risk of death following 4vHPV or other vaccines Food and Drug Administration (FDA). Mention of a product or com- [41,42]. pany name does not constitute endorsement by the CDC or FDA. None of the pre-specified conditions met the threshold for an Information based on GardasilÒ doses distributed is presented with empirical Bayesian data mining signal. Syncope, and the associ- the permission of Merck Inc. ated condition of dizziness, following 4vHPV were relatively fre- quently reported, and has been observed and assessed in Funding source/sponsors previous post-licensure surveillance [7,13,16] and is not a new or unexpected finding. Two 4vHPV-AE pairs, ‘‘infertility female” No external sources of funding or sponsorship supported this work. and ‘‘Tourette’s disorder,” exceeded the empirical Bayesian data mining threshold and were further assessed. Review of reports for these conditions did not reveal any information that would Conflict of interest statement suggest a causal association with 4vHPV; many of the reports were hearsay reports or lacked sufficient information to confirm None of the authors have any financial or personal relationships a diagnosis. to disclose or any conflict of interest. J.E. Arana et al. / Vaccine 36 (2018) 1781–1788 1787

Appendix

Medical Dictionary for Regulatory Activities (MedDRA) Preferred Terms used to identify pre-specified conditions.

Pre-specified conditions MedDRA Preferred Terms Syncope Syncope, syncope vasovagal, loss of consciousness Anaphylaxis Anaphylactic shock, anaphylactic reaction, anaphylactoid reaction, anaphylactoid shock Autoimmune disorders Antinuclear antibody positive, autoantibody positive, autoimmune disorder, autoimmune haemolytic anaemia, autoimmune thyroiditis, autoimmune thrombocytopenia, Behcet’s syndrome, colitis ulcerative, dermatomyositis, mixed connective tissue disease, myasthenia gravis, polymyalgia rheumatica, Reiter’s syndrome, rheumatoid arthritis, scleroderma, sicca syndrome, Sjogren’s syndrome, systemic lupus erythematosus, polymyalgia rheumatica Venous thromboembolism Thrombosis, deep vein thrombosis, mesenteric vein thrombosis, cerebral venous thrombosis, cavernous sinus thrombosis, intracranial venous sinus thrombosis, pulmonary embolism, embolism venous, axillary vein thrombosis, venous thrombosis Guillain-Barré syndrome Guillain-Barré syndrome, Miller Fisher syndrome, demyelinating polyneuropathy Postural orthostatic tachycardia syndrome Postural orthostatic tachycardia syndrome, dizziness postural, postural reflex impairment Complex regional pain syndrome Complex regional pain syndrome, mononeuropathy multiplex Primary ovarian insufficiency Premature menopause, ovarian disorder, amenorrhea Deatha Died

a Also included reports where the checkbox for ‘‘Patient died” was checked in Box 8 of Vaccine Adverse Event Reporting System (VAERS) form.

References [14] Miranda S, Chaignot C, Collin C, Dray-Spira R, Weill A, Zureik M. Human papillomavirus vaccination and risk of autoimmune diseases: A large cohort study of over 2 million young girls in France. Vaccine 2017 Aug 24;35 [1] Markowitz LE, Dunne EF, Saraiya M, Lawson HW, Chesson H, Unger ER. Centers (36):4761–8. for disease control and prevention (CDC); advisory committee on [15] Gee J, Sukumaran L, Weintraub E, et al. Risk of Guillain-Barré Syndrome immunization practices (ACIP). Quadrivalent human papillomavirus vaccine: following quadrivalent human papillomavirus vaccine in the vaccine safety recommendations of the advisory committee on immunization practices datalink. Vaccine 2017 Oct;35(43):5756–8. (ACIP). MMWR Recomm Rep 2007;56(RR-2):1–24. [16] IOM (Institute of Medicine). 2012. Adverse effects of vaccines: Evidence and [2] Centers for Disease Control and Prevention. CDC). FDA licensure of causality. Washington, DC: The National Academies Press. males and guidance from the advisory committee on immunization practices [Accessed August 17,2016]. (ACIP). MMWR Morb Mortal Wkly Rep 2010;59(20):630–2. [17] Reagan-Steiner S, Yankey D, Jeyarajah J, et al. National, regional, state, and [3] Centers for Disease Control and Prevention (CDC). Recommendations on the selected local area vaccination coverage among adolescents aged 13–17 years- use of quadrivalent human papillomavirus vaccine in males–advisory United States, 2014. MMWR Morb Mortal Wkly Rep. 2015 Jul 31;64 committee on immunization practices (ACIP), 2011. MMWR Morb Mortal (29):784–92. Wkly Rep 2011;60(50):1705–8. Ò [18] Williams WW, Lu PJ, O’Halloran A, et al. Surveillance of vaccination coverage [4] GARDASIL package insert. . 2016;65(1):1–36. [Accessed August 17, 2016]. [19] Shimabukuro TT, Nguyen M, Martin D, DeStefano F. Safety monitoring in the [5] Markowitz LE, Dunne EF, Saraiya M, et al. Centers for disease control and Vaccine Adverse Event Reporting System (VAERS). Vaccine 2015;33 prevention (CDC). Human papillomavirus vaccination: recommendations of (36):4398–405. the advisory committee on immunization practices (ACIP). MMWR Recomm [20] Medical Dictionary for Regulatory Activities (MedDRA). . [Accessed August 17, 2016]. (49):1182. [21] Code of Federal Regulations Title 21, Volume 7 (21CFR600.80). . Prevention (CDC). Use of 9-valent human papillomavirus (HPV) vaccine: [Accessed August 17, 2016]. updated HPV vaccination recommendations of the advisory committee on [22] Sejvar JJ, Kohl KS, Gidudu J, et al. Guillain-Barré syndrome and Fisher immunization practices. MMWR Morb Mortal Wkly Rep. 2015;64(11):300–4. syndrome: case definitions and guidelines for collection, analysis, and [7] Slade BA, Leidel L, Vellozzi C, et al. Postlicensure safety surveillance for presentation of immunization safety data. Vaccine 2011;29(3):599–612. quadrivalent human papillomavirus recombinant vaccine. JAMA 2009;302 [23] Rüggeberg JU, Gold MS, Bayas JM, et al. Anaphylaxis: case definition and (7):750–7. guidelines for data collection, analysis, and presentation of immunization [8] Gee J, Naleway A, Shui I, et al. Monitoring the safety of quadrivalent human safety data. Vaccine 2007;25(31):5675–84. papillomavirus vaccine: findings from the vaccine safety datalink. Vaccine [24] Arana J, Mba-Jonas A, Jankosky C, et al. Reports of postural orthostatic 2011;29(46):8279–84. tachycardia syndrome after human papillomavirus vaccination in the vaccine [9] Arnheim-Dahlström L, Pasternak B, Svanström H, Sparén P, Hviid A. adverse event reporting system. J Adolescent Health 2017;61(5):577–82. Autoimmune, neurological, and venous thromboembolic adverse events after [25] European Medicines Agency. Pharmacovigilance Risk Assessment Committee immunisation of adolescent girls with quadrivalent human papillomavirus (PRAC) Assessment report: Human papillomavirus (HPV) vaccines. Review vaccine in Denmark and Sweden: cohort study. BMJ 2013;347:f5906. under Article 20 of Regulation (EC) No 726/2004. November 11, 2015 (EMA/ [10] Naleway AL, Crane B, Smith N, et al. Absence of venous thromboembolism risk 762033/2015). . [Accessed August [11] Chao C, Klein NP, Velicer CM, et al. Surveillance of autoimmune conditions 17, 2016]. following routine use of quadrivalent human papillomavirus vaccine. J Intern [26] Moro PL, Zheteyeva Y, Lewis P, et al. Safety of quadrivalent human Med 2012;271(2):193–203. papillomavirus vaccine (Gardasil) in pregnancy: adverse events among non- [12] Scheller NM, Svanström H, Pasternak B, et al. Quadrivalent HPV vaccination manufacturer reports in the vaccine adverse event reporting system, 2006– and risk of multiple sclerosis and other demyelinating diseases of the central 2013. Vaccine 2015;33(4):519–22. nervous system. JAMA 2015;313(1):54–61. [27] Goss MA, Lievano F, Buchanan KM, Seminack MM, Cunningham ML, Dana A. [13] Vichnin M, Bonanni P, Klein NP, et al. An overview of quadrivalent human Final report on exposure during pregnancy from a pregnancy registry for papillomavirus vaccine safety: 2006 to 2015. Pediatr Infect Dis J. 2015;34 quadrivalent human papillomavirus vaccine. Vaccine 2015;33(29):3422–8. (9):983–91. 1788 J.E. Arana et al. / Vaccine 36 (2018) 1781–1788

[28] DuMouchel W. Bayesian data mining in large frequency tables, with an nih.gov/disorders/postural_tachycardia_syndrome/postural_tachycardia_ application to the FDA spontaneous reporting system. Am Stat syndrome.htm>. [Accessed August 17, 2016]. 1999;53:177–90. [36] National Institute of Neurological Disorders and Stroke (NINDS). Complex [29] Szarfman A, Machado SG, O’Neill RT. Use of screening algorithms and Regional Pain Syndrome Fact Sheet. . drugs and events in the US FDA’s spontaneous reports database. Drug Saf [Accessed August 17, 2016]. 2002;25(6):381–92. [37] Committee opinion no. 605: primary ovarian insufficiency in adolescents and [30] Banks D, Woo EJ, Burwen DR, Perucci P, Braun MM, Ball RR. Comparing data young women. Obstet Gynecol 2014;124(1):193–7. mining methods on the VAERS database. Pharmacoepidemiol Drug Saf 2005 [38] Nelson LM. Clinical practice. Primary ovarian insufficiency. N Engl J Med Sep;14(9):601–9. 2009;360:606–14. [31] Raj SR. Postural tachycardia syndrome (POTS). Circulation 2013;127 [39] Gruber N, Shoenfeld Y. A link between human papilloma virus vaccination and (23):2336–42. primary ovarian insufficiency: current analysis. Curr Opin Obstet Gynecol [32] Hawkes D, Buttery JP. Human papillomavirus vaccination and primary ovarian 2015;27(4):265–70. insufficiency: an association based on ideology rather than evidence. Curr [40] Centers for Disease Control and Prevention (CDC). 10 Leading Causes of Death Opin Obstet Gynecol 2016;28(1):70–2. by Age Group, United States – 2014. . [Accessed adolescence. Arch Dis Child 2007;92(2):170–5. August 17, 2016]. [34] Parachuri V, Inglese C. Neurological problems in the adolescent population. [41] McCarthy NL, Gee J, Sukumaran L, et al. Vaccination and 30-day mortality risk Adolesc Med State Art Rev 2013;24(1):1–28. x. in children, adolescents, and young adults. Pediatrics 2016;137(3):e20152970. [35] National Institute of Neurological Disorders and Stroke (NINDS). NINDS [42] McCarthy NL, Weintraub E, Vellozzi C, et al. Mortality rates and cause-of-death Postural Tachycardia Syndrome Information Page:

Published case series have suggested a potential association between human papillomavirus (HPV) vaccination and primary ovarian insufficiency (POI). We describe POI incidence and estimate POI risk after HPV; tetanus toxoid, reduced diphtheria toxoid, and acellular pertussis, adsorbed (Tdap); inactivated influenza (II); and meningococcal METHODS: conjugate (MenACWY) vaccination. We searched Kaiser Permanente Northwest electronic health records for outpatient diagnoses suggestive of POI in female patients aged 11 to 34 years between 2006 and 2014. We reviewed and adjudicated the medical record to confirm diagnoses and estimate symptom onset dates. We excluded cases with known causes and calculated the incidence of idiopathic POI. We estimated risk by calculating hazard ratios and 95% confidence intervals RESULTS: (CIs). From a cohort of 199078 female patients, we identified 120 with diagnoses suggestive of POI. After adjudication and exclusion of 26 POI cases with known causes, we confirmed 46 idiopathic POI cases. POI incidence was low in 11- to 14-year-olds (0.87 per 1000000 person-months) and increased with age. One confirmed case patient received – – the HPV vaccine 23 months before the first clinical evaluation for delayed menarche. The – – adjusted hazard ratio was 0.30 (95% CI: 0.07 1.36) after HPV, 0.88 (95% CI: 0.37 2.10) after Tdap, 1.42 (95% CI: 0.59 3.41) after II, and 0.94 (95% CI: 0.27 3.23) after MenACWY CONCLUSIONS: vaccination. We did not find a statistically significant elevated risk of POI after HPV, Tdap, II, or MenACWY vaccination in this population-based retrospective cohort study. These findings should lessen concern about POI risk after adolescent vaccination.

WHAT’S KNOWN ON THIS SUBJECT: Authors of published case series have suggested an association aCenter for Health Research, Kaiser Permanente Northwest, Portland, Oregon; bInstitute for Health Research, Kaiser Permanente Colorado, Denver, Colorado; cDepartment of Pediatrics, School of Medicine, University of between human papillomavirus vaccination and Colorado, Aurora, Colorado; and dImmunization Safety Office, Centers for Disease Control and Prevention, primary ovarian insufficiency, but no population- Atlanta, Georgia based epidemiological studies have been reported.

Dr Naleway conceptualized and designed the study, designed the data collection instruments, WHAT THIS STUDY ADDS: No significant elevated risk collected data, supervised data analysis, drafted the initial manuscript, and reviewed and revised of primary ovarian insufficiency after adolescent the manuscript; Dr Henninger and Ms Irving conceptualized and designed the study, designed the vaccination was observed in this population-based data collection instruments, collected data, and critically reviewed the manuscript; Dr Mittendorf retrospective cohort study of nearly 200 000 young drafted the initial manuscript and reviewed and revised the manuscript; Dr Daley and Ms Gee women. These findings should lessen concern conceptualized and designed the study and critically reviewed the manuscript; Dr Smith and Mr about potential impact on fertility from adolescent Crane collected data, conducted the analyses, and critically reviewed the manuscript; and all vaccination. authors approved the final manuscript as submitted and agree to be accountable for all aspects of the work. DOI: https://​doi.​org/​10.​1542/​peds.​2018-​0943 Accepted for publication Jun 5, 2018 Address correspondence to Allison L. Naleway, PhD, Kaiser Permanente Center for Health To cite: Naleway AL, Mittendorf KF, Irving SA, et al. Pri­ Research, 3800 N. Interstate Ave, Portland, OR 97227. E-mail: [email protected] mary Ovarian Insufficiency and Adolescent Vaccination. Pediatrics. 2018;142(3):e20180943

Downloaded from www.aappublications.org/news by guest on March 12, 2019 PEDIATRICS Volume 142, number 3, September 2018:e20180943 ARTICLE Human papillomavirus (HPV) is the with underlying autoimmune December 31, 2014. This study most common sexually transmitted1 (eg, rheumatoid arthritis or systemic period was selected to maximize infection in the United States and lupus erythematosus), metabolic the potential number of POI cases is associated with several cancers (eg, galactosemia), or11 infectious in the period after HPV vaccine as well as anogenital warts. Since disease (eg, mumps). licensure and recommendation. 2006, HPV vaccines have been We also wanted to focus on female Reports of POI after HPV vaccination licensed and recommended for use patients who were age-eligible have garnered national media in adolescent girls by the Advisory for HPV vaccination and to allow attention and have circulated widely Committee on Immunization – for the follow-up of women who 2 on social media and other Internet Practices. Rates of HPV vaccination 12 15 were vaccinated at 26 years of age. sites,​ ‍ ‍ but to our knowledge, have lagged behind coverage rates Cohort participants were managed no population-based studies of POI ’ for other recommended adolescent until health plan disenrollment, the after HPV vaccination have been vaccinations, such as tetanus participant s 35th birthday, or the conducted to date. The published toxoid, reduced diphtheria toxoid, end of the study period, whichever case series must be interpreted with and acellular pertussis, adsorbed occurred first. caution because the authors of the Case Ascertainment (Tdap) and meningococcal conjugate series included small samples of (MenACWY). On the basis of national young women presenting for care coverage estimates from 2016, 65% at selected clinical sites, relied on We identified presumptive cases of 13- to 17-year old girls received self-reported vaccine exposures, of POI among young women in at least 1 HPV vaccination and only ∼ and lacked controls. We conducted a the cohort by searching electronic 49.5% were up to date with the retrospective cohort study to health record (EHR) databases series, compared with 88% of International Classification (1) identify and describe for outpatient encounters coded adolescents who had received the of Diseases, Ninth Revision 3 characteristics of idiopathic POI with Tdap vaccine. Although authors of diagnosed in female patients 11 (ICD-9) large population-based studies have to 34 years of age, (2) describe codes for premature menopause demonstrated the safety of HPV the prevalence and age-specific (256.31), other ovarian failure vaccination, parental safety concerns, incidence of POI, and (3) estimate (256.39), other ovarian dysfunction including potential impacts on future – the risk of idiopathic POI in female (256.8), and unspecified ovarian fertility, are often cited as 1 reason 4 8 patients after HPV vaccination or dysfunction (256.9). We then for lower HPV coverage. ‍‍ other recommended adolescent manually reviewed the medical vaccinations (Tdap, MenACWY, and record of the presumptive cases to Concern about infertility after HPV inactivated influenza [II]). collect data on diagnostic and other vaccination developed after case METHODS laboratory testing, symptom onset, series were published describing and other POI risk factors. The study the onset of primary ovarian abstraction form was developed insufficiency (POI), also known in partnership with subject matter as premature ovarian failure or The Vaccine Safety Datalink (VSD) experts from the Centers for Disease premature menopause, within is a collaborative project involving Control and Prevention and 2 KPNW 12 months after vaccination in 8 integrated health care delivery systems and the Centers for Disease obstetrics and gynecology providers. 6 young9,10​ women from 13 to 21 years of age. ‍ POI is characterized by Control and Prevention. The current Each presumptive case was abstracted either the dysfunction or depletion VSD study was conducted at a single by at least 1 study investigator (MLH, site, Kaiser Permanente Northwest of ovarian follicles, menopausal ∼ SAI, and ALN). Presumptive cases that symptoms (eg, amenorrhea or hot (KPNW), serving a population of were clearly miscoded or ruled out flashes), or reduced fertility. In girls 570000 members in Oregon and diagnoses, or cases for whom adequate <20 years of age, POI is uncommon Washington. The study protocol, medical records were not available, data collection tools, and study with an estimated11 prevalence of were excluded from additional review. 1 case per 10000. Chromosomal procedures were approved by the Remaining presumptive cases were abnormalities, including Turner StudyKPNW Period Institutional and Population Review Board. then abstracted again independently syndrome and Fragile X syndrome, by a second investigator. Any as well as the gonadotoxic treatment discrepant information obtained of cancer (chemotherapy or We identified all female patients by the 2 independent abstractions radiation) are known etiologies for 11 to 34 years of age with at least was discussed and resolved by the POI; however, most POI is idiopathic 30 days of health plan enrollment investigator team. As part of the but may be associated at KPNW from August 1, 2006, to medical record review, we either Downloaded from www.aappublications.org/news by guest on March 12, 2019 2 NALEWAY et al recorded the date of POI symptom was classified if the case clearly did HPV, we identified all vaccine onset when noted or estimated the Descriptivenot meet the Analysis ACOG definition. exposures in the cohort occurring onset date on the basis of other 1 year before (August 1, 2005) and information documented in the during the study period. Vaccinations record (eg, age at onset or date of last We calculated the proportion of were identified by using both the menses) or the earliest encounter the presumptive cases identified by KPNW EHR databases and the Oregon for delayed menarche, amenorrhea, each ICD-9 code that met the case and Washington state immunization oligomenorrhea, or infertility definitions of probable and possible information systems to increase the evaluation. POI as defined above after medical capture of vaccinations received record review and adjudication. outside of the KPNW delivery system. After medical record review, We also described demographic To calculate person-time in the Cox we excluded cases of POI with a characteristics, diagnostic testing models, we managed subjects until known cause, including those with patterns and results, symptom the symptom onset dates for cases, surgical menopause (ie, removal onset, and time to diagnosis among and the end of the study period, the of the ovaries), those with a confirmed (probable and possible) 35th birthday, or last health plan genetic condition, such as Fragile POIidiopathic Incidence cases. disenrollment during the study period X Syndrome, Turner syndrome, for noncases. We defined vaccine or another X-linked chromosomal exposure as the first dose received of a disorder, and those who had received specific vaccine from 1 year before the either chemotherapy or radiotherapy We calculated the prevalence of start of the observation period through for cancer. The remaining idiopathic idiopathic diagnosed POI by dividing the end of follow-up. In the absence presumptive POI cases were the number of confirmed (probable of any data to suggest a biologically reviewed and adjudicated by an and possible) cases identified during plausible exposure window, we obstetrics and gynecology nurse the study period by the number considered any person-time after practitioner to confirm case status. of eligible cohort members; POI incidence was calculated by using a specific vaccine exposure to be exposed person-time. We therefore did The American College of eligible person-months as the not specifically model multiple doses of Obstetricians and Gynecologists denominator. Person-months were calculated based on the beginning a vaccine or a combination of multiple (ACOG) guidelines for the diagnosis ’ 16 vaccines. We estimated both crude of POI include the presence of and end of the study period and each individual s health plan enrollment and age-adjusted HRs for each vaccine menstrual irregularity for at least – dates as described above. We and described the timing of vaccine 3 months and elevated follicle- – – – – calculated age-stratified (11 14, exposures related to POI symptom stimulating hormone (FSH) in – onset. On the basis of post hoc power the postmenopausal range and 15 18, 19 22, 23 26, 27 30, and calculations, we estimated the smallest low estradiol levels on 2 separate 31 34 years) rates on the basis of the α significant HRs we could detect with occasions. Other clinical tests date of POI diagnosis and calculated 80% power, and = .05 were 3.23 for used to establish or refine the 95% confidence intervals (CIs) HPV, 2.94 for Tdap, 2.94 for II, and 3.45 diagnosis include karyotyping, for these rates on the basis of the for MenACWY exposures. adrenal antibody titers, pelvic PoissonRisk Estimation distribution. ultrasonography, and antimullerian RESULTS hormone levels. Many presumptive cases could not be classified with We used time-dependent Cox During the study period (August the ACOG definition because the proportional hazards modeling to diagnostic tests were not consistently estimate the hazard ratios (HRs) 1, 2006, to December 31, 2014), ordered by the treating providers. and 95% CIs associated with HPV, 199078 female patients 11 to 34 years of age had at least 30 days of The clinical adjudicator was Tdap, MenACWY, or II exposures health plan enrollment in KPNW; instructed to classify presumptive independently. Because we observed 119078 received Tdap, 84783 cases as probable POI if there long intervals between symptom was strong evidence to support a onset and POI diagnosis, we limited received II, 58871 received HPV, diagnosis of POI and/or most or the cases included in the models and 46231 received MenACWY all of the ACOG definition was met; to those with symptom onset after vaccination. From this cohort, we possible POI was classified if there August 1, 2006 (ie, the date of HPV identified 120 patients with an was some evidence to support a vaccine availability at KPNW). To outpatient diagnosis of premature diagnosis of POI, but the ACOG capture data on additional adolescent menopause, ovarian failure, or definition was not met; and not POI vaccines that were licensed before ovarian dysfunction (Fig 1). We Downloaded from www.aappublications.org/news by guest on March 12, 2019 PEDIATRICS Volume 142, number 3, September 2018 3 patients were white (61%) and non-Hispanic (67%) individuals. Six patients (13%) presented with primary amenorrhea, and 8 patients (17%) had a comorbid autoimmune disease diagnosis. Symptom onset could not always be estimated if there was insufficient documentation in the medical record or if the patient presented with primary amenorrhea. Excluding these cases, the median ∼ time from symptom onset to – diagnosis was 3 years (range: 75 days 16 years, data not shown).

Most patients had at least 1 FSH (94%) or estradiol (78%) test in their medical record, but multiple tests were less common, especially for estradiol (Table 2). Only 9 (20%) confirmed cases met the strict ACOG diagnostic criteria for POI. Testing for antiadrenal and thyroperoxidase antibodies was uncommon, as were abnormal antimullerian hormone levels.

The prevalence of idiopathic POI in the study period was 2.31 per 10000 female patients. The incidence of

diagnosed POI increased with age, from a low of 0.87 per 1000000 person-

months in 11- to 14-year-olds to a peak FIGURE 1 of 12.85 per 1000000 person-months Ascertainment and adjudication of POI cases, KPNW, August 2006 to December 2014. in 30- to 34-year-olds (Table 3).

Of the confirmed cases, 18 (39%) had symptom onset before August 1, excluded 41 presumptive cases Two ICD-9 codes, 256.8 (other 2006, and were excluded from Cox ovarian dysfunction) and 256.9 after the initial chart review; 35 – models. Of the remaining 28 cases, patients had a specific diagnosis (unspecified ovarian dysfunction), only 1 was vaccinated against HPV other than POI (miscoded POI yielded no medical record confirmed before symptom onset; this 16-year- ∼ diagnoses), 4 patients were ruled out POI cases. Of the 28 presumptive old girl received her third dose of POI diagnoses, and 2 patients had cases diagnosed with ICD-9 code HPV vaccine 23 months before inadequate medical records available 256.31 (premature menopause), the estimated symptom onset. This for review. After the second round 50% were classified as possible patient presented with primary of chart review, we excluded 26 POI or probable idiopathic POI after amenorrhea, so the symptom onset full review and adjudication. cases with a known cause, including date was estimated as the earliest Similarly, we classified 43% of the the following: 6 patients with surgical documented encounter for clinical 76 presumptive cases diagnosed with menopause, 12 patients with a evaluation of delayed menarche; ICD-9 256.39 (other ovarian failure) genetic disorder, and 14 patients POI onset likely occurred before as possible or probable idiopathic POI. this estimated date. The age-adjusted treated for cancer (note categories – are not mutually exclusive). The More than one-half of the confirmed HR of POI after HPV vaccination was ≥ clinical adjudicator classified 7 0.30 (95% CI: 0.07 1.36) (Table 4). POI cases were patients who were – presumptive cases as not POI, leaving diagnosed at 27 years of age, and Six confirmed patients with POI 46 confirmed, idiopathic POI cases only 1 patient was diagnosed at were vaccinated with Tdap (5 40 (33 probable and 13 possible). <15 years of age (Table 1). Most months before symptom onset), Downloaded from www.aappublications.org/news by guest on March 12, 2019 4 NALEWAY et al TABLE 1 Characteristics of Confirmed Idiopathic POI Cases, KPNW, August 2006 to December 2014 – Characteristic N (%) – CI: 0.37 2.10) with Tdap, 1.42 (95% – Age at diagnosis, y CI: 0.59 3.41) with II, and 0.94 11–14 1 (2) (95% CI: 0.27 3.23) with MenACWY 15–18 5 (11) 19–22 4 (9) vaccination (Table 4). 23–26 7 (15) DISCUSSION 27–30 13 (28) 31–34 16 (35) Age at estimated symptom onset, y 11–14 6 (13) To our knowledge, this is the first 15–18 9 (20) population-based study in which POI is 19 22 5 (11) – evaluated as a possible vaccine adverse 23–26 11 (24) 27–30 10 (28) event. We did not find an elevated 31–34 5 (11) risk of POI after HPV vaccination, nor Race did we find any elevated risk after White 28 (61) Tdap, II, or MenACWY vaccination. Multiracial or people of color 5 (11) We observed only 1 patient with Unknown 13 (28) POI who possibly had symptom Ethnicity Latina and/or Hispanic 7 (15) onset after HPV vaccination from a Non-Latina or non-Hispanic 31 (67) population of 58871 young women Unknown 8 (17) who received the HPV vaccine during Autoimmune comorbid diagnosis 8 (17) Primary amenorrhea 6 (13) the study period. If POI is triggered Family history of POI 4 (9) by HPV or other adolescent vaccine Family history of infertility 4 (9) exposure, we would have expected to see elevated incidence in the younger TABLE 2 Diagnostic Testing for POI, KPNW, August 2006 to December 2014 women who were most likely to be Test N (%) vaccinated, but instead we observed ACOG tests higher incidence in older women (>26 FSH tested 43 (94) years of age), which is consistent with FSH tested on at least 2 separate occasions 33 (72) 1 other population-based11 study of FSH was abnormala on at least 2 separate occasions 33 (72) POI prevalence. With our study, we Estradiol tested 36 (78) overcame some of the limitations of Estradiol tested on at least 2 separate occasions 16 (35) a the previously published case reports Estradiol was abnormal on at least 2 separate occasions 9 (20) Met ACOG POI case definitionb 9 (20) describing POI after HPV vaccination, Other diagnostic tests including small sample sizes and lack Abnormal antimullerian hormone 4 (9) of population-based controls, and this Antiadrenal antibodies 0 (0) should provide additional reassurance Thyroperoxidase antibodies 4 (9) about the safety of HPV vaccination a Abnormal is defined as being in the postmenopausal range by using reference values documented in the medical record. Reference values varied by patient age, reference laboratory, and the timing of collection relative to menstrual cycle and with respect to POI risk. However, hormonal therapy use. it should be noted that this study b ACOG POI case definition requires the presence of menstrual irregularity for at least 3 mo, elevated FSH in the was underpowered to detect small postmenopausal range, and low estradiol levels on 2 separate occasions. increases in POI risk associated with vaccination. TABLE 3 Age-Specific Incidence of Diagnosed POI, KPNW, August 2006 to December 2014 Studying POI as a vaccine adverse Age at Initial Diagnosis, y Cases Person-Mo Incidence per 1 000 000 Person-Mo (95% CI) event is challenging for many reasons. 11–14 1 1 151 805 0.87 (0.12–6.16) First, the time from symptom onset to 15 18 5 1226602 4.08 (1.70 9.79) – – diagnosis with POI may be variable or 19–22 4 1 109 535 3.61 (1.35–9.61) 17 23–26 7 1 059 109 6.61 (3.15–13.86) long. We observed that the median 27–30 13 1 151 201 11.29 (6.56–19.45) time from symptom onset to POI 31–34 16 1 245 185 12.85 (7.87–20.97) diagnosis was 3 years in this cohort, and the authors of another study reported that 25% of patients required – 17 – >5 years from onset to diagnosis. We 11 patients were vaccinated with with MenACWY (2 27 months were able to capture cases that were II (2 91 months before symptom before symptom onset). The age- diagnosed in our health plan, but we onset), and 3 patients were vaccinated adjusted HR of POI was 0.88 (95% may have misclassified some true cases Downloaded from www.aappublications.org/news by guest on March 12, 2019 PEDIATRICS Volume 142, number 3, September 2018 5 TABLE 4 POI Incidence in Vaccinated and Unvaccinated Young Women and Associated HRs With 95% CIs Cases Vaccinated Before Unexposed Cases Unadjusted HR Age-Adjusted HR Symptom Onset (95% CI) (95% CI) observe any temporal clustering of vaccine exposures among patients. HPV 1 27 0.27 (0.06–1.16) 0.30 (0.07–1.36) CONCLUSIONS Tdap 6 22 0.78 (0.34–1.84) 0.88 (0.37–2.10) MenACWY 3 25 0.70 (0.24–2.03) 0.94 (0.27–3.23) II 11 17 1.35 (0.57–3.22) 1.42 (0.59–3.41) Concern about a potential association between HPV vaccination and POI generated by earlier published case series and media attention may of POI who were symptomatic but did they were attempting to conceive, negatively impact decision-making not have a long enough follow-up time and subsequently presented for about HPV vaccine acceptance. We to reach a diagnosis. However, because diagnostic evaluation of amenorrhea found no evidence of increased 81% of our cohort was managed and infertility. We were not able to risk of POI after HPV vaccination, for >24 months (mean follow-up adjust for contraceptive use because or other routine adolescent time = 5.14 years), we consider that these data are not routinely included vaccination exposures, in this this potential for misclassification in the VSD data files. Additionally, the population-based retrospective was minimal. capture of contraceptives for young cohort study with nearly 200000 women may not be complete in health young women. Despite the challenges Secondly, diagnoses of POI are plan databases if young women are and limitations discussed above, difficult to accurately identify. We receiving and filling prescriptions we believe this study should lessen found that using the ACOG diagnostic at outside pharmacies and clinics. concern surrounding potential definition was difficult to implement We believe that the potential impact on fertility from HPV or other in observational studies derived misclassification of case status in adolescent vaccination. from EHR data because patients women using hormonal contraceptives ACKNOWLEDGMENT often did not undergo all of the would be nondifferential because diagnostic testing16 required to meet authors of previous studies have not the definition. In our health care found that hormonal conceptive use We thank Kate Beadle, nurse system, we observed that patients is more prevalent among vaccinated practitioner, for her assistance with were likely to receive 1 FSH and 1 young women compared23,24​ with the adjudication of medical records. estradiol test but not the multiple unvaccinated women. ‍ ABBREVIATIONS tests required for positive diagnosis ’ If vaccination is associated with POI based on the ACOG definition; review onset, the exposure would most likely and adjudication by a women s health induce POI through an autoimmune ACOG: American College of clinician was therefore needed to 25,26​ pathway. ‍ There are no mechanistic Obstetricians and establish case status. Additionally, studies available to suggest what a Gynecologists we excluded some POI cases with a biologically plausible exposure window CI: confidence interval known cause, such as certain genetic might be, and so we examined any EHR: electronic health record disorders like Turner syndrome, and exposure before symptom onset. Using FSH: follicle-stimulating focused on idiopathic POI that might this long exposure window has the hormone be triggered by vaccination. Other potential to mask a true acute risk of HPV: human International papillomavirus Classification case definitions for epidemiological POI after vaccination, but limiting our HR: hazard of Diseases, ratio Ninth Revision studies have varied, and many study to shorter windows might miss ICD-9:  researchers did not exclude POI with an association that requires a longer a known cause, making comparison of – latent period. II: inactivated influenza rates and estimation18 20 of risk difficult KPNW: Kaiser Permanente All vaccinated patients were exposed across studies. ‍‍ Northwest at least 2 months before symptom MenACWY: meningococcal Another challenge of this study, and onset, and many were vaccinated years conjugate all observational studies of POI, is before onset. The 1 patient exposed to POI: primary ovarian adequately capturing hormonal HPV who was observed was vaccinated insufficiency contraceptive use, which may mask the nearly 2 years before the estimated Tdap: tetanus toxoid, reduced symptoms and onset of POI and can symptom onset date, which is not 21,22​ diphtheria toxoid, and result in misclassification. ‍ Many of consistent with the authors of case acellular pertussis, the patients we observed in this cohort reports who observed onset within 9,10​ adsorbed were women in their late 20s who 12 months of vaccination. ‍ Although VSD: Vaccine Safety Datalink stopped using contraceptives because we did not formally test, we did not Downloaded from www.aappublications.org/news by guest on March 12, 2019 6 NALEWAY et al PEDIATRICS (ISSN Numbers: Print, 0031-4005; Online, 1098-4275). Copyright © 2018 by the American Academy of Pediatrics FINANCIAL DISCLOSURE: Dr Naleway has received research funding from Pfizer, MedImmune AstraZeneca, and Merck for unrelated studies. Ms Irving has received research support from MedImmune AstraZeneca for an unrelated study. Dr Henninger has received research funding from Pfizer for an unrelated study; the other authors have indicated they have no financial relationships relevant to this article to disclose. FUNDING: Supported by the US Centers for Disease Control and Prevention (task order 200-2012-53584-0006). The findings and conclusions in this report are those of the authors and do not necessarily represent the official position of the Centers for Disease Control and Prevention. POTENTIAL CONFLICT OF INTEREST: Dr Naleway has received research funding from Pfizer, MedImmune AstraZeneca, and Merck for unrelated studies. Ms Irving has received research support from MedImmune AstraZeneca for an unrelated study. Dr Henninger has received research funding from Pfizer for an unrelated study; the other authors have indicated they have no potential conflicts of interest to disclose.

REFERENCES 1. Satter white CL, Torrone E, Meites E, qualitative study post-vaccination. BMC young-​women-​claim-​hpv-​vaccine-​left-​ et al. Sexually transmitted infections Public Health. 2012;12:629 them-infer​ tile/​. Accessed June 22, 2018 among US women and men: prevalence 8. Glenn BA, Tsui J, Coronado GD, et 15. Jonathan; SaneVax, Inc. Infertility and incidence estimates, 2008. Sex al. Understanding HPV vaccination concern with gardasil HPV vaccine: Transm Dis. 2013;40(3):187–193 among Latino adolescent girls in three does the HPV vaccine LITERALLY mean 2. Petrosky E, Bocchini JA Jr, Hariri S, U.S. regions. J Immigr Minor Health. “one less”? 2011. Available at: http://​ et al; Centers for Disease Control and 2015;17(1):96–103 sanevax.​org/​infertility-​concern-​with-​ Prevention. Use of 9-valent human 9. Little DT, Ward HR. Premature ovarian gardasil-​hpv-​vaccine/​. Accessed June papillomavirus (HPV) vaccine: updated failure 3 years after menarche in 22, 2018 HPV vaccination recommendations a 16-year-old girl following human 16. The American College of Obstetricians of the advisory committee on papillomavirus vaccination. BMJ Case and Gynecologists. Committee opinion immunization practices. MMWR Morb Rep. 2012;2012:bcr2012006​879 no. 605: primary ovarian insufficiency Mortal Wkly Rep. 2015;64(11):300–304 10. Colafrancesco S, Perricone C, in adolescents and young women. 3. Walker TY, Elam-Evans LD, Singleton Tomljenovic L, Shoenfeld Y. Human Obstet Gynecol. 2014;124(1): JA, et al. National, regional, state, papilloma virus vaccine and primary 193–197 and selected local area vaccination ovarian failure: another facet of the 17. Alzubaidi NH, Chapin HL, Vanderhoof coverage among adolescents aged autoimmune/inflammatory syndrome VH, Calis KA, Nelson LM. Meeting the 13-17 years - United States, 2016. induced by adjuvants. Am J Reprod needs of young women with secondary MMWR Morb Mortal Wkly Rep. Immunol. 2013;70(4):309–316 amenorrhea and spontaneous 2017;66(33):874–882 11. Cox L, Liu JH. Primary ovarian premature ovarian failure. Obstet 4. Bingham A, Drake JK, LaMontagne insufficiency: an update. Int J Womens Gynecol. 2002;99(5, pt 1):720–725 DS. Sociocultural issues in the Health. 2014;6:235–243 18. Coulam CB, Adamson SC, Annegers JF. introduction of human papillomavirus 12. Sears A. They ve been robbed of their Incidence of premature ovarian failure. vaccine in low-resource settings. “ ’ womanhood: two sisters face one life- Obstet Gynecol. 1986;67(4):604–606 Arch Pediatr Adolesc Med. ” changing diagnosis. 2014. Available at: 19. Sadrzadeh S, Painter RC, van Kasteren 2009;163(5):455 461 – http://​fox6now.​com/​2014/​11/​13/​theyve-​ YM, Braat DD, Lambalk CB. Premature 5. Verhoeven V, Baay MF, Baay PE, been-​robbed-​of-​their-​womanhood-​ ovarian insufficiency and perinatal Lardon F, Van Royen P, Vermorken two-​sisters-​face-​one-​life-​changing-​ parameters: a retrospective case- JB. Everything you always wanted to diagnosis/​. Accessed June 22, 2018 control study. Maturitas. 2017; know about HPV (but could not ask 13. Wetzstein C. HPV vaccine cited in 96:72–76 your doctor). Patient Educ Couns. infertility case: Wis. sisters say drug 20. Haller -Kikkatalo K, Uibo R, Kurg A, 2010;81(1):101 105 – led to ovarian failure by age 16. Salumets A. The prevalence and 6. Young A. HPV vaccine acceptance The Washington Times. November phenotypic characteristics of among women in the Asian 11, 2013. Available at: https://​www.​ spontaneous premature ovarian Pacific: a systematic review of the washingtontimes.​com/​news/​2013/​nov/​ failure: a general population literature. Asian Pac J Cancer Prev. 11/​hpv-​vaccine-​cited-​in-​infertility-​case/​. registry-based study. Hum Reprod. 2010;11(3):641–649 Accessed June 22, 2018 2015;30(5):1229–1238 7. Cover JK, Nghi NQ, LaMontagne DS, 14. Favoloro M. Young women claim HPV 21. Welt CK. Clinical manifestations and Huyen DT, Hien NT, Nga le T. Acceptance vaccine left them infertile. The Legal evaluation of spontaneous primary patterns and decision-making for Examiner. December 5, 2013. Available ovarian insufficiency (premature human papillomavirus vaccination at: http://​norfolk.​legalexaminer.​ ovarian failure). 2017. Available at: among parents in Vietnam: an in-depth com/​fda-​prescription-​drugs/​ https://​www.​uptodate.​com/​contents/​

Downloaded from www.aappublications.org/news by guest on March 12, 2019 PEDIATRICS Volume 142, number 3, September 2018 7 clinical-​manifestations-​and-​diagnosis-​ 23. Jena AB, Goldman DP, Seabury SA. 25. Pellegrino P, Carnovale C, Pozzi M, of-​spontaneous-​primary-​ovarian-​ Incidence of sexually transmitted et al. On the relationship between insufficiency-​premature-​ovarian-​ infections after human papillomavirus human papilloma virus vaccine and failure. Accessed April 27, 2017 vaccination among adolescent females. autoimmune diseases. Autoimmun Rev. 22. Little D. Quadrivalent human JAMA Intern Med. 2015;175(4):617–623 2014;13(7):736–741 papillomavirus vaccine and the young 24. Bednarczyk RA. Human papillomavirus 26. Gruber N, Shoenfeld Y. A link between ovary: review of safety research vaccine and sexual activity: how human papilloma virus vaccination following two case series of premature do we best address parent and and primary ovarian insufficiency: ovarian insufficiency.J Immunol Infect physician concerns? JAMA Intern Med. current analysis. Curr Opin Obstet Dis. 2017;4(1):101 2015;175(4):624–625 Gynecol. 2015;27(4):265–270

Downloaded from www.aappublications.org/news by guest on March 12, 2019 8 NALEWAY et al Primary Ovarian Insufficiency and Adolescent Vaccination Allison L. Naleway, Kathleen F. Mittendorf, Stephanie A. Irving, Michelle L. Henninger, Bradley Crane, Ning Smith, Matthew F. Daley and Julianne Gee Pediatrics 2018;142; DOI: 10.1542/peds.2018-0943 originally published online August 21, 2018;

Updated Information & including high resolution figures, can be found at: Services http://pediatrics.aappublications.org/content/142/3/e20180943 References This article cites 21 articles, 1 of which you can access for free at: http://pediatrics.aappublications.org/content/142/3/e20180943#BIBL Subspecialty Collections This article, along with others on similar topics, appears in the following collection(s): Adolescent Health/Medicine http://www.aappublications.org/cgi/collection/adolescent_health:med icine_sub Infectious Disease http://www.aappublications.org/cgi/collection/infectious_diseases_su b Vaccine/Immunization http://www.aappublications.org/cgi/collection/vaccine:immunization _sub Permissions & Licensing Information about reproducing this article in parts (figures, tables) or in its entirety can be found online at: http://www.aappublications.org/site/misc/Permissions.xhtml Reprints Information about ordering reprints can be found online: http://www.aappublications.org/site/misc/reprints.xhtml

Downloaded from www.aappublications.org/news by guest on March 12, 2019 Primary Ovarian Insufficiency and Adolescent Vaccination Allison L. Naleway, Kathleen F. Mittendorf, Stephanie A. Irving, Michelle L. Henninger, Bradley Crane, Ning Smith, Matthew F. Daley and Julianne Gee Pediatrics 2018;142; DOI: 10.1542/peds.2018-0943 originally published online August 21, 2018;

The online version of this article, along with updated information and services, is located on the World Wide Web at: http://pediatrics.aappublications.org/content/142/3/e20180943

Pediatrics is the official journal of the American Academy of Pediatrics. A monthly publication, it has been published continuously since 1948. Pediatrics is owned, published, and trademarked by the American Academy of Pediatrics, 141 Northwest Point Boulevard, Elk Grove Village, Illinois, 60007. Copyright © 2018 by the American Academy of Pediatrics. All rights reserved. Print ISSN: 1073-0397.

Downloaded from www.aappublications.org/news by guest on March 12, 2019 Sexual Activity–Related Outcomes After Human Papillomavirus Vaccination of 11- to 12-Year-Olds

WHAT’S KNOWN ON THIS SUBJECT: Concerns persist about sexual AUTHORS: Robert A. Bednarczyk, PhD,a,b Robert Davis, MD, disinhibition after human papillomavirus (HPV) vaccination of MPH,a Kevin Ault, MD,c Walter Orenstein, MD,c,d and Saad B. preteenage girls. Self-reported surveys have indicated few antici- Omer, MBBS, PhD, MPHa,b,c,d pated behavior changes after HPV vaccination. Little is known about aCenter for Health Research-Southeast, Kaiser Permanente, sexual activity–related clinical outcomes after HPV vaccination. Atlanta, Georgia; and bRollins School of Public Health, cSchool of Medicine, and dEmory Vaccine Center, Emory University, Atlanta, Georgia WHAT THIS STUDY ADDS: Utilizing managed care organization KEY WORDS electronic data, we evaluated the incidence of adverse outcomes human papillomavirus, vaccine, sexual activity, disinhibition of sexual activity among vaccinated preteenage girls and found ABBREVIATIONS little difference between those who received HPV vaccine and CI—confidence interval those who did not. HPV—human papillomavirus ICD-9—International Classification of Diseases, Ninth Revision IRD—incidence rate difference IRR—incidence rate ratio MCO—managed care organization abstract MCV4—quadrivalent meningococcal conjugate vaccine 2 OBJECTIVE: r —bivariate correlation coefficient Previous surveys on hypothesized sexual activity changes STI—sexually transmitted infection after human papillomavirus (HPV) vaccination may be subject to self- Tdap—tetanus toxoid, reduced diphtheria toxoid, and acellular response biases. To date, no studies measured clinical markers of sexual pertussis, adsorbed activity after HPV vaccination. This study evaluated sexual activity–related VD-NOS—venereal disease, not otherwise specified clinical outcomes after adolescent vaccination. All authors are responsible for the reported research. Dr Bednarczyk conceptualized and designed the study, conducted METHODS: We conducted a retrospective cohort study utilizing longi- the analysis of and interpreted the data, drafted the initial tudinal electronic data from a large managed care organization. Girls article and approved the final article as submitted; Dr Davis enrolled in the managed care organization, aged 11 through 12 years participated in the conceptualization and design of the study, between July 2006 and December 2007, were classified by adolescent participated in the analysis and interpretation of the data, reviewed and revised the article, and approved the final article vaccine (HPV; tetanus toxoid, reduced diphtheria toxoid, and acellular as submitted; Dr Ault participated in the conceptualization and pertussis, adsorbed; quadrivalent meningococcal conjugate) receipt. design of the study, participated in the interpretation of the Outcomes (pregnancy/sexually transmitted infection testing or diagno- data, reviewed and revised the article, and approved the final sis; contraceptive counseling) were assessed through December 31, article as submitted; Dr Orenstein participated in the design of the study, participated in the interpretation of the data, 2010, providing up to 3 years of follow-up. Incidence rate ratios reviewed and revised the article, and approved the final article comparing vaccination categories were estimated with multivariate as submitted; and Dr Omer participated in the conceptualization Poisson regression, adjusting for health care–seeking behavior and and design of the study, participated in the analysis and demographic characteristics. interpretation of the data, reviewed and revised the article, and approved the final article as submitted. RESULTS: – The cohort included 1398 girls (493 HPV vaccine exposed; This study has not been published in any other journal. Portions 905 HPV vaccine–unexposed). Risk of the composite outcome (any of this research were presented at the 15th Annual National pregnancy/sexually transmitted infection testing or diagnosis or Foundation for Infectious Diseases Annual Conference on contraceptive counseling) was not significantly elevated in HPV vaccine– Vaccine Research, Baltimore, MD. exposed girls relative to HPV vaccine–unexposed girls (adjusted incidence www.pediatrics.org/cgi/doi/10.1542/peds.2012-1516 rate ratio: 1.29, 95% confidence interval [CI]: 0.92 to1.80; incidence doi:10.1542/peds.2012-1516 rate difference: 1.6/100 person-years; 95% CI: 20.03 to 3.24). Incidence Accepted for publication Jul 6, 2012 rate difference for Chlamydia infection (0.06/100 person-years [95% Address correspondence to Robert A. Bednarczyk, PhD, Kaiser CI: 20.30 to 0.18]) and pregnancy diagnoses (0.07/100 person-years Permanente Center for Health Research-Southeast, 11 Piedmont [95% CI: 20.20 to 0.35]), indicating little clinically meaningful absolute Center, 3495 Piedmont Rd NE, Suite 110, Atlanta GA 30305. E-mail: [email protected]; [email protected] risk differences. CONCLUSIONS: HPV vaccination in the recommended ages was not asso- (Continued on last page) ciated with increased sexual activity–related outcome rates. Pediatrics 2012;130:798–805

798 BEDNARCZYK et al Downloaded from www.aappublications.org/news by guest on February 5, 2019 ARTICLE

In 2006, the Advisory Committee on Im- A frequently discussed concern, both in by the MCO. The cohort comprised girls munization Practices recommended peer-reviewed literature and mass me- who had the opportunity to receive the that all US girls aged 11 to 12 receive the dia, about vaccinating preteenage girls HPV vaccine within the recommended human papillomavirus (HPV) vaccine, against HPV is that vaccination against age range (11–12 years) between July 1, with catch-up vaccination recommended an STI could lead to increased pro- 2006 and December 31, 2007, with follow- through age 26, and administration miscuity through risk compensation up for outcomes through December 31, permitted as young as 9 years.1 The or behavioral disinhibition.13–18 Most 2010. These time frames were selected recommendation for preteenage girls teenage girls surveyed on knowledge, to allow up to 3 years of follow-up while to be vaccinated against a sexually attitudes, and practices related to HPV increasing homogeneity of the cohort transmitted infection (STI) is based on vaccination reported they would not by including girls receiving vaccines the need to develop immunity before modify their sexual behaviors after HPV only during the prespecified window. HPV exposure. vaccination.15,19–25 Most of these stud- Restricting eligibility to girls in the Early onset of sexual activity and mul- ies were limited by the use of self- recommended ages for vaccination was 15,19–25 tiple sexual partners are risk factors reported, cross-sectional surveys, done to minimize issues of confounding for HPV infection.1 Rates of adolescent however, and it is unknown if these by indication related to sexually active sexual activity among 15- to 17-year- survey results would directly translate girls potentially being more likely to to clinical outcomes. olds have declined in recent years,2 seek HPV vaccination, by focusing on from 39% in 1995% to 27% between To date, there has been no evaluation girls who were less likely to have al- 2006 and 2010. Nearly half of sexually of changes in sexual activity–related ready initiated sexual activity. active girls reported .2 sexual part- outcomes after HPV vaccination that Girls born between July 2, 1993 and De- ners.3 Additionally, ∼3% of high school avoidstheriskofresponsebiasthatmay cember 31, 1996 and enrolled in the MCO girls report initiating sexual activity occur in sexual activity surveys.26–28 In as of July 1, 2006 were identified. Girls before age 13.4 This early initiation of this study, we directly examined sexual were excluded from analysis if they: (1) sexual activity is accompanied by a activity–related outcomes (ie, STI disenrolled from the MCO before De- high prevalence of adolescent genital or pregnancy testing or diagnosis, or cember 31, 2007; (2) received adolescent HPV infection, with 33% of 14- to 19- counseling on contraceptives) among vaccines of interest after December 31, year-olds infected with at least 1 HPV girls enrolled in Kaiser Permanente 2007 or before recommendations for use strain,5 and 12% infected with 1 of the Georgia, a large managed care organi- in the United States (HPV vaccine: July 4 quadrivalent vaccine strains.6 zation (MCO) in the metropolitan Atlanta 2006; MCV4: January 2005; Tdap: May area. Specifically, we evaluated girls in Nationally, HPV vaccine 3-dose series 2005); (3) were either ,11 years or $13 the recommended age range for HPV initiation among 13- to 17-year-old girls years old when vaccinated; (4) had a vaccination (11–12 years) during its increased from 25% in 2007% to 49% in history of any outcome of interest on or first 18 months of availability, with up 2010. The HPV vaccine coverage is lower before December 31, 2007; and (5) did to 3 years of follow-up to identify out- than the uptake of other recently rec- not receive any adolescent vaccines comes. We sought to test the hypoth- ommended adolescent vaccines among during the study period. esis of a clinically meaningful increase – 13- to 17-year-old girls and boys, such (alternative hypothesis incidence rate Girls were considered HPV vaccine as the combination tetanus toxoid, re- ratio [IRR] of 1.5) in rates of testing or exposed if they received at least 1 dose duced diphtheria toxoid, and acellular diagnosis for pregnancy or STIs or phy- of HPV vaccine, regardless of receipt of pertussis, adsorbed (Tdap) vaccine sician counseling on contraceptives after any other adolescent vaccine, and HPV – (69% in 2010) and quadrivalent me- receipt of HPV vaccine in this age range. vaccine unexposed if they received ningococcal conjugate vaccine (MCV4, any doses of Tdap and/or MCV4 in the 63% in 2010).7 Recommendations for absence of HPV vaccination. For the fi- METHODS Tdap and MCV4 vaccination were ap- nal cohort, follow-up for outcomes be- proved ,18 months before the HPV Data on health plan enrollment, vac- gan on a common date, January 1, vaccination recommendation.1,8,9 Barriers cination history, and sexual activity– 2008. Person-time at risk began ac- to HPV vaccination exist at the struc- related outcomes of interest (ie, STI, cruing on this date and ended at the tural (cost, multidose series, phy- pregnancy testing or diagnosis, or coun- first of either (1) the date of incident sicians not encouraging vaccination) seling on hormonal contraception) were outcome, (2) MCO disenrollment date, and individual (concerns about vaccine obtained from clinical/administrative or (3) December 31, 2010. The age at safety and fear of needles) levels.7,10–12 and laboratory databases maintained vaccination of interest was the age at

PEDIATRICS Volume 130, Number 5, November 2012 799 Downloaded from www.aappublications.org/news by guest on February 5, 2019 receipt of the first dose of HPV vaccine outcomes for C. trachomatis infection, coverage in an anticipated adolescent for HPV vaccine–exposed girls and the pregnancy, or VD-NOS, to capture ac- population of 1400, a control group in- earliest of the date(s) of first Tdap or tual STI or pregnancy findings. Inci- cidence of 5/100 person-years over MCV4 vaccine(s) for HPV vaccine– dence of these composite outcomes 3 years of follow-up and potential bi- unexposed girls. was defined as the first occurrence of variate correlation coefficient (r2)be- Data on outcomes occurring through any of the component findings, and the tween modeled covariates to be 0.00, the December 31, 2010 were obtained using age at incident outcome was the age at study had 83% power to detect an IRR of 2 International Classification of Dis- the first component finding. Secondary 1.5. With a potential r of 0.05 between eases, Ninth Revision (ICD-9) and Cur- analysis examined each component modeled covariates to be 0.05, the study rent Procedural Terminology codes separately. Recurrent findings of the had 81% power to detect an IRR of 1.5. In fi 2 for pregnancy or Chlamydia tracho- same outcome were not considered, the nal analysis, the r between expo- matis testing; diagnoses of pregnancy, although a girl could be positive for sure and modeled covariates was 0.06, C. trachomatis, trichomoniasis, cervi- multiple different outcomes. which corresponds to a power of 80%. citis or unspecified STI; or physician Incidence counts and total person-time All analyses were conducted by using counseling about contraceptives. Out- at risk were calculated for each out- SAS (version 9.2; SAS Institute Inc, Cary, comes of trichomoniasis, cervicitis, or come. IRRs comparing HPV vaccine– NC), at a significance level of a = .05. unspecified STI were grouped into the exposed to unexposed girls were This study was approved by the MCO’s single outcome of venereal disease, not computed with multivariate Poisson institutional review board. otherwise specified (VD-NOS). Because regression, by using the log of person- hormonal contraceptive medications years at risk as the offset variable, with RESULTS can be prescribed for medical con- a robust error variance to account A total of 6795 girls met the initial birth ditions other than birth control (eg, for overdispersion in the Poisson date criteria and were enrolled in the dysmenorrhea or acne), we excluded model. Models for more common out- MCO on July 1, 2006. After applying the contraceptive counseling with previous comes (Testing/Diagnosis/Counseling, exclusion criteria, 5393 girls were ex- or concomitant dysmenorrhea or acne C. trachomatis testing, pregnancy test- cluded from the final analysis (1874 diagnoses from the category of contra- ing, and contraception counseling) were disenrolled from the MCO before ceptive outcomes; other outcomes that adjusted for health care–seeking be- December31,2007;1817receivedvaccine these girls may have had at different havior in the previous year, age at vac- [s] of interest on or after age 13; 678 medical encounters were retained for cination, race, and census tract–level received vaccine[s] of interest after analysis. For girls with ICD-9 or Current socioeconomic status (proportions of December 31, 2007; 66 received vaccine Procedural Terminology testing codes residents living at or above the poverty [s] of interest before their 11th birthday in their records without a correspond- line and with at least a high school di- or vaccine recommendation; 23 had an ing diagnosis code (eg, V74.5, Screen- ploma or equivalent). Socioeconomic incident outcome on or before December ing examination for venereal disease), status data were obtained from the 31, 2007; and 939 did not receive any we consulted laboratory test records 2009 American Community Survey 5- adolescentvaccines). Hence, the analysis to confirm diagnoses of interest. To year estimates.29 Models for less com- cohort included 1398 girls. – assess baseline health care seeking mon outcomes were adjusted for health In this cohort, nearly all girls received – behaviors, we counted the number of care seeking behavior and age at vac- either Tdap (93%) or MCV4 (91%), all-cause medical encounters in the year cination (pregnancy, C. trachomatis)or whereas 35% initiated the HPV vac- – before the vaccination of interest. health care seeking behavior only (VD- cine series. Exposure classification for We used 2 main outcome definitions. The NOS). Unadjusted incidence rate differ- analysis resulted in 493 HPV vaccine– first (“Testing/Diagnosis/Counseling”) ences (IRDs) were computed. Age at in- exposed girls and 905 HPV vaccine– incorporated medical outcomes relating cident outcome was compared by vaccine unexposed girls (Fig 1). Nearly all (474/493) ’ to sexual activity and includes any oc- exposure by using Students t test. HPV vaccine–exposed girls received at currence of testing for C. trachomatis or Sample size and power calculations least 1 of the comparison vaccines pregnancy; diagnoses of C. trachomatis were performed to determine if this (Fig 1). The age at vaccination of interest infection, pregnancy, or VD-NOS; and analysis would be sufficiently powered was higher for HPV vaccine–exposed physician counseling on contracep- for the Testing/Diagnosis/Counseling girls than unexposed girls (11.9 years tives. The second (“Diagnosis Only”) outcome. Using a Poisson regression versus 11.6 years, respectively, P , .001). includes any occurrence of diagnostic framework, for 35% HPV vaccine HPV vaccine–exposed girls had more

800 BEDNARCZYK et al Downloaded from www.aappublications.org/news by guest on February 5, 2019 ARTICLE all-cause medical encounters in the pre- were 107 girls tested for pregnancy The Diagnosis-Only incidence rate was vious year than HPV vaccine–unexposed and 55 tested for C. trachomatis, but low (0.26/100 person-years in the HPV girls (mean: 2.6 vs 2.1; P = .024). HPV only 4 pregnancies and 4 C. trachomatis vaccine–exposed group versus 0.25/ vaccine uptake was more common in infections (Table 2). 100 person-years in the HPV vaccine– white girls than in those who were Afri- Girls receiving HPV vaccine did not have unexposed group; adjusted IRR: 1.11; can American. Census tract–level socio- a significantly higher incidence rate of 95% CI: 0.26 to 4.64) (Table 2). The mean economic status measures were similar Testing/Diagnosis/Counseling (5.5/100 age at first Testing/Diagnosis/Counseling across vaccination groups (Table 1). person-years; adjusted IRR: 1.29; 95% outcome for HPV vaccine–exposed The Testing/Diagnosis/Counseling out- CI: 0.92 to 1.80) compared with the HPV girls (14.4 years) was similar to that of come was more common than the vaccine–unexposed group (3.9/100 the unexposed group (14.6 years, P =.33). Diagnosis-Only outcome (n = 137, 9.8%, person-years; IRD: 1.6/100 person- A similar pattern in age at first diagnostic and n = 8, 0.6%, respectively). There years; 95% CI: 20.03 to 3.24) (Table 2). outcome was observed (HPV vaccine– exposed: 14.8 years; HPV vaccine– unexposed: 14.6 years; P = .82) (Fig 2). Incidence rates, IRR, and IRD estimates for each of 6 individual secondary outcomes (C. trachomatis testing and diagnosis, pregnancy testing and di- agnosis, VD-NOS diagnosis, and contra- ceptive counseling) are presented in Table 2. No significantly increased IRRs were estimated for individual outcomes comparing HPV vaccine–exposed and FIGURE 1 unexposed girls. The unadjusted IRD Receipt of HPV, MCV4, or Tdap vaccines by adolescent girls in a large MCO, highlighting the frequent for counseling on contraceptive use was receipt of .1 type of vaccine. slightly higher in HPV vaccine–exposed TABLE 1 Uptake of Routinely Recommended Adolescent Vaccines and Baseline Health Care– girls (0.90/100 person-years, 95% CI: Seeking Behavior Among 11- to 12-Year-Old Girls in a Large MCO, Between July 1, 2006 0.15 to 1.65), although the adjusted and December 31, 2007 IRR was not significantly elevated (2.31, HPV Vaccine–Exposed HPV Vaccine–Unexposed P 95% CI: 0.99 to 5.38) (Table 2). (N = 493) (N = 905) Age at vaccination Age at vaccination of interest, y; mean (SD)a 11.9 (0.60) 11.6 (0.51) ,.001 DISCUSSION Received vaccination of interest when 11 y 272 (55.2) 697 (77.0) ,.001 old, N (%)a We present the first evaluation of sexual Health care–seeking behavior activity–related outcomes after ado- All-cause medical encounters in year before 2.6 (3.9) 2.1 (3.3) .024 lescent HPV vaccination in the recom- vaccination of interest, mean (SD)a Had 0 all-cause medical encounters in year 104 (21.1) 269 (29.7) ,.002 mended age range of 11 to 12 years to before vaccination of interest, N (%)a use clinical outcomes and show that Race receipt of HPV vaccine is not associated White, N (%) 189 (38.3) 235 (26.0) ,.001 – Black, N (%) 159 (32.3) 389 (43.0) with an increased rate of sexual activity Other, N (%) 35 (7.1) 52 (5.8) related outcomes. This study’s results Unknown 110 (22.3) 231 (25.3) are not limited by the use of self-reported Socioeconomic statusb,c surveys,26,27 and instead use adminis- Percent of population living at or above 87.5 (10.1) 87.2 (8.9) .579 poverty level, mean (SD) trative data from a large MCO. This Percent of population with at least a high 87.0 (8.8) 87.1 (8.2) .708 study provides a clinical validation school diploma or equivalent, mean (SD) and extends our understanding of nu- Length of enrollment Years enrolled from January 1, 2008; mean 2.3 (0.82) 2.2 (0.90) .028 merous surveys reporting that most (SD) young women did not plan to modify a For HPV vaccine–exposed, the age at first HPV vaccine dose; for HPV vaccine–unexposed, the earliest age at either first Tdap their sexual behaviors after HPV vacci- and/or MCV4 dose. nation.15,19–25 Although most previous b As estimated from census tract–level data obtained from 2009 American Community Survey 5-y estimates. c Socioeconomic status data missing for 2 HPV vaccine–exposed girls and 5 HPV vaccine–unexposed girls. surveys have been cross-sectional, a

PEDIATRICS Volume 130, Number 5, November 2012 801 Downloaded from www.aappublications.org/news by guest on February 5, 2019 recent longitudinal survey conducted – nding,

fi in the United Kingdom documented no difference in the proportion of women 0.03 to 3.24) 0.35 to 0.38) 0.32 to 1.71) 0.20 to 0.35) 0.30 to 0.18) 0.14 to 2.75) 0.23 to 0.21) 2 2 2 2 2 2 2 reporting initiation of sexual activity 30 0.06 ( 0.01 ( after HPV vaccination ; however, that 2 2 study was conducted in women aged 16 to 18, of whom 37% had already Unexposed 30

Exposed Versus HPV Vaccine become sexually active. In our analy- – sis, HPV vaccination at ages 11 through 12 did not increase the likelihood of

HPV Vaccine seeking medical attention for out- 1.29 (0.92 to 1.80) 1.61 ( 1.11 (0.26 to 4.64)1.28 (0.74 to 2.22)0.68 (0.06 to 7.71) 0.01 ( 0.69 ( 2.31 (0.99 to 5.38) 0.90 (0.15 to 1.65) 0.90 (0.09 to 9.07) 1.28 (0.88 to 1.85) 1.30 ( 1.89 (0.33 to 10.79) 0.07 ( comes related to sexual activity with up to 3 years of follow-up. This study was designed with sufficient power to de- tect a meaningful difference in the main outcome by HPV vaccine expo- 91.6 96.69 98.9 99.45 99.67 99.78 93.48 99.78 sure, so the probability of type II error was relatively low. = 905 N Similarities in the age at first outcome by vaccination category indicate that 3.91 1.50 0.50 0.25 0.15 0.10 3.02 0.10 there may not be any earlier onset of Unexposed, – sexual activity after HPV vaccination. It is likely that any disinhibition or risk compensation would occur closer to the HPV Vaccine timeofvaccination(ie,within18months) rather than much later. If HPV vaccina- 5/2020.4 3/2023.4 2/2023.3 2/2024.3 76/1944.1 30/1997.9 10/2015.0 59/1956.7 “ ”31 per Person-Years IR % Without Outcome aIRR (95% CI) IR Difference (95% CI) tion was a license for sex, we would N have expected to see more adverse outcomes shortly after vaccination, when the girls were more aware of their re- 1 type of diagnosis. The summary outcomes (Testing/Diagnosis/Counseling and Diagnosis Only) only address the occurrence of any component Related Outcomes After Receipt of Routinely Recommended Adolescent Vaccines Among 11- to 12-Year-Old Girls in a Large .

– cent vaccination status. These findings are predicated on the assumption that 87.63 99.39 94.93 99.8 96.75 99.8 90.26 99.59 adverse outcomes after initiation of sexual activity would be followed by = 493

N health care seeking; within this frame- work, we are unable to identify girls 1 type of testing performed or Exposed, 5.51 2.20 1.39 0.26 0.09 0.09 4.32 0.17

. who initiated sexual activity and did not – seek reproductive health care.

seeking behavior in the previous year, age at vaccination of interest, race, and socioeconomic status: Testing/Diagnosis/Counseling, Chlamydia testing, pregnancy testing, and counseling on 22 – Recently, Liddon et al reported that

HPV Vaccine more than a quarter of young women may overestimate the scope of protection 3/1155.0 1/1155.4 1/1155.8 2/1156.1 61/1106.2 25/1138.8 16/1147.1 48/1111.6 of the HPV vaccine, extending it to other

per Person-Years IR % Without Outcome STIs. Women in that study were older N than girls in this cohort, and there may be age-related differences in education about, or understanding of, the effect of HPV vaccination. Additionally, that Incidence Rates and Unadjusted andMCO, Adjusted Between IRRs January for Sexual 1, Activity 2008 and December 31, 2010 study documented no difference in self- Outcome reported sexual activity between girls Counseling contraceptives TABLE 2 The following outcomes were adjusted for health care so the total outcomes for individual components may not add up to the values observed for the summary outcomes. aIRR, adjusted incidence rate ratio; IR, incidence rate (per 100 person-years). contraceptives. The following outcomes were adjusted forhealth care health seeking care in seeking the in previous the year. Some previous girls year may and have age had at vaccination of interest: Diagnosis Only, Chlamydia diagnosis, and pregnancy diagnosis. The VD-NOS diagnosis outcome was adjusted only for Testing/Diagnosis/ Diagnosis Only Chlamydia testing Chlamydia diagnosis VD-NOS diagnosis Pregnancy testing Pregnancy diagnosis Counseling on who did and did not receive the HPV

802 BEDNARCZYK et al Downloaded from www.aappublications.org/news by guest on February 5, 2019 ARTICLE

retrospective analysis of administra- tive data. Although the MCO captures all medical encounters within their prac- tices, some girls may have received vaccines in outside clinics (eg, school- located vaccination clinics) or received reproductive health care outside of the MCO (eg, at independent reproductive health centers). Second, the use of this type of data precludes an assessment of motivations for care; because most outcomes were coded by using ICD-9 codes specifying screening, we could FIGURE 2 notdetermineifthesetestswerepartof Mean ages at first Testing/Diagnosis/Counseling and Diagnosis-Only outcomes among adolescent girls standard clinical practice or if they who received $1 adolescent vaccine in a large MCO. Error bars represent 95% CIs for the mean. at test were due to presenting complaints re- b P = .325; t test P = .818. lated to sexual activity. Third, our pop- ulation was restricted in terms of age at vaccine, and an increasedlikelihood of active. Additionally, most health care vaccine uptake, which may limit our condom use among HPV-vaccinated girls, decisions for girls age 11 through 12 are ability to generalize outside of this age possibly indicating a greater understand- made by parentsor guardians,33 and it is range. Further analysis with wider time ing of reproductive health and pre- not likely that perceptions of sexuality frames and age ranges is needed to vention.22 A recent study from Australia led at these ages to the decision to re- examine adolescent vaccination and supports this finding, with HPV-vaccinated ceive the HPV vaccine. If this type of reproductive health care practices women aged 18 to 30 equally likely to confounding by indication were present, among other segments of the adoles- it would result in an overestimation of be sexually active than nonvaccinated cent population. Fourth, as our analy- risk of sexual activity–related outcomes; women, with vaccinated women holding sis was powered to address the most the lack of significant associations in the stronger attitudes toward safer sexual sensitive outcome (Testing/Diagnosis/ presence of potential overestimation behaviors, although there was no differ- Counseling), the low incidence of preg- further supports our findings. ence in condom use by vaccination nancy or STI diagnoses in this cohort 32 fi group. Our nding of slightly, but not We identified differences in baseline resulted in imprecise IRR estimates for fi signi cantly, increased contraceptive health care–seeking behavior in the these outcomes. More preciseestimates – counseling among HPV vaccine exposed year before receipt of the vaccine of for these individual diagnoses will need fi girlssupportsthisprevious nding re- interest. HPV vaccine–exposed girls additional, future analyses. Finally, there garding contraception use, and may were more likely to have had any all- is the possibility of confounding on the actually have a positive impact on ado- cause medical encounters in this pe- part of the physicians (eg, physicians lescent preventive health services by riod than HPV vaccine–unexposed girls. who are more likely to administer HPV establishing a long-term relationship Although we adjusted for this differ- vaccine may be more likely to initiate between these girls and their physi- ence in health care–seeking behavior conversations about contraception or cian. The administrative data used for in the multivariate regression analysis, conduct more routine screening preg- this study did not provide an opportu- there may still be differences between nancy or STI-screening tests). We were nity to do a detailed examination of the these groups with respect to health unable to examine any potential clus- reasons for this counseling or of the care–seeking behaviors, particularly tering of outcomes by specificproviders extent of hormonal contraceptive use with the need for receipt of 3 doses of of practice offices within the MCO. among girls in this cohort. HPV vaccine. This increased exposure CONCLUSIONS We attempted to address issues of to health care providers presents more confounding by indication related to opportunities for medical counseling Receipt of HPV vaccine by 11- to 12-year- sexually active girls potentially being and evaluation through the adolescent old girls was not associated with clinical more likely to seek HPV vaccination by and young adult period.34 markers of increased sexual activity– restricting the analysis to younger girls This study has some limitations. First, related outcomes, such as sexually who are less likely to already be sexually the study was conducted by using a transmitted diseases or pregnancy.

PEDIATRICS Volume 130, Number 5, November 2012 803 Downloaded from www.aappublications.org/news by guest on February 5, 2019 REFERENCES

1. Markowitz LE, Dunne EF, Saraiya M, Lawson Advisory Committee on Immunization Prac- papillomavirus vaccine would encourage HW, Chesson H, Unger ER; Centers for Dis- tices (ACIP). MMWR Recomm Rep. 2006; sexual activity in their children. J Low Genit ease Control and Prevention (CDC); Advi- 55(RR-3):1–34 Tract Dis. 2010;14(3):179–184 sory Committee on Immunization Practices 10. Bednarczyk RA, Birkhead GS, Morse DL, 21. Kahn JA, Ding L, Huang B, Zimet GD, (ACIP). Quadrivalent Human Papillomavirus Doleyres H, McNutt LA. Human papilloma- Rosenthal SL, Frazier AL. Mothers’ intention Vaccine: Recommendations of the Advisory virus vaccine uptake and barriers: associ- for their daughters and themselves to re- Committee on Immunization Practices ation with perceived risk, actual risk and ceive the human papillomavirus vaccine: (ACIP). MMWR Recomm Rep. 2007;56(RR-2): race/ethnicity among female students at a national study of nurses. Pediatrics. 2009; 1–24 a New York State university, 2010. Vaccine. 123(6):1439–1445 2. Centers for Disease Control and Pre- 2011;29(17):3138–3143 22. Liddon NC, Leichliter JS, Markowitz LE. vention (CDC). Sexual experience and 11. Liddon NC, Hood JE, Leichliter JS. Intent Human papillomavirus vaccine and sex- contraceptive use among female teens— to receive HPV vaccine and reasons for ual behavior among adolescent and United States, 1995, 2002, and 2006-2010. not vaccinating among unvaccinated young women. Am J Prev Med. 2012;42 MMWR Morb Mortal Wkly Rep. 2012;61 adolescent and young women: findings (1):44–52 (17):297–301 from the 2006-2008 National Survey of 23. Litton AG, Desmond RA, Gilliland J, Huh WK, 3. Martinez G, Copen C, Abma J. Teenagers in Family Growth. Vaccine. 2012;30(16):2676– Franklin FA. Factors associated with in- the United States: Sexual Activity, Contra- 2682 tention to vaccinate a daughter against HPV: ceptive Use, and Childbearing, 2006-2010 12. Patel DA, Zochowski M, Peterman S, a statewide survey in Alabama. J Pediatr National Survey of Family Growth. Atlanta, Dempsey AF, Ernst S, Dalton VK. Human Adolesc Gynecol. 2011;24(3):166–171 GA: Centers for Disease Control and Pre- papillomavirus vaccine intent and uptake 24. Mullins TL, Zimet GD, Rosenthal SL, et al. vention; 2011 among female college students. J Am Coll Adolescent perceptions of risk and need 4. Eaton DK, Kann L, Kinchen S, et al; Centers Health. 2012;60(2):151–161 for safer sexual behaviors after first hu- for Disease Control and Prevention (CDC). 13. United Press International. Promiscuity man papillomavirus vaccination. Arch — Youth risk behavior surveillance United fears hinders HPV vaccine use. Available at: Pediatr Adolesc Med. 2012;166(1):82–88 States, 2009. MMWR Surveill Summ. 2010; www.upi.com/Health_News/2008/12/19/ 25. Read DS, Joseph MA, Polishchuk V, Suss AL. 59(5):1–142 Promiscuity-fears-hinders-HPV-vaccine-use/ Attitudes and perceptions of the HPV vaccine 5. Hariri S, Unger ER, Sternberg M, et al. UPI-54571229744264/. Accessed October in Caribbean and African-American adoles- Prevalence of genital human papillomavi- 28, 2011 cent girls and their parents. J Pediatr Ado- rus among females in the United States, 14. Rubin R. Injected into a controversy. USA lesc Gynecol. 2010;23(4):242–245 the National Health And Nutrition Exami- Today. Available at: www.usatoday.com/ 26. Graham CA, Catania JA, Brand R, Duong T, nation Survey, 2003-2006. J Infect Dis. 2011; news/health/2005-10-19-cervical-cancer- Canchola JA. Recalling sexual behavior: 204(4):566–573 injection_x.htm. Accessed October 28, a methodological analysis of memory re- 6. Dunne EF, Sternberg M, Markowitz LE, et al. 2011 call bias via interview using the diary as Human papillomavirus (HPV) 6, 11, 16, and 15. Schuler CL, Reiter PL, Smith JS, Brewer NT. the gold standard. J Sex Res. 2003;40(4): 18 prevalence among females in the United Human papillomavirus vaccine and behavioural 325–332 States—National Health And Nutrition Ex- disinhibition. Sex Transm Infect. 2011;87(4): amination Survey, 2003-2006: opportunity 27. Siegel DM, Aten MJ, Roghmann KJ. Self- 349–353 to measure HPV vaccine impact? J Infect reported honesty among middle and high Dis. 2011;204(4):562–565 16. Forster A, Wardle J, Stephenson J, Waller J. school students responding to a sexual Passport to promiscuity or lifesaver: press behavior questionnaire. J Adolesc Health. 7. Centers for Disease Control and Pre- coverage of HPV vaccination and risky 1998;23(1):20–28 vention (CDC). National and state vacci- sexual behavior. J Health Commun. 2010;15 nation coverage among adolescents aged 28. Clark LR, Brasseux C, Richmond D, Getson P, (2):205–217 ’ 13 through 17 years—United States, 2010. D Angelo LJ. Are adolescents accurate in MMWR Morb Mortal Wkly Rep. 2011;60(33): 17. Marlow LA, Forster AS, Wardle J, Waller J. self-report of frequencies of sexually ’ ’ 1117–1123 Mothers and adolescents beliefs about transmitted diseases and pregnancies? risk compensation following HPV vacci- J Adolesc Health. 1997;21(2):91–96 8. Bilukha OO, Rosenstein N; National Cen- – ter for Infectious Diseases, Centers for nation. J Adolesc Health. 2009;44(5):446 29. American FactFinder. Available at: http:// Disease Control and Prevention (CDC). 451 factfinder2.census.gov/faces/nav/jsf/pages/ Prevention and control of meningococcal 18. Waller J, Marlow LA, Wardle J. Mothers’ index.xhtml. Accessed March 29, 2012 disease. Recommendations of the Advi- attitudes towards preventing cervical 30. Forster AS, Marlow LA, Stephenson J, sory Committee on Immunization Practices cancer through human papillomavirus Wardle J, Waller J. Human papillomavirus (ACIP). MMWR Recomm Rep. 2005;54(RR-7): vaccination: a qualitative study. Cancer vaccination and sexual behaviour: cross- 1–21 Epidemiol Biomarkers Prev. 2006;15(7): sectional and longitudinal surveys con- – 9. Broder KR, Cortese MM, Iskander JK, 1257 1261 ducted in England. Vaccine. 2012;30(33): et al; Advisory Committee on Immuniza- 19. Fazekas KI, Brewer NT, Smith JS. HPV 4939–4944 tion Practices (ACIP). Preventing tetanus, vaccine acceptability in a rural Southern 31. Turning Point Ministries International. HPV diphtheria, and pertussis among ado- area. JWomensHealth(Larchmt). 2008; vaccine: A license for sex? Available at: lescents: use of tetanus toxoid, reduced 17(4):539–548 www.tpmi.org/apps/articles/web/articleid/ diphtheria toxoid and acellular pertus- 20. Ferris DG, Cromwell L, Waller JL, Horn L. 30884/columnid/3149/default.asp. Accessed sis vaccines recommendations of the Most parents do not think receiving human May 16, 2012

804 BEDNARCZYK et al Downloaded from www.aappublications.org/news by guest on February 5, 2019 ARTICLE

32. Mather T, McCaffery K, Juraskova I. Does 33. Hughes CC, Jones AL, Feemster KA, Fiks AG. 34. National Vaccine Advisory Committee. The HPV vaccination affect women’sattitudesto HPV vaccine decision making in pediatric promise and challenge of adolescent im- cervical cancer screening and safe sexual primary care: a semi-structured interview munization. Am J Prev Med. 2008;35(2): behaviour? Vaccine. 2012;30(21):3196–3201 study. BMC Pediatr. 2011;11:74 152–157

(Continued from first page) PEDIATRICS (ISSN Numbers: Print, 0031-4005; Online, 1098-4275). Copyright © 2012 by the American Academy of Pediatrics FINANCIAL DISCLOSURE: Dr Davis served as Chair of the Data and Safety Monitoring Board for the study “A Post Licensure Surveillance Program for the Safety of Gardasil in a Managed Care Setting,” funded by Merck Pharmaceuticals, Inc. The honoraria for this effort was paid to Kaiser Permanente Georgia, for work occurring from December 2007 through March 2009. Dr Ault has conducted human papillomavirus vaccine clinical trials, with funding paid to his institution by Merck, Roche, and the National Institute of Allergy and Infectious Diseases. Dr Omer was awarded the Maurice R. Hilleman Early-stage Career Investigator Award by the National Foundation for Infectious Diseases. The award was funded by an unrestricted educational grant to the National Foundation for Infectious Diseases from Merck; however, Dr Omer had no direct interaction with Merck. Drs Bednarczyk and Orenstein have indicated they have no financial relationships relevant to this article to disclose. FUNDING: No external funding.

PEDIATRICS Volume 130, Number 5, November 2012 805 Downloaded from www.aappublications.org/news by guest on February 5, 2019 Sexual Activity−Related Outcomes After Human Papillomavirus Vaccination of 11- to 12-Year-Olds Robert A. Bednarczyk, Robert Davis, Kevin Ault, Walter Orenstein and Saad B. Omer Pediatrics 2012;130;798 DOI: 10.1542/peds.2012-1516 originally published online October 15, 2012;

Updated Information & including high resolution figures, can be found at: Services http://pediatrics.aappublications.org/content/130/5/798 References This article cites 29 articles, 3 of which you can access for free at: http://pediatrics.aappublications.org/content/130/5/798#BIBL Subspecialty Collections This article, along with others on similar topics, appears in the following collection(s): Infectious Disease http://www.aappublications.org/cgi/collection/infectious_diseases_su b Vaccine/Immunization http://www.aappublications.org/cgi/collection/vaccine:immunization _sub Permissions & Licensing Information about reproducing this article in parts (figures, tables) or in its entirety can be found online at: http://www.aappublications.org/site/misc/Permissions.xhtml Reprints Information about ordering reprints can be found online: http://www.aappublications.org/site/misc/reprints.xhtml

Downloaded from www.aappublications.org/news by guest on February 5, 2019 Sexual Activity−Related Outcomes After Human Papillomavirus Vaccination of 11- to 12-Year-Olds Robert A. Bednarczyk, Robert Davis, Kevin Ault, Walter Orenstein and Saad B. Omer Pediatrics 2012;130;798 DOI: 10.1542/peds.2012-1516 originally published online October 15, 2012;

The online version of this article, along with updated information and services, is located on the World Wide Web at: http://pediatrics.aappublications.org/content/130/5/798

Pediatrics is the official journal of the American Academy of Pediatrics. A monthly publication, it has been published continuously since 1948. Pediatrics is owned, published, and trademarked by the American Academy of Pediatrics, 141 Northwest Point Boulevard, Elk Grove Village, Illinois, 60007. Copyright © 2012 by the American Academy of Pediatrics. All rights reserved. Print ISSN: 1073-0397.

Downloaded from www.aappublications.org/news by guest on February 5, 2019 Law and the Public’s Health

Public Health Reports 2016, Vol. 131(5) 728-731 ª 2016, Association of Schools and Human Papillomavirus and Mandatory Programs of Public Health. All rights reserved. Immunization Laws: What Can We Reprints and permission: sagepub.com/journalsPermissions.nav Learn From Early Mandates? DOI: 10.1177/0033354916663184 phr.sagepub.com

Leila Barraza, JD, MPH1, Kim Weidenaar, JD2, Doug Campos-Outcalt, MD, MPA3, and Y. Tony Yang, ScD, LLM, MPH4

Human papillomavirus (HPV) is a sexually transmitted virus least 1 dose of HPV vaccine and 40% received all 3 doses,7 that infects an estimated 14 million Americans annually.1 whereas 86% received the DTaP (diphtheria, tetanus, acel- More than 150 subtypes of HPV have been identified. Some lular pertussis) vaccine.8 types are carcinogenic and cause an estimated 27,000 can- HPV vaccination has been controversial in the United cers among men and women in the United States annually; States because the vaccine touches on 2 contentious topics: non-carcinogenic subtypes can cause anogenital warts.2 teenage sexuality and mandatory vaccination. Because of the The first HPV vaccine, a quadrivalent (HPV4) vaccine, public’s perceptions about HPV’s status as a sexually trans- licensed in 2006, offers protection against 4 subtypes of HPV mitted virus and dissent about the recommended age of vac- (6, 11, 16, and 18) that cause 70% of cervical cancers and cination, states have had difficulty making the vaccine 90% of genital warts. The vaccine is administered as a series mandatory for school admission. Some opponents of a school of 3 injections at 0, 2, and 6 months. A second HPV vaccine, entry mandate for this vaccine have argued that it infringes which offers protection against subtypes 16 and 18 and pro- upon parental rights to discuss the topic of sex on their own tects against cervical cancer but not genital warts, was terms.9 Others have raised concerns that HPV vaccination licensed in 2009.2 The quadrivalent vaccine, HPV4, is now may increase teenage promiscuity,9 although evidence dis- being replaced with HPV9, a vaccine that protects against 9 proves this claim.10 subtypes of HPV (6, 11, 16, 18, and 5 others) that cause 85% of cervical cancers.3 HPV Vaccination School Entry Mandates The Centers for Disease Control and Prevention’s Advi- sory Committee on Immunization Practices (ACIP) recom- Vaccine requirements for school entry are a proven method for increasing child immunization rates and decreasing the mended in 2007 that all females receive an HPV vaccine at 11-13 11-12 years of age so that protection is gained before expo- incidence of vaccine-preventable diseases (VPDs). Laws sure to the virus is likely to occur. The current recommen- vary, however, because each state establishes its own vaccine dations include all adolescents and the vaccine is also requirements for school entry. All states provide exceptions recommended for unvaccinated women through 26 years of from vaccination for medical reasons: as of July 2016, 47 states provided religious exceptions and 18 states allowed age, heterosexual males through 21 years of age, and men 14 2 exemptions for personal beliefs. States with exemptions who have sex with men through 26 years of age. HPV vaccines are highly effective in preventing infection if administered before exposure. A 2016 study found that 1 Mel and Enid Zuckerman College of Public Health, University of Arizona, HPV infection rates decreased from 11.5% in 2003-2006 to Tucson, AZ, USA 4.3% in 2009-2012 among 14- to 19-year-old girls after the 2 Public Health Law & Policy Program, Sandra Day O’Connor College of 4 Law, Arizona State University, Phoenix, AZ, USA vaccine became available. Evidence supports the vaccine’s 3 safety; the only serious adverse reactions associated with the Mel and Enid Zuckerman College of Public Health, University of Arizona, 2,5,6 Phoenix, AZ, USA vaccine are syncope and skin infections. Despite strong 4 George Mason University College of Health and Human Services, Fairfax, evidence of the vaccine’s effectiveness and safety, however, VA, USA public acceptance and uptake of the vaccine are slow. Although the proportion of adolescents receiving other rec- Corresponding Author: Leila Barraza, JD, MPH, Mel and Enid Zuckerman College of Public Health, ommended vaccines is increasing, the proportion of adoles- 6 Department of Community, Environment & Policy, University of Arizona, cents receiving the HPV vaccine appears to be plateauing. 1295 N Martin, PO Box 245210, Tucson, AZ 85724, USA. In 2014, 60% of adolescents aged 13-17 years received at Email: [email protected] Barraza et al 729 for personal beliefs have higher rates of unvaccinated chil- Columbia in 2009; Rhode Island’s regulation became effec- dren and higher incidence rates of VPDs.2,11,12 tive in 2015.15 A key difference between these 3 jurisdictions School immunization requirements serve 2 purposes: (1) was the process for establishing such a mandate. Although to protect children from infectious diseases while in a setting Virginia and the District of Columbia passed laws through with high rates of disease transmission and (2) to achieve the legislative process, Rhode Island took a regulatory higher immunization rates in society for better herd immu- approach through the rule-making authority of the Rhode nity and lower disease rates. An HPV vaccination require- Island Department of Health. Another key difference was the ment would be added primarily for the latter reason, because populations to which the requirements apply. The mandates the risk of transmitting the virus at school (i.e., through sex- for the District of Columbia and Rhode Island apply to both ual routes) is low compared with the risk of transmitting sexes, whereas Virginia’s mandate only applies to girls.22-24 infections through respiratory or fecal–oral routes. However, The District of Columbia’s original mandate, which was many states require vaccination against other diseases with passed in 2007, applied only to girls.25 However, in 2014, routes of infection that are similar to the route of infection of the rule was amended to clarify that religious exemptions and HPV (e.g., hepatitis B). Nevertheless, parents have expressed opt-outs to the HPV vaccination needed to be filed each year concerns about vaccinating their adolescent children against they are claimed and also expanded HPV vaccination to HPV, including not knowing about the vaccine, believing the include all children in grades 6 through 12.24,26,27 vaccine is unnecessary, questioning the safety of the vaccine, The 3 jurisdictions also differ in their timing of the vac- and not receiving a recommendation from their health care cine requirement. In Rhode Island, students must receive the provider.2,6 first dose before entering the seventh grade, and the series Legislators and health officials have recently increased must be completed before entering the ninth grade.23 In Vir- their attention toward HPV vaccination. In the past 10 years, ginia, the first of 3 doses must be administered before enter- 42 states have seen legislation proposed on HPV vaccination, ing the sixth grade, and in the District of Columbia, students many of which included unsuccessful attempts to require entering the sixth grade must receive the first of 3 doses at HPV vaccination for school enrollment. Texas was the first 11 years of age.22,24 state to require HPV vaccination through an executive order Although all 3 jurisdictions require the HPV vaccination in 2007, but the legislature overrode the order later that for school entry, they also have broad exemptions to the yaer.15 Now, however, 10 years after approval of the vaccine requirements, which could temper their impact. Virginia and ACIP recommendations, and despite a finding that three- Governor Timothy Kaine’s amendment to Virginia’s legis- quarters of physicians who provide care for adolescents favor lation (later approved by the General Assembly) allows par- a school entry requirement for HPV vaccination,16 only Vir- ents to exempt their child from the vaccination requirement ginia, the District of Columbia, and Rhode Island currently after merely reviewing educational materials about the virus require HPV vaccination for school entry.15 and signing a waiver.22,28,29 Virginia’s law does not require documentation of the vaccination, unlike with other vac- cines; rather, parents and guardians are encouraged to pro- A Tale of 3 Mandates vide such documentation upon their child’s entry to school.28 States have the authority to mandate certain vaccines based In the District of Columbia, the Committee on Health on their inherent police powers.17,18 However, states have amended its bill to add 3 exceptions to the HPV vaccination been slow to mandate HPV vaccination for school entry. requirement: (1) when the parent or guardian objected in Uptake of the HPV vaccination 8 years after the ACIP rec- good faith, in writing, asserting that the vaccine would vio- ommendation for its administration to adolescent girls was late his or her religious beliefs; (2) when the child’s physi- slow compared with uptake of several other adolescent vac- cian certified in writing that the vaccination would be cines.19 Although many other vaccines were incorporated medically inadvisable; and (3) when the parent or guardian into most state vaccination laws within 8 years of the rec- opted out for any reason by signing a document stating that ommendation, the HPV vaccine has not followed this path. the parent or guardian was informed of the vaccination Commentaries have suggested that one obstacle to uptake of requirement and chose not to participate.30,31 the HPV vaccine was the movement by many states, includ- Rhode Island’s HPV vaccination requirement is subject to ing Texas, to mandate the vaccine for school entry so soon the state’s exclusion, waiting period, and exemption after the vaccine’s approval and ACIP recommendation. rules.23,32,33 Accordingly, students are exempt from the vac- Many public health experts believe that vaccines should be cine only if they have a medical exemption signed by certain mandated only after adequate financing, supply, and ‘‘evi- health practitioners (e.g., physician, physician assistant, reg- dence of long-term safety’’ are established.20 In addition, istered nurse practitioner) or if the parent or guardian attests many individuals who were concerned about the new vaccine that the immunization conflicts with his or her religious viewed the involvement of Merck, the vaccine manufacturer beliefs.33 However, a spokeswoman for the Rhode Island of HPV4 and HPV9, in state mandate efforts negatively.21 Department of Health suggested that it will not prohibit HPV vaccination requirements for school entry were suc- schoolchildren who are not vaccinated against HPV from cessfully implemented in Virginia in 2008 and the District of entering school, asserting that ‘‘no one’s child is being forced 730 Public Health Reports 131(5) to be vaccinated against HPV,’’ and that parents can exempt that considerable public health benefits are not being rea- their child ‘‘if they feel a deep conviction that HPV vaccina- lized; many vaccine-preventable cancers caused by HPV will tion is not right for their child.’’34 Whether the spokeswoman occur. Although numerous jurisdictions have faced difficulty was referring to the existing religious and medical exemption passing an HPV vaccination mandate for school entry, now is found in the regulations or if the Rhode Island Department an opportune time to move forward. Experts suggest that of Health intends to treat the vaccine differently in practice attempts to mandate the HPV vaccination failed because is unclear. such attempts were made too closely after FDA approval and the ACIP recommendation. Ten years later, ample evidence supports the safety and effectiveness of HPV vaccines. Man- Lessons Learned dating HPV vaccination for school entry is a move that will Ten states unsuccessfully proposed legislation on HPV in protect the public’s health by preventing HPV-related mor- 2015-2016, but only 3 states (Hawaii, New Jersey, and New bidity and mortality. York) would have mandated HPV vaccination for school 15 entry. Going forward, other states seeking to create a man- Declaration of Conflicting Interests date similar to those in Virginia, the District of Columbia, or The authors declared no potential conflicts of interest with respect Rhode Island need to look to these jurisdictions for lessons to the research, authorship, and/or publication of this article. learned. One lesson is to gain the support of influential orga- nizations. For example, the American Cancer Society pro- Funding vided a statement supporting the legislation in the District of Columbia, asserting that (1) the US Food and Drug Admin- The authors received no financial support for the research, author- ship, and/or publication of this article. istration (FDA) determined the vaccine to be safe and effec- tive for females aged 9-26 years and (2) the vaccine ‘‘is an extraordinary biomedical advance and holds remarkable References potential for preventing the most common kinds of cervical 1. Satterwhite CL, Torrone E, Meites E, et al. Sexually trans- cancer.’’31 Despite the support of multiple organizations, mitted infections among US women and men: prevalence and however, opposition will inevitably arise and will need to incidence estimates, 2008. Sex Transm Dis. 2013;40:187-193. be addressed. 2. Markowitz LE, Dunne EF, Saraiya M, et al. Human papilloma- Another lesson is that compromises may be needed to virus vaccination: recommendations of the Advisory Commit- ensure passage. Virginia’s General Assembly compromised tee on Immunization Practices (ACIP). Morb Mortal Wkly Rep. on several key elements of the legislation, creating ‘‘safe- 2014;63(RR-05):1-30. Erratum in 2014;63(49):1182. guards to make sure it was a cautious [sic] approach.’’ The 3. Petrosky E, Bocchini JA Jr, Hariri S, et al. Use of 9-valent first safeguard was to delay the effective date of the require- human papillomavirus (HPV) vaccine: updated HPV vaccina- ment. Although it was passed in 2007, the requirement tion recommendations of the Advisory Committee on Immuni- became effective on October 1, 2008, meaning that it did not zation Practices. Morb Mortal Wkly Rep. 2015;64(11):300-304. go into effect until the start of the 2009-2010 school year, 4. Markowitz LE, Liu G, Hariri S, Steinau M, Dunne EF, Unger which allowed 2 years for parents to prepare for the ER. Prevalence of HPV after introduction of the vaccination requirement.28 program in the United States. Pediatrics. 2016;137:1-9. Furthermore, even after a mandate is passed, jurisdictions 5. Centers for Disease Control and Prevention. Human papillo- should be aware that legislative opposition may continue. mavirus (HPV) vaccine safety. http://www.cdc.gov/vaccinesaf Every year since passage of the HPV vaccination mandate ety/vaccines/hpv-vaccine.html. Accessed April 1, 2016. in Virginia, members of the General Assembly have intro- 6. Sukumaran L. Human papillomavirus (HPV) vaccine safety duced legislation to repeal the law, but such attempts have not update. Presented to Advisory Committee on Immunization succeeded.15 A bill introduced in the Rhode Island General Practices. October 21, 2015. http://www.cdc.gov/vaccines/ Assembly in February 2016 would restrict the authority of the acip/meetings/downloads/slides-2015-10/hpv-04-sukumaran. Rhode Island Department of Health from setting immuniza- pdf. Accessed April 1, 2016. tion standards for ‘‘diseases which are not transmissible in a 7. Centers for Disease Control and Prevention. Estimated vacci- school environment,’’ and would have explicitly prohibited nation coverage with selected vaccines among adolescents the Rhode Island Department of Health from mandating the aged 13-17 years, by state and selected area—National Immu- HPV vaccination as a requirement for school entry (the bill nization Survey-Teen, United States, 2014. 2015. http://www. was held for further study on March 31, 2016).35 cdc.gov/vaccines/imz-managers/coverage/nis/teen/tables/14/ tab01_iap_2014.pdf. Accessed April 1, 2016. 8. Reagan-Steiner S. HPV vaccination coverage among U.S. Public Health Implications adolescents: results from the National Immunization Survey- Coverage estimates for HPV vaccination are low despite Teen, 2014. October 1, 2015. http://www.cdc.gov/vaccines/ evidence of the vaccine’s effectiveness and safety. This large acip/meetings/downloads/slides-2015-10/hpv-02-reagan-stei pool of unvaccinated adolescents in the United States means ner.pdf. Accessed April 1, 2016. Barraza et al 731

9. Lo B. HPV vaccine and adolescents’ sexual activity. BMJ. 21. Mello MM, Abiola S, Colgrove J. Pharmaceutical companies’ 2006;332:1106-1107. role in state vaccination policymaking: the case of human 10. Jena AB, Goldman DP, Seabury SA. Incidence of sexually papillomavirus vaccination. AmJPublicHealth. 2012;102: transmitted infections after human papillomavirus vaccination 893-898. among adolescent females. JAMA Intern Med. 2015;175: 22. Va. Code Ann. §32.1-46 (2007). 617-623. 23. R.I. Admin. Code 31-1-38:3.3.7 (2014). 11. Bradford WD, Mandich A. Some state vaccination laws con- 24. 22-B D.C. Mun. Regs. §146 (2014). tribute to greater exemption rates and disease outbreaks in the 25. 22-B D.C. Mun. Regs. §146 (2008). United States. Health Aff (Millwood). 2015;38:1383-1390. 26. District of Columbia Department of Health. Notice of final 12. Wang E, Clymer J, Davis-Hayes C, Buttenheim A. Nonmedical rulemaking—human papillomavirus immunizations (PHV) exemptions from school immunization requirements: a sys- amendment. December 19, 2014. http://www.dcregs.dc.gov/ tematic review. Am J Public Health. 2014;104:e62-e84. Gateway/RuleHome.aspx?RuleID¼948580. Accessed March 13. Community Preventive Services Task Force. Increasing appro- 18, 2016. priate vaccinations: vaccination requirements for child care, 27. 22-B D.C. Mun. Regs. §129.8 (2014). school, and college attendance. The Community Guide. 28. Joint Legislative Audit and Review Commission of the Virgi- http://www.thecommunityguide.org/vaccines/require nia General Assembly. Evaluation of House Bill 2877: man- ments_school.html. Accessed April 5, 2015. dated coverage of human papillomavirus (HPV) vaccine. 14. National Conference of State Legislatures. States with religious September 2007. http://jlarc.virginia.gov/pdfs/reports/Rpt356. and philosophical exemptions from school immunization pdf. Accessed April 1, 2016. requirements. January 21, 2016. http://www.ncsl.org/research/ 29. H.B. 2035, Bill Tracking (Virginia 2007). https://lis.virginia. health/school-immunization-exemption-state-laws.aspx. gov/cgi-bin/legp604.exe?071þsumþHB2035. Accessed Accessed March 22, 2016. August 19, 2016. 15. National Conference of State Legislatures. HPV vaccine: state 30. Bill B17-30. 17, 70th Council Session (D.C. 2007). legislation and statutes. June 28, 2016. http://www.ncsl.org/ 31. Council of the District of Columbia, Committee on Health, research/health/hpv-vaccine-state-legislation-and-statutes. Committee Report on B17-30 (March 9, 2007). aspx. Accessed June 28, 2016. 32. R.I. Admin. Code 31-1-38:3.3.5.3(a) (2014). 16. Califano S, Calo WA, Weinberger M, Gilkey MB, Brewer NT. 33. Rhode Island Department of Health. Rules and Regulations Physician support of HPV vaccination school-entry require- Pertaining to Immunization and Communicable Disease Test- ments. Hum Vaccin Immunother. 2016;12:1626-1632. ing in Preschool, School, Colleges or Universities. Amended in 17. US Constitution, 10th Amendment. July 2014. Providence: Rhode Island Department of Health; 18. Jacobson v. Massachusetts, 197 U.S. 11 (1905). 1995. 19. Schwartz JL, Easterling LA. State vaccination requirements for 34. Mcdermott J. Health officials say they won’t change HPV vac- HPV and other vaccines for adolescents, 1990–2015. JAMA. cine mandate. Business Insider. August 17, 2015. http://www. 2015;314:185-186. businessinsider.com/ap-health-officials-say-they-wont- 20. Schwartz JL. HPV vaccination’s second act: promotion, com- change-hpv-vaccine-mandate-2015-8. Accessed March 31, petition, and compulsion. AmJPublicHealth. 2010;100: 2016. 1841-1844. 35. S.B. 2295, 2016 Leg. Sess. (R.I. 2016). Opinion

VIEWPOINT Human Papillomavirus Vaccination and School Entry Requirements Politically Challenging, but Not Impossible

Ellen Daley, PhD, MPH Anogenital and oropharyngeal cancers that are attrib- Limiting discussions to only school-located pro- Community and Family uted to oncogenic strains of the human papillomavirus grams does not encompass the broader approach of Health Concentration, (HPV) number approximately 31 500 per year in the implementing school entry requirements for HPV vac- College of Public United States (https://www.cdc.gov/cancer/hpv/ cination, which still involves the same interaction Health, University of South Florida, Tampa. statistics/cases.htm), and despite the availability of a safe between parents and health care clinicians. In fact, a and effective HPV vaccine that protects against these school-entryapproachhastheadded“incentive”forboth Erika Thompson, PhD, strains, the United States has a disappointing record of parents and clinicians that the vaccine is now required MPH vaccine uptake after more than a decade of promoting before beginning middle school. At the time of the Presi- Department of Health the vaccine for adolescents. dent’s Cancer Panel report, there had been limited suc- Behavior & Health Systems, School of To date, national strategies to improve HPV vac- cess with HPV vaccine school entry policies, and until the Public Health, cine uptake have centered primarily on health care cli- recent program enacted in Rhode Island in 2015, it would University of North nicians (pediatricians, obstetrician-gynecologists, and have been difficult to provide an evidence-based argu- Texas Health Science Center, Fort Worth. family practice physicians) and campaigns to encour- ment for adding these policies to the broader strate- age and train primary care practitioners on HPV vacci- gies. Virginia and the District of Columbia instituted ear- Gregory Zimet, PhD nation; the returns on these campaigns have been mod- lier policies in 2008 and 2009, respectively, but with Department of est. Data from 2016 showed that 60.4% of adolescents generous opt-out provisions and, in the case of Vir- Pediatrics, Indiana aged 13 to 17 years received at least 1 dose of the vac- ginia, no consequences for nonvaccination. Not surpris- University School of Medicine, Indianapolis. cine, and only 43.4% of adolescents were up to date with ingly,these policies were not particularly effective, dem- the vaccine.1 Focusing solely on individual- and inter- onstrating that limited policies do not work.4 personal-level strategies (ie, the medical clinician and the Furthermore, evidence with other vaccines shows that parent) limits the opportunity to reach the level of HPV broadening opt-out provisions leads to lower vaccina- vaccine coverage that is desired in the United States and tion rates and outbreaks of vaccine-preventable that has been achieved by other nations that have de- infections.5 However, the Rhode Island program re- veloped successful HPV immunization programs, such quires HPV vaccine initiation before seventh grade, is as Australia, Canada, and the United Kingdom. gender-neutral, and has also implemented a more re- After more than a decade of promoting a strategy strictive exemption strategy to opt out of vaccinations that is not sufficient, additional approaches to improv- than previous programs, such as the one implemented ing vaccine uptake should be considered. One of the in Virginia.6 Results of the Rhode Island initiative are im- most obvious tactics would be promoting and imple- pressive, with a vaccination initiation rate of 90% and menting state-level school entry requirements for the 88% among girls and boys, respectively,1 and an in- HPV vaccine along with other adolescent vaccines, such creased predicted vaccine initiation rate of 11% among as the tetanus-diphtheria-pertussis vaccine, that al- adolescent boys compared with boys in other states af- ready are required for middle school entry in most ter the policy implementation.7 No similar change was states.2 However, discussions related to including HPV found for girls in Rhode Island, as the vaccination rate vaccine school entry requirements at a policy level are was already high, which was likely the result of a “ceil- not only rare, but are also often overlooked, such as in ing effect” of an already impressive vaccine program. the case of the 2014 report of the President’s Cancer However, replicating this approach of gender-neutral Panel.3 The panel described the low vaccine rates and school entry has the potential to improve uptake over- recommended 3 goals for increasing rates in the United all and may help ameliorate gender disparities in vacci- States that focused on clinician-based vaccination strat- nation. egies, such as developing communication programs that Promoting school entry campaigns on a state-by- areassociatedwiththeHPVvaccineforphysicians.There state basis may seem daunting, but 2 factors may miti- is a brief mention of school-based and school-located gate that concern: the continued efforts to improve the programs, which have been successful in other coun- current strategy of clinician recommendation and the Corresponding tries, but the panel concluded that there are too many successful efforts, such as the Rhode Island program, Author: Ellen Daley, PhD, MPH, University barriers to implementing those programs in the United that may encourage vaccine coalitions in other states to of South Florida College States. However, there was a caveat that stated “Fur- promote similar programs. At present, statewide ef- of Public Health, 13201 thermore, if vaccination rates in the United States do not forts may lack the collaborative political will to pursue Bruce B. Downs Blvd, improve dramatically over the next several years, the fea- HPV vaccine school entry requirements, but supple- MDC 56, Tampa, FL 33612 (edaley sibility of school-located vaccination should be menting the current efforts is critical. Depending solely @health.usf.edu). reexamined.” 3 on clinician recommendations for the HPV vaccination

6 JAMA Pediatrics January 2019 Volume 173, Number 1 (Reprinted) jamapediatrics.com

© 2018 American Medical Association. All rights reserved. Downloaded from jamanetwork.com by Yale University user on 02/05/2019 Viewpoint Opinion

has proven insufficient. Unlike clinician rates for required vaccines, What we do know is that when school entry programs are imple- which exceed 80%, the rates of HPV vaccination are both lower and mented correctly in the United States, they work well. Despite the inconsistent.1 Moreover, clinician recommendations and school en- barriers to instituting school entry requirements, we should not stop try requirements can be complementary HPV vaccination ap- promoting them just because it is a challenging task. To paraphrase proaches. When school entry vaccination becomes a policy, it the President’s Cancer Panel of 2014, vaccination rates in the United removes the burden from the clinician to recommend the vaccine States have not improved dramatically in the past several years since and encourages parents to vaccinate their children. These 2 that report was written. Thus, reexamining strategies that are as- approaches would act complementarily with one another. sociated with school-based and school entry programs is overdue. So what is the way forward? First, get the critical medical pro- There is evidence that school entry requirements can be effective fessional organizations, such as the American Medical Association, tools in complementing the existing clinician-focused approach. To the American Academy of Pediatrics, the American College of protect children from preventable cancers, such as those caused by Obstetricians and Gynecologists, and the American Academy of Fam- HPV, we need a robust, multilayered plan that includes ongoing re- ily Physicians, to endorse a move toward school entry require- search into public opinions about the vaccine and school entry re- ments for HPV vaccination. Commendations from these groups will quirements and the development of effective strategies to commu- provide the needed reassurance to parents and clinicians that this nicate with the public about the value of HPV vaccine school entry vaccine is as important and necessary as any other vaccine that their requirements for promoting personal and public health. In addi- child receives. Second, identify stakeholders statewide, including lo- tion, the promotion of school entry requirements should include cal vaccine coalitions, who can work together with policy makers to strict opt-out provisions that are similar to other vaccines, contin- implement a state-level school entry requirement. This effort to cre- ued training and implementation programs for health care clini- ate the political will that is necessary for a new policy is critical and cians, and messaging campaigns to encourage parents to vacci- would also act to reassure parents and clinicians. nate their children against HPV.

ARTICLE INFORMATION REFERENCES 2009-2013 National Immunization Survey-Teen. Published Online: November 5, 2018. 1. Walker TY, Elam-Evans LD, Singleton JA, et al. Hum Vaccin Immunother. 2016;12(6):1615-1622. doi:10.1001/jamapediatrics.2018.3327 National, regional, state, and selected local area doi:10.1080/21645515.2016.1150394 Conflict of Interest Disclosures: Dr Daley has vaccination coverage among adolescents aged 13-17 5. Atwell JE, Van Otterloo J, Zipprich J, et al. served as a member of the US Human years—United States, 2016. MMWR Morb Mortal Nonmedical vaccine exemptions and pertussis in Papillomavirus Vaccine advisory board (Science and Wkly Rep. 2017;66(33):874-882. doi:10.15585/ California, 2010. Pediatrics. 2013;132(4):624-630. Policy) of Merck Pharmaceutical Corporation and mmwr.mm6633a2 doi:10.1542/peds.2013-0878 received travel support and a stipend from Merck 2. Ferris DG. Change suboptimal tactics and 6. Barraza L, Weidenaar K, Campos-Outcalt D, for participating on this board. Dr Zimet has served promote a national mandatory human Yang YT. Human papillomavirus and mandatory as an investigator on investigator-initiated human papillomavirus vaccination program. J Low Genit immunization laws: what can we learn from early papillomavirus vaccine research funded by Merck Tract Dis. 2016;20(4):348-351. doi:10.1097/LGT. mandates? Public Health Rep. 2016;131(5):728-731. Pharmaceutical Corporation and received travel 0000000000000247 doi:10.1177/0033354916663184 support from Merck to present research findings at 3. Accelerating HPV Vaccine Uptake. Urgency for 7. Thompson EL, Livingston MDI, Daley EM, a scientific meeting. He has also received an Action to Prevent Cancer. A Report to the President Zimet GD. Human papillomavirus vaccine initiation honoraria for serving as a member of the of the United States from the President’s Cancer Panel. for adolescents following Rhode Island’s Adolescent Immunization Initiative, which is Bethesda, MD: National Cancer Institute; 2014. school-entry requirement, 2010-2016 [published cosponsored by Sanofi Pasteur and the 4. Perkins RB, Lin M, Wallington SF, Hanchate AD. online July 19, 2018]. Am J Public Health. doi:10. Immunization Action Coalition. No other 2105/AJPH.2018.304552 disclosures are reported. Impact of school-entry and education mandates by states on HPV vaccination coverage: analysis of the

jamapediatrics.com (Reprinted) JAMA Pediatrics January 2019 Volume 173, Number 1 7

© 2018 American Medical Association. All rights reserved. Downloaded from jamanetwork.com by Yale University user on 02/05/2019 AJPH OPEN-THEMED RESEARCH

Human Papillomavirus Vaccine Initiation for Adolescents Following Rhode Island’s School-Entry Requirement, 2010–2016

Erika L. Thompson, PhD, MPH, Melvin D. Livingston III, PhD, Ellen M. Daley, PhD, MPH, and Gregory D. Zimet, PhD

Objectives. To assess changes in human papillomavirus (HPV) vaccine initiation for vaccination rates before implementation of adolescent girls and boys in Rhode Island compared with all other states. a policy, and the studies were carried out Methods. We estimated the gender-specific effects of Rhode Island’s school-entry before the Rhode Island requirement was HPV vaccination policy on self-reported HPV vaccination initiation by using a difference- implemented and therefore focused on only in-differences design with the National Immunization Survey–Teen from 2010 through the policies in Virginia and the District of Columbia.6,7 A recent qualitative analysis of 2016. policies found that a multifactorial approach Results. Compared with boys in other states, boys in Rhode Island increased their HPV to policy may be most effective at promoting fi vaccine initiation rate by 11% (b = 0.11; 95% con dence interval [CI] = 0.05, 0.18) after HPV vaccination but also recommended enactment of the requirement. No difference was seen in the probability of HPV vaccine complementary statistical analyses to evaluate initiation among girls in Rhode Island compared with girls in the multistate control the effects of HPV vaccine policy.8 Given the (b = –0.01; 95% CI = –0.08, 0.05). recent enactment of Rhode Island’s HPV Conclusions. Our analysis identified an 11% increase in HPV vaccine initiation rate vaccine school-entry requirement, we among boys in Rhode Island after the school-entry requirement was enacted, whereas no assessed changes in HPV vaccine initiation for significant change was observed for girls. adolescent girls and boys in Rhode Island Public Health Implications. Given suboptimal vaccine uptake rates in the United States, compared with all other states. continued pursuit of state-level public policy to improve HPV vaccination is needed. School-entry requirements for HPV vaccination may be a strategy for closing the gap in HPV vaccine uptake for boys and girls. (Am J Public Health. 2018;108:1421–1423. doi: 10.2105/AJPH.2018.304552) METHODS We used parental report of HPV vacci- nation initiation from the National Immu- nization Survey–Teen (NIS-Teen) from he human papillomavirus (HPV) vaccine an HPV vaccine school-entry requirement; it 2010 through 2016 to estimate the gender- T fi is approved for the prevention of requires receipt of the rst dose of the vaccine specific effects of Rhode Island’s school-entry HPV-attributable cancers (i.e., cervical, before entering seventh grade and comple- HPV vaccination policy with a difference-in- vaginal, vulvar, anal, and penile) and genital tion of the series before entering ninth grade.4 1 differences design. The difference-in-differ- warts. Despite the well-established safety and Moreover, it uses a narrow definition of ences design treats Rhode Island’s policy effectiveness of this vaccine, uptake has been exemption—making it more difficult to opt change as a natural experiment, comparing suboptimal in the United States. In 2016, out of vaccination.5 the change in parental reporting of HPV uptake (‡ 1 dose) among US adolescents was 2 Previous studies evaluated the imple- vaccination among adolescents in Rhode 65.1% for girls and 56.0% for boys. Differ- mentation of school-entry requirements and Island with the change in HPV vaccination in ences in male and female vaccine uptake are other HPV vaccine state policies. However, a control group of all other states before and attributed primarily to a delay in HPV vaccine these studies did not take into account the after Rhode Island’s policy change. To approval among males.3 Few states have enacted policies for HPV ABOUT THE AUTHORS vaccine school-entry requirements. Virginia Erika L. Thompson is with the Department of Health Behavior & Health Systems, School of Public Health, University of North and the District of Columbia were early Texas Health Science Center, Fort Worth. Melvin D. Livingston III, is with the Department of Biostatistics and Epidemiology, adopters of HPV vaccine school requirements School of Public Health, University of North Texas Health Science Center. Ellen M. Daley is with the Department of Community and Family Health, College of Public Health, University of South Florida, Tampa. Gregory D. Zimet is with the in 2008 and 2009, respectively. However, in Department of Pediatrics, Indiana University School of Medicine, Indianapolis. both cases, the requirements focus on only Correspondence should be sent to Erika L. Thompson, PhD, MPH, Department of Health Behavior & Health Systems, School of girls and involve broadly defined, easily ob- Public Health, University of North Texas Health Science Center, 3500 Camp Bowie Blvd, EAD709, Fort Worth, TX 76107 (e-mail: [email protected]). Reprints can be ordered at http://www.ajph.org by clicking the “Reprints” link. tained exemptions, particularly in the District This article was accepted May 12, 2018. of Columbia. In 2015, Rhode Island enacted doi: 10.2105/AJPH.2018.304552

October 2018, Vol 108, No. 10 AJPH Thompson et al. Peer Reviewed Research 1421 AJPH OPEN-THEMED RESEARCH

100

90 Boys (RI) Boys (control) Girls (RI) Girls (control)

80

70

60

50

40

30

20

Predicted HPV Vaccination Initiation, % Predicted HPV Vaccination 10

0 2010 2011 2012 2013 2014 2015 2016 Year

FIGURE 1—Predicted Percentage of Human Papillomavirus (HPV) Vaccine Initiation Stratified by Policy Enactment and Gender: National Immunization Survey–Teen, Rhode Island and United States, 2010–2016 further strengthen our quasi-experimental throughout the calendar year. Consequently, age, race/ethnicity, income, mother’s edu- approach, we allowed for differential trends in adolescents whose parents were surveyed in cation, and number of health care visits. dj, gt, * * HPV vaccine initiation over time by gender the first half of 2015 were not subject to the Gender dj, and Gender gt are gender- and controlled for individual-level demo- mandate, but those whose parents were specific state and year fixed effects. These state graphic covariates. surveyed in the second half of 2015 were and year fixed effects allow for differential subject to the mandate. To be conservative, baseline HPV vaccination rates for boys and our primary coding of the policy indicator for girls in each state, as well as gender-specific Data Rhode Island’s HPV requirement included nonlinear trends over time. We examined individual data from the only adolescents surveyed in 2016 as being All analyses accounted for the complex NIS-Teen, an annual repeated cross-sectional subject to the mandate. We used a secondary survey design of NIS-Teen by using PROC study assessing vaccination initiation at the indicator including adolescents from both SURVEYREG in SAS version 9.4 (SAS In- national and state level for 13- to 17-year-old 2015 and 2016 during sensitivity analyses. stitute, Cary, NC). We used linear probability persons. HPV vaccination was determined by models to aid in the interpretation of the in- aparentalreportof“yes” to whether the ad- teraction term, although we found similar re- Statistical Analysis olescent had ever received any immunization sults with multiplicative models. In addition to To evaluate the gender-specific effects of for HPV. Gender was coded as male or female. determining model estimates, we estimated the Rhode Island’s HPV vaccination require- Age was coded in years. Race/ethnicity was predicted probability of an HPV vaccination in ment, we estimated the following linear coded as non-Hispanic White, non-Hispanic Rhode Island and the control group of all other probability model: Black, Hispanic, or other. Income was coded as states by gender and year. greater than $75 000, from the poverty line to Yijt ¼ b0 þ b1Mandateijt þ b2Genderijt $75 000, and below the poverty line. Mother’s þ b3Mandate Genderijt ð1Þ education was coded as fewer than 12 years, 12 þ b4Xijt þ dj þ gt þ Gender dj – years, more than 12 years but non college þ Gender gt RESULTS graduate, and college graduate. Number of Our analysis showed a statistically signifi- health care visits in the past 12 months was Yijt is an indicator for HPV vaccination for cant difference in the effect of Rhode Island’s treated as a continuous variable. person i in state j in year t. Mandateijt is the HPV vaccination requirement by gender Rhode Island’s HPV vaccination school- policy indicator for Rhode Island’s HPV (F1 = 7.38; P = .007). Compared with boys in entry requirement was officially enacted in vaccination mandate. Genderijt is the partic- other states, boys in Rhode Island increased August 2015, whereas NIS-Teen collects data ipant’s self-reported gender. Xijt is a vector of their predicted HPV vaccination initiation

1422 Research Peer Reviewed Thompson et al. AJPH October 2018, Vol 108, No. 10 AJPH OPEN-THEMED RESEARCH

rate by 11% (b = 0.11; 95% confidence in- PUBLIC HEALTH IMPLICATIONS 8. Roberts MC, Murphy T, Moss JL, Wheldon CW, Psek terval [CI] = 0.05, 0.18) after enactment of W. A qualitative comparative analysis of combined state Given suboptimal vaccine uptake rates in health policies related to human papillomavirus vaccine the requirement. No difference was seen in the United States, continued pursuit of uptake in the United States. Am J Public Health. 2018; – the predicted probability of HPV vaccination state-level public policy to improve HPV 108(4):493 499. among girls in Rhode Island compared with vaccination is needed, especially to achieve 9. Daley EM, Vamos CA, Zimet GD, Rosberger Z, girls in the multistate control (b = –0.01; 95% Thompson EL, Merrell L. The feminization of HPV: the Healthy People 2020 objective of 80% reversing gender biases in US human papillomavirus – 12 CI = 0.08, 0.05). However, girls in Rhode coverage for HPV vaccination completion. vaccine policy. Am J Public Health. 2016;106(6):983–984. Island maintained the highest HPV vaccina- In particular, school-entry requirements with 10. Reagan-Steiner S, Yankey D, Jeyarajah J, et al. Na- tion rates throughout the study period, with narrow opt-out policies, such as Rhode Is- tional, regional, state, and selected local area vaccination boys in Rhode Island achieving similar levels ’ coverage among adolescents aged 13-17 years - United land s, may be more successful; however, States, 2015. MMWR Morb Mortal Wkly Rep. 2016;65(33): of HPV vaccination after enactment of the these policies face more resistance for 850–858. requirement (Figure 1). implementation. 11. Ojha RP, Tota JE, Offutt-Powell TN, Klosky JL, We found similar results when we used the Ashokkumar R, Gurney JG. The accuracy of human policy indicator including 2015 and when we CONTRIBUTORS papillomavirus vaccination status based on adult proxy recall or household immunization records for adolescent All authors conceptualized and contributed to the design excluded jurisdictions that enacted a school- females in the United States: results from the National of the study, contributed to the analysis and interpretation entry requirement for HPV vaccination be- Immunization Survey-Teen. Ann Epidemiol. 2013;23(5): of the data, and approved the final version of the article. 281–285. fore the start of the study (Virginia, District of E. L. Thompson and M. D. Livingston III, also drafted Columbia) from the multistate control (data portions of the article. E. M. Daley and G. D. Zimet 12. Healthy People 2020. Immunization and infectious also revised the article. diseases. 2015. Available at: http://www.healthypeople. not shown). gov/2020/topics-objectives/topic/immunization-and- infectious-diseases/objectives. Accessed April 6, 2016. ACKNOWLEDGMENTS These data were derived from public use data sets on the 2010–2016 National Immunization Survey–Teen, available on the Centers for Disease Control and Pre- DISCUSSION vention Web site. Our analysis identified an 11% increase in HPV vaccine initiation among boys in Rhode HUMAN PARTICIPANT PROTECTION Island after the school-entry requirement was This study was considered exempt by the North Texas Regional institutional review board as this work did not enacted, whereas no significant change was involve human participant research. observed for girls. This set of findings indicates that school-entry requirements may reduce REFERENCES gender disparities and close the gap in HPV 1. Petrosky E, Bocchini JA Jr, Hariri S, et al. Use of 9 9-valent human papillomavirus (HPV) vaccine: updated vaccine uptake. We probably did not ob- HPV vaccination recommendations of the Advisory serve significant differences in HPV vaccine Committee on Immunization Practices. MMWR Morb initiation among girls because of their already Mortal Wkly Rep. 2015;64(11):300–304. high HPV vaccination rate (87.9%) in 2015.10 2. Walker TY, Elam-Evans LD, Singleton JA, et al. 8 National, regional, state, and selected local area vacci- Roberts et al. found that multiple policies nation coverage among adolescents aged 13-17 years— may be needed to improve HPV vaccination, United States, 2016. MMWR Morb Mortal Wkly Rep. but this study found that a policy focused 2017;66(33):874–882. solely on an HPV vaccination school-entry 3. Centers for Disease Control and Prevention. Rec- ommendations on the use of quadrivalent human pap- requirement may have an effect on HPV illomavirus vaccine in males—Advisory Committee on vaccine initiation, especially among boys. Immunization Practices (ACIP), 2011. MMWR Morb This study should be considered in the Mortal Wkly Rep. 2011;60(50):1705–1708. context of its limitations. We could not assess 4. Barraza L, Weidenaar K, Campos-Outcalt D, Yang YT. other school-entry requirements for HPV Human papillomavirus and mandatory immunization laws: what can we learn from early mandates? Public Health vaccination (i.e., Virginia, District of Co- Rep. 2016;131(5):728–731. lumbia) because data before these policies 5. Barraza L, Campos-Outcalt D. More research needed were implemented were not available. Ad- to increase policies for HPV vaccine uptake. Am J Public – ditionally, our reliance on self-reported Health. 2018;108(4):430 431. vaccination status is a limitation; however, 6. Moss JL, Reiter PL, Truong YK, Rimer BK, Brewer NT. School entry requirements and coverage of non- self-report has been shown to be reasonably targeted adolescent vaccines. Pediatrics. 2016;138(6):pii: accurate in the NIS-Teen.11 Finally, this e20161414. analysis captured the early implementation of 7. Perkins RB, Lin M, Wallington SF, Hanchate AD. the school-entry requirement, and future Impact of school-entry and education mandates by states on HPV vaccination coverage: analysis of the 2009-2013 research should continue to examine this National Immunization Survey-Teen. Hum Vaccin policy transition. Immunother. 2016;12(6):1615–1622.

October 2018, Vol 108, No. 10 AJPH Thompson et al. Peer Reviewed Research 1423 AJPH OPEN-THEMED RESEARCH

Weaponized Health Communication: Twitter Bots and Russian Trolls Amplify the Vaccine Debate

David A. Broniatowski, PhD, Amelia M. Jamison, MAA, MPH, SiHua Qi, SM, Lulwah AlKulaib, SM, Tao Chen, PhD, Adrian Benton, MS, Sandra C. Quinn, PhD, and Mark Dredze, PhD

Objectives. To understand how Twitter bots and trolls (“bots”) promote online health measles, mumps, and pertussis and increased content. mortality from vaccine-preventable diseases 14 Methods. We compared bots’ to average users’ rates of vaccine-relevant messages, such as influenza and viral pneumonia which we collected online from July 2014 through September 2017. We estimated the underscore the importance of combating likelihood that users were bots, comparing proportions of polarized and antivaccine online misinformation about vaccines. Much health misinformation may be tweets across user types. We conducted a content analysis of a Twitter hashtag asso- promulgated by “bots”15—accounts that ciated with Russian troll activity. automate content promotion—and “trolls”16— Results. Compared with average users, Russian trolls (c2(1) = 102.0; P < .001), so- individuals who misrepresent their identi- 2 P “ ” 2 P phisticated bots (c (1) = 28.6; < .001), and content polluters (c (1) = 7.0; < .001) ties with the intention of promoting discord. tweeted about vaccination at higher rates. Whereas content polluters posted more One commonly used online disinformation 2 antivaccine content (c (1) = 11.18; P < .001), Russian trolls amplified both sides. Un- strategy, amplification,17 seeks to create im- 2 identifiable accounts were more polarized (c (1) = 12.1; P < .001) and antivaccine pressions of false equivalence or consensus (c2(1) = 35.9; P < .001). Analysis of the Russian troll hashtag showed that its messages through the use of bots and trolls. We seek to were more political and divisive. understand what role, if any, they play in the Conclusions. Whereas bots that spread malware and unsolicited content disseminated promotion of content related to vaccination. antivaccine messages, Russian trolls promoted discord. Accounts masquerading as le- Efforts to document how unauthorized gitimate users create false equivalency, eroding public consensus on vaccination. users—including bots and trolls—have influ- Public Health Implications. Directly confronting vaccine skeptics enables bots to enced online discourse about vaccines have ’ legitimize the vaccine debate. More research is needed to determine how best to been limited. DARPA s (the US Defense combat bot-driven content. (Am J Public Health. 2018;108:1378–1384. doi:10.2105/ Advanced Research Projects Agency) 2015 Bot Challenge charged researchers with identifying AJPH.2018.304567) “influence bots” on Twitter in a stream of vaccine-related tweets. The teams effectively See also Sutton, p. 1281; and Galea and Vaughan, p. 1288. identified bot networks designed to spread vaccine misinformation,18 but the public health ealth-related misconceptions, mis- about vaccines is associated with increased community largely overlooked the implications Hinformation, and disinformation spread 8–10 vaccine hesitancy and delay. Vaccine- of these findings. Rather, public health research over social media, posing a threat to public hesitant parents are more likely to turn to the has focused on combating online antivaccine health.1 Despite significant potential to enable Internet for information and less likely to trust content, with less focus on the actors who dissemination of factual information,2 social health care providers and public health experts produce and promote this content.1,19 Thus, media are frequently abused to spread harmful 9,11 on the subject. Exposure to the vaccine theroleofbots’ and trolls’ online activity health content,3 including unverified and debate may suggest that there is no scientific pertaining to vaccination remains unclear. erroneous information about vaccines.1,4 This consensus, shaking confidence in vaccina- We report the results of a retrospective potentially reduces vaccine uptake rates and tion.12,13 Additionally, recent resurgences of observational study assessing the impact of increases the risks of global pandemics, es- pecially among the most vulnerable.5 Some of ABOUT THE AUTHORS this information is motivated: skeptics use David A. Broniatowski is with the Department of Engineering Management and Systems Engineering, School of Engineering 6 online platforms to advocate vaccine refusal. and Applied Science, The George Washington University, Washington, DC. Amelia M. Jamison and Sandra C. Quinn are Antivaccine advocates have a significant with the Department of Family Science, School of Public Health, University of Maryland, College Park. Sihua Qi and Lulwah 6 Alkulaib are with the Department of Computer Science, School of Engineering and Applied Science, The George Washington presence in social media, with as many as University. Tao Chen, Adrian Benton, and Mark Dredze are with the Department of Computer Science, Whiting School of 50% of tweets about vaccination containing Engineering, Johns Hopkins University, Baltimore, MD. antivaccine beliefs.7 Correspondence should be sent to David A. Broniatowski, 800 22nd St. NW #2700, Washington, DC 20052 (e-mail: [email protected]). Reprints can be ordered at http://www.ajph.org by clicking the “Reprints” link. Proliferation of this content has conse- This article was accepted May 22, 2018. quences: exposure to negative information doi: 10.2105/AJPH.2018.304567

1378 Research Peer Reviewed Broniatowski et al. AJPH October 2018, Vol 108, No. 10 AJPH OPEN-THEMED RESEARCH

bots and trolls on online vaccine discourse a supplement to the online version of this L. A.) coded relevant tweets as “provaccine,” on Twitter. Using a set of 1 793 690 tweets article at http://www.ajph.org). For each data “antivaccine,” or “neutral” using a codebook collected from July 14, 2014, through Sep- set, we extracted tweets from accounts known developed by 1 of the authors (A. M. J.). When tember 26, 2017, we quantified the impact of to be bots or trolls and identified in 7 publicly coders disagreed, we employed a second round – known and suspected Twitter bots and trolls available lists of Twitter user IDs.20 26 We of annotation. We resolved any remaining on amplifying polarizing and antivaccine compared these with a roughly equal number disagreements by a fourth annotator (D. A. messages. This analysis is supplemented of randomly selected tweets that were posted B.). We compared all users’ proportions of by a qualitative study of #VaccinateUS— in the same time frame. We calculated the polarized (nonneutral) tweets to users with a Twitter hashtag designed to promote dis- relative frequency with which each type of bot scores below 20% (likely humans). We cord using vaccination as a political wedge account tweeted about vaccines by counting also tested the hypothesis that nonneutral issue. #VaccinateUS tweets were uniquely the total number of tweets containing at least content posted by bots and trolls was more identified with Russian troll accounts linked 1 word beginning with “vax” or “vacc.” likely to be antivaccine by comparing the to the Internet Research Agency—a com- In our second analysis, we collected a relative proportions of polarized tweets that pany backed by the Russian government random subset of tweets from all users in the were antivaccine across all user types. We specializing in online influence operations.20 vaccine stream containing the strings “vax” or assessed all hypothesis tests for statistical sig- Thus, health communications have become “vacc” and tagged them as relevant to vac- nificance using the distribution-free c2 “weaponized”: public health issues, such as cines by a machine-learning classifier de- goodness of fittest. vaccination, are included in attempts to veloped for that purpose by Dredze et al.27 Thematic analysis of tweets by Russian trolls. spread misinformation and disinformation We used the Botometer28 API—a widely During annotation, an unfamiliar hashtag, by foreign powers. In addition, Twitter used29 bot-detection tool—to estimate each #VaccinateUS, appeared 25 times in tweets bots distributing malware and commercial tweet’s “bot score,” reflecting the likelihood posted by known Russian troll accounts content (i.e., spam) masquerade as human that its author is a bot. Botometer returns identified by NBC News and documenting users to distribute antivaccine messages. A a likelihood score between 0% and 100% for Russian interference in the US political sys- full 93% of tweets about vaccines are gen- each query and cannot make an accurate tem.20 Searching Twitter on February 20, erated by accounts whose provenance can assessment for all accounts. Thus, we seg- 2018, we found only 5 messages including this be verified as neither bots nor human users mented accounts into 3 categories: those hashtag, suggesting that #VaccinateUS had yet who exhibit malicious behaviors. These with scores less than 20% (very likely to be been primarily used by suspended accounts unidentified accounts preferentially tweet humans), greater than 80% (very likely to and that most instances had been purged. antivaccine misinformation. We discuss be bots), and between 20% and 80% (of Turning to data stored in the vaccine stream, implications for online public health uncertain provenance). Finally, we applied we identified 253 messages with #Vacci- communications. the same criteria to a subset of tweets from the nateUS. We conducted an exploratory the- vaccine stream for each of the 7 types of matic analysis of these messages to identify and known bot and troll accounts identified in the describe major themes. Our goal was to ex- first analysis. All data collection procedures are plore unifying patterns in the #VaccinateUS 30 METHODS described in detail in Appendix A, available as data and to illustrate some of the behaviors In our first analysis, we examined whether a supplement to the online version of this that known Russian trolls exhibit on Twitter. Twitter bots and trolls tweet about vaccines article at http://www.ajph.org. Consistent with this aim, we annotated more frequently than do average Twitter messages as pro- or antivaccine. Next, 1 au- users. In a second analysis, we examined the thor (A. M. J.) categorized messages into pro- Analysis and antivaccine themes using a combination relative rates with which each type of account 31 tweeted provaccine, antivaccine, and neutral Are bots and trolls more likely to tweet about of inductive and deductive codes. We de- vaccines? We tested the hypothesis that bot termined these categories loosely from messages. Finally, in a third analysis, we 12 identified a hashtag uniquely used by Russian and troll accounts generated proportionally existing research, and we incorporated trolls and used qualitative methods to describe more tweets about vaccines. We derived emergent themes in the data into them. We its content. estimates of vaccine tweet frequencies for compared these tweets with the randomly each account type from the vaccine stream, selected vaccine-relevant tweets we used in and we derived base rate estimates for average the second analysis, which we considered Data Collection Twitter users from the 1% sample (Table B, representative of the general vaccine We drew all tweets in our first analysis available as a supplement to the online version discourse. from 1 of 2 data sets derived from the Twitter of this article at http://www.ajph.org). streaming application programming interface Are bots and trolls more likely to tweet po- (API): (1) a random sample of 1% of all tweets larizing and antivaccine content? We next (“the 1% sample”), and (2) a sample of tweets tested the hypothesis that bots and trolls RESULTS containing vaccine-related keywords (“the produced higher proportions of polarizing ma- Raw counts of tweets by source are shown vaccine stream”; Table A, available as terial. Three of the authors (A. M. J., S. Q., and in Table C (available as a supplement to the

October 2018, Vol 108, No. 10 AJPH Broniatowski et al. Peer Reviewed Research 1379 AJPH OPEN-THEMED RESEARCH

online version of this article at http://www. agreement on the first round of annotation of unique users—a possible indicator of bot- or ajph.org). Figure 1 shows that accounts these tweets (Fleiss k = 0.48). In addition, we troll-like behavior—to assess whether ac- 20 identified by NBC News as Russian trolls collected 9895 tweets from the vaccine counts with higher bot scores exhibited such 25 or by Varol et al. as sophisticated bots or stream, representing the activity of assorted behavior. Figure 2 shows that accounts with 21 content polluters (i.e., accounts that dis- Twitter users, also with moderate initial intermediate bot scores posted more tweets seminate malware and unsolicited content) agreement between annotators (Fleiss per account overall. Similarly, intermediate- are significantly more likely to tweet about k = 0.49). In all cases, annotators reached scored accounts posted significantly more vaccination than are average Twitter users. consensus after 2 more rounds. We seg- polarized and neutral tweets per account; Additionally, accounts the US Congress mented these tweets into 3 subsets: 450 (5%) however, their rates of provaccine tweets did identifies as Russian trolls26 were signifi- tweets possessing Botometer scores of 20% or not differ significantly from nonbots’ after cantly more likely to tweet about vaccine- lower, 290 (3%) tweets possessing scores of preventable illnesses (e.g., Zika) but not controlling for multiple comparisons. By 80% or higher, and 7518 (76%) tweets pos- necessarily about vaccines. Finally, traditional contrast, accounts with high bot scores posted sessing intermediate scores. A total of 1587 spambots23,24 (designed to be recognizable as more neutral, but not polarized, tweets per (16%) tweets were associated with users bots) and content polluters were significantly account. less likely to tweet about vaccine-preventable whose scores could not be determined (e.g., Table 1 shows the proportions of polarized illnesses than was the average Twitter user. because their accounts had been deleted). and antivaccine messages by user type. Results One strategy used by bots and trolls is to show that accounts with intermediate bot generate several tweets about the same topics, scores post content that is significantly more 17 Antivaccine Content with the intention of flooding the discourse. polarized than are nonbots’ posts. These ac- We collected 899 tweets from the vaccine Thus, to better understand the behavior of counts, and those whose bot scores could not stream, representing the activity of known each type of account, we examined the total be determined, posted antivaccine content at bots and trolls. Annotators achieved moderate proportion of tweets that were generated by a significantly higher rate than did nonbots.

100

Contains vaccine keyword Contains vaccine keyword and word starting with "vax" or "vacc"

***

*** 10

***

***

*** ** Likelihood

1 *

*** 0.1 Social spambots Traditional spambots Congressional list Content polluters Sophisticated bots NBC russian trolls russian trolls Type of Bot

Note. NBC = National Broadcasting Network. All results remained significant after controlling for multiple comparisons using the Holm–Bonferroni procedure. Raw counts are given in Table B (available as a supplement to the online version of this article at http://www.ajph.org). *P < .05; **P < .01; ***P < .001.

FIGURE 1—Bots’ Likelihood of Tweeting About Vaccines Compared With Average Twitter Users: July 14, 2014–September 26, 2017

1380 Research Peer Reviewed Broniatowski et al. AJPH October 2018, Vol 108, No. 10 AJPH OPEN-THEMED RESEARCH

1.35 Bot score < 20% Intermediate bot score Bot score > 80% Unknown bot score ** 1.3

1.25

*** *** *** *** 1.2

1.15 *** *** **

1.1 *** **a No. Tweets per Unique Account Tweets No. 1.05

1 All Tweets All Polarized Antivaccine Provaccine Neutral Type of Tweet aNot significant after controlling for multiple comparisons using the Holm–Bonferroni procedure. Raw counts are given in Table D (available as a supplement to the online version of this article at http://www.ajph.org). **P < .01; ***P < .001.

FIGURE 2—Number of Tweets per Unique Account, Separated by Sentiment and Bot Score Category: July 14, 2014–September 26, 2017

By contrast, known bots and trolls posted messages contain fewer spelling and punc- almost singularly focused on the US govern- messages that were no more polarized than tuation errors than do comparable ment (e.g., “At first our government creates the messages of average nonbot users. Con- tweets from the general vaccine stream. The diseases then it creates #vaccines.what’snext?! tent polluters—malicious accounts identified #VaccinateUS messages are also distinctive in #VaccinateUS”). In general, users of #Vac- as promoting commercial content and mal- that they contain no links to outside content, cinateUS talk in generalities and fail to provide ware—posted significantly more antivaccine rare @mentions of other users, and no images the level of detail commensurate with what is content. Troll accounts and sophisticated bots (but occasionally use some emojis). found in other vaccine-relevant tweets. For posted roughly equal amounts of pro- and Thematically, the messages with #Vacci- example, the author might summarize an ar- antivaccine content. nateUS mirror the general vaccine discourse on gument (e.g., “#VaccinateUS #vaccines cause Twitter (the box on page 1383). Although the serious and sometimes fatal side effects”), Qualitative Analysis of authors of these tweets have a fairly compre- whereas tweets from the vaccine stream would #VaccinateUS hensive understanding of the content of both typically use as many specifics as possible to Of the 253 messages containing #Vacci- pro- and antivaccine arguments, small differ- sound convincing. nateUS, 43% were provaccine, 38% were ences set the messages apart. The authors of #VaccinateUS messages included several antivaccine, and the remaining 19% were #VacccinateUS messages tend to tie both pro- distinctive arguments that we did not observe in neutral. Whereas most nonneutral vaccine- and antivaccine messages explicitly to US the general vaccine discourse. These included relevant hashtags were clearly identifiable as politics and frequently use emotional appeals to arguments related to racial/ethnic divisions, either provaccine (#vaccineswork, #vax- “freedom,”“democracy,” and “constitutional appeals to God, and arguments on the basis of withme) or antivaccine (#, #b1less, rights.” By contrast, other tweets from the animal welfare. These are divisive topics in US #CDCWhistleblower), with limited appro- vaccine stream focus more on “parental choice” culture,whichwedidnotseefrequentlydis- priation by the opposing side, #VaccinateUS and specific vaccine-related legislation. cussed in other tweets related to vaccines. For is unique in that it appears with very polarized Like other antivaccine tweets, antivaccine instance, “Apparently only the elite get ‘clean’ messages on both sides, without other non- messages with #VaccinateUS often reference #vaccines. And what do we, normal ppl, get?! neutral hashtags. conspiracy theories. However, whereas #VaccinateUS” appears to target socioeco- Messages containing #VaccinateUS con- conspiracy theories tend to target a variety of nomic tensions that exist in the United States. tain a combination of grammatical errors, culprits (e.g., specific government agencies, By contrast, standard antivaccine messages tend unnatural word choices, and irregular individual philanthropists, or secret organi- to characterize vaccines as risky for all people phrasing. However, the #VaccinateUS zations), the #VaccinateUS messages are regardless of socioeconomic status.

October 2018, Vol 108, No. 10 AJPH Broniatowski et al. Peer Reviewed Research 1381 AJPH OPEN-THEMED RESEARCH

fi TABLE 1—Proportions of Polarized and communications about vaccination. The Unidenti ed Accounts Antivaccine Tweets by User Type: July 14, nature of this impact differs by account type. Several accounts could not be positively 2014–September 26, 2017 identified as either bots or humans because Russian Trolls of intermediate or unavailable Botometer Polarized, Antivaccine, scores. These accounts, together constituting User Type % % Russian trolls and sophisticated Twitter bots post content about vaccination at sig- 93% of our random sample from the vaccine Assorted users, bot score, % nificantly higher rates than does the average stream, tweeted content that was both more 20 31 35 < user. Content from these sources gives equal polarized and more opposed to vaccination 20–80 39*** 60*** attention to pro- and antivaccination argu- than is that of the average nonbot account. 80 26 49*,a > ments. This is consistent with a strategy of Although the provenance of their tweets is Unknown 37*,a 62*** promoting discord across a range of contro- unclear, we speculate that these accounts may Known bots and trolls versial topics—a known tactic employed by possess a higher proportion of trolls or cyborgs— 20 ,a NBC Russian trolls 20* 47 Russian troll accounts.20,26 Such strategies accounts nominally controlled by human 21 Content polluters 38 60*** may undermine the public health: normal- users that are, on occasion, taken over by bots 22 Fake followers 0NA izing these debates may lead the public to or otherwise exhibit bot-like or malicious 23,24 15 Traditional spambots 3*** 0 question long-standing scientific consensus behavior. Cyborg accounts are more likely 23,24 ,a Social spambots 18** 56* regarding vaccine efficacy.13 Indeed, several to fall into this middle range because they only 25 Sophisticated bots 28 44 antivaccine arguments claim to represent both display bot-like behaviors sometimes. This Congressional list Russian 39 48 sides of the debate—like the tactics used by middle range is also likely to contain tweets 26 trolls the trolls identified in this study—while si- from more sophisticated bots that are Note. NA = not applicable because of insufficient multaneously communicating a clear gist designed to more closely mimic human data; NBC = National Broadcasting Network. A (i.e., a bottom-line meaning). We recently behaviors. statistically significant result indicates that found that this strategy was effective for Finally, trolls—exhibiting malicious be- a certain type of account posts polarized or — antivaccine tweets at a rate that differs signifi- propagating news articles through social haviors yet operated by humans are also cantly from that of accounts with bot scores media in the context of the 2015 Disneyland likely to fall within this middle range. This < 20% (likely humans). Polarized proportion is measles outbreak.32 suggests that proportionally more antivaccine the ratio of all nonneutral tweets to all tweets. Antivaccine proportion is the ratio of antivaccine tweets may be generated by accounts using tweets to polarized tweets. Raw counts are Commercial and Malware a somewhat sophisticated semiautomated shown in Table E (available as a supplement to approach to avoid detection. This creates the the online version of this article at http://www. Distributors false impression of grassroots debate regarding ajph.org). Unlike troll accounts, content polluters a fi fi — No longer signi cant after controlling for (i.e., disseminators of malware, unsolicited vaccine ef cacy a technique known as multiple comparisons using the Holm–Bonfer- “astroturfing”17 (as in the #VaccineUS tweets roni procedure. commercial content, and other disruptive shown in the box on page 1383). There are *P < .05; **P < .01; ***P < .001. material that typically violates Twitter’s terms of service)21 post antivaccine messages 75% certainly standard human accounts that also more often than does the average nonbot fall within this middle range. Although #VaccinateUS messages also include sev- Twitter user. This suggests that vaccine op- technological limitations preclude us from fi eral messages that seem designed to provoke ponents may disseminate messages using bot drawing de nitive conclusions about these a response and prolong an argument, in- networks that are primarily designed for account types, the fact that middle-range cluding open-ended items and comments on marketing. By contrast, spambots,3,4 which tweets tend to post proportionately more the debate itself (e.g., “I believe in #vaccines, can be easily recognized as nonhuman, are less antivaccine messages suggests strongly that why don’t you? #VaccinateUS”). Comments likely to promote an antivaccine agenda than these antivaccine messages may be dissemi- were also used to bait other users into are nonbots. Notably, content polluters and nated at higher rates by a combination of responding, specifically by posting content traditional spambots are both less likely to malicious actors (bots, trolls, cyborgs, and that advocates of vaccination would take discuss vaccine-preventable illnesses than is human users) who are difficult to distinguish for granted, such as “#VaccinateUS Major the average Twitter user, suggesting that from one another. medical organizations state that #vaccines are when they do tweet vaccine-relevant mes- This interpretation is supported by the fact safe” and “#vaccine injuries are rare, despite sages, their specific focus is on vaccines per se, that users within this intermediate range parental worrying #VaccinateUS.” rather than the viruses that require them. tended to produce more tweets, and espe- Thus, it is unclear to what extent their pro- cially antivaccine tweets, per account, sug- motion of vaccine-related content is driven gesting that antivaccine activists may by true antivaccine sentiment or is used as preferentially use these channels. In addition, DISCUSSION a tactic designed to drive up click-through users whose accounts had been deleted posted Results suggest that Twitter bots and rates by propagating motivational content more polarized messages per user and were trolls have a significant impact on online (“clickbait”). also significantly more likely to post

1382 Research Peer Reviewed Broniatowski et al. AJPH October 2018, Vol 108, No. 10 AJPH OPEN-THEMED RESEARCH

disseminate content such as news and may not EXAMPLES OF TWEETS WITH #VACCINATEUS AND CORRESPONDING be considered credible sources of grassroots THEMES: JULY 14, 2014–SEPTEMBER 26, 2017 antivaccine information. Antivaccine theme Example tweet Freedom of choice/antimandatory vaccines VaccinateUS mandatory #vaccines infringe on constitutionally Public Health Implications protected religious freedoms Survey data show a general consensus Can’t trust government on vaccines Did you know there was a secret government database of regarding the efficacy of vaccines in the #vaccine-damaged children? #VaccinateUS general population.35 Consistent with these Pharmaceutical companies want vaccine Pharmacy companies want to develop #vaccines to cash, not to results, accounts unlikely to be bots are sig- profits prevent deaths #VaccinateUS nificantly less likely to promote polarized and Vaccines cause bad side effects #VaccinateUS #vaccines can cause serious and sometimes antivaccine content. Nevertheless, bots and fatal side effects trolls are actively involved in the online public Natural immunity is better #VaccinateUS natural infection almost always causes better health discourse, skewing discussions about immunity than #vaccines vaccination. This is vital knowledge for risk General vaccine conspiracy theories Dont get #vaccines. Iluminati are behind it. #VaccinateUS communicators, especially considering that neither members of the public nor algorith- Vaccines cause autism Did you know #vaccines caused autism? #VaccinateUS mic approaches may be able to easily identify Vaccine ingredients are dangerous #VaccinateUS #vaccines contain mercury! Deadly poison! bots, trolls, or cyborgs. ’ Diseases aren t so dangerous #VaccinateUS most diseases that #vaccines target are Malicious online behavior varies by ac- relatively harmless in many cases, thus making #vaccines count type. Russian trolls and sophisticated unnecessary bots promote both pro- and antivaccination Provaccine theme Example tweet narratives. This behavior is consistent with Vaccines work #VaccinateUS #vaccines save 2.5 million children from a strategy of promoting political discord. Bots preventable diseases every year and trolls frequently retweet or modify Vaccines should be mandatory Your kids are not your property! You have to #vaccinate them content from human users. Thus, well- to protect them and all the others! #VaccinateUS intentioned posts containing provaccine People who don’t vaccinate are stupid #VaccinateUS You can’t fix stupidity. Let them die from content may have the unintended effect of measles, and I’m for #vaccination! “feeding the trolls,” giving the false impres- Vaccination protects herd immunity #VaccinateUS #vaccines protect community immunity sion of legitimacy to both sides, especially if this content directly engages with the anti- People who don’t vaccinate put me/my #VaccinateUS My freedom ends where another person’s vaccination discourse. Presuming bot and kids at risk begins. Then children should be #vaccinated if disease is troll accounts seek to generate roughly equal dangerous for OTHER children numbers of tweets for both sides, limiting ’ — Vaccines don t cause autism #vaccines cause autism Bye, you are not my friend anymore. access to provaccine content could potentially And try to think with your brain next #VaccinateUS also reduce the incentive to post antivaccine You deserve bad things if you don’t #vaccines are a parent’s choice. Choice of a color of a little content. vaccinate coffin #VaccinateUS By contrast, accounts that are known to Alternative medicine doesn’t work Do you still treat your kids with leaves? No? And why don’t you distribute malware and commercial content #vaccinate them? Its medicine! #VaccinateUS are more likely to promote antivaccination People died without vaccines Most parents in Victorian times lost children regularly to messages, suggesting that antivaccine advo- preventable illnesses. #vaccines can solve this problem cates may use preexisting infrastructures of bot #VaccinateUS networks to promote their agenda. These accounts may also use the compelling nature of antivaccine content as clickbait to drive up advertising revenue and expose users to antivaccine messages. Although reasons for scores are malicious actors; however, we malware. When faced with such content, account deletion vary, recent movement by expect that a higher proportion of malicious public health communications officials may 33,34 20 Twitter to remove bots, trolls, cyborgs, actors are present in this subset of the data. By consider emphasizing that the credibility of and other violators of Twitter’s terms of contrast, randomly sampled accounts that the source is dubious and that users exposed to service suggests that these violators may be were easily identifiable as bots generated more such content may be more likely to encounter overrepresented among the deleted accounts neutral, but not polarized tweets per account. malware. Antivaccine content may increase in our sample. We cannot claim that all, or Presumably, accounts that are obviously au- the risks of infection by both computer and even most, accounts with unknown bot tomated are more frequently used to biological viruses.

October 2018, Vol 108, No. 10 AJPH Broniatowski et al. Peer Reviewed Research 1383 AJPH OPEN-THEMED RESEARCH

The highest proportion of antivaccine 6. Betsch C, Brewer NT, Brocard P, et al. Opportunities tools for the arms race. In: Barrett R; International World content is generated by accounts with un- and challenges of Web 2.0 for vaccination decisions. Wide Web Conferences Steering Committee, eds. Pro- Vaccine. 2012;30(25):3727–3733. ceedings of the 26th International Conference on World Wide known or intermediate bot scores. Although Web Companion.Republic and Canton of Geneva: ACM 7. Tomeny TS, Vargo CJ, El-Toukhy S. Geographic and – we speculate that this set of accounts contains demographic correlates of autism-related anti-vaccine Press; 2017:963 972. more sophisticated bots, trolls, and cyborgs, beliefs on Twitter, 2009–15. Soc Sci Med. 2017;191: 25. Varol O, Ferrara E, Davis CA, Menczer F, Flammini – – their provenance is ultimately unknown. 168 175. A. Online human bot interactions: detection, estimation, and characterization. Available at: https://aaai.org/ocs/ Therefore, beyond attempting to prevent 8. Smith MJ, Marshall GS. Navigating parental vaccine hesitancy. Pediatr Ann. 2010;39(8):476–482. index.php/ICWSM/ICWSM17/paper/view/15587. bots from spreading messages over social Accessed March 11, 2018. 9. Dubé E, Vivion M, MacDonald NE. Vaccine hesi- 26. Frommer D. Twitter’s list of 2,752 Russian trolls. media, public health practitioners should tancy, vaccine refusal and the anti-vaccine movement: focus on combating the messages them- influence, impact and implications. Expert Rev Vaccines. 2017. Available at: https://www.recode.net/2017/11/2/ 16598312/russia-twitter-trump-twitter-deactivated- 2015;14(1):99–117. selves while not feeding the trolls. This is handle-list. Accessed March 11, 2018. 10. Jolley D, Douglas KM. The effects of anti-vaccine a ripe area for future research, which might 27. Dredze M, Broniatowski DA, Smith MC, Hilyard conspiracy theories on vaccination intentions. PLoS One. fi KM. Understanding vaccine refusal: why we need social include emphasizing that a signi cant pro- 2014;9(2):e89177. portion of antivaccination messages are media now. Am J Prev Med. 2016;50(4):550–552. 11. Jones AM, Omer SB, Bednarczyk RA, Halsey NA, 28. Davis CA, Varol O, Ferrara E, Flammini A, Menczer organized “astroturf” (i.e., not grassroots) Moulton LH, Salmon DA. Parents’ source of vaccine F. BotOrNot: A System to evaluate social bots. Available information and impact on vaccine attitudes, beliefs, and and other bottom-line messages that put at: http://dl.acm.org/citation.cfm?doid=2872518. nonmedical exemptions. Adv Prev Med. 2012;2012: antivaccine messages in their proper 2889302. Accessed July 25, 2018. 932741. contexts. 29. Wojcik S, Messing S, Smith A, Rainie L, Hitlin P. Bots 12. Kata A. A postmodern Pandora’s box: anti- in the Twittersphere. 2018. Available at: http://www. vaccination misinformation on the internet. Vaccine. 2010; pewinternet.org/2018/04/09/bots-in-the- CONTRIBUTORS 28(7):1709–1716. twittersphere. Accessed April 26, 2018. D. A. Broniatowski designed the study, conducted the 13. Dixon G, Clarke C. The effect of falsely balanced fi 30. Sandelowski M, Barroso J. Classifying the findings in statistical analyses, and wrote the rst draft of the article. reporting of the autism-vaccine controversy on vaccine qualitative studies. Qual Health Res. 2003;13(7):905–923. A. M. Jamison conducted the qualitative analysis. A. M. safety perceptions and behavioral intentions. Health Educ Jamison, S. Qi, and L. Alkulaib conducted the sentiment Res. 2013;28(2):352–359. 31. Fereday J, Muir-Cochrane E. Demonstrating rigor coding. A. M. Jamison, S. Qi, L. Alkulaib, T. Chen, using thematic analysis: a hybrid approach of inductive 14. Centers for Disease Control and Prevention. Mor- A. Benton, S. C. Quinn, and M. Dredze critically revised and deductive coding and theme development. Int J Qual tality. 2017. Available at: https://www.cdc.gov/nchs/ the article. S. Qi, L. Alkulaib, T. Chen, and A. Benton Methods. 2006;5(1):80–92. collected and analyzed Twitter data. S. C. Quinn and health_policy/mortality.htm. Accessed April 27, 2018. 32. Broniatowski DA, Hilyard KM, Dredze M. Effective M. Dredze assisted with study design. 15. Chu Z, Gianvecchio S, Wang H, Jajodia S. Detecting vaccine communication during the Disneyland measles automation of Twitter accounts: are you a human, bot, or outbreak. Vaccine. 2016;34(28):3225–3228. ACKNOWLEDGMENTS cyborg? IEEE Trans Depend Secure Comput. 2012;9(6): – fl “ ” This research was supported by the National Institute of 811 824. 33. Flynn K. Twitter in uencers suspect a bot purge. Available at: https://mashable.com/2018/01/29/ General Medical Sciences, National Institutes of Health 16. Collins English Dictionary. Troll definition and twitter-bots-purge-influencers-accounts. Accessed April (NIH; award 5R01GM114771). meaning. Available at: https://www.collinsdictionary. 27, 2018. Note. The content is solely the responsibility of the com/dictionary/english/troll. Accessed April 27, 2018. authors and does not necessarily represent the official 34. Madrak S. Wingers melt down as twitter finally purges 17. Ferrara E, Varol O, Davis C, Menczer F, Flammini A. views of NIH. Russian bots. Available at: https://crooksandliars.com/ The rise of social bots. Commun ACM. 2016;59(7): 2018/02/wingers-have-sad-twitter-purges-russian-0. 96–104. HUMAN PARTICIPANT PROTECTION Accessed April 27, 2018. 18. Subrahmanian VS, Azaria A, Durst S, et al. The DARPA The data used in this article are from publicly available 35. Funk C, Kennedy B, Hefferon M. Vast majority of Twitter bot challenge. Computer. 2016;49(6):38–46. online sources, the uses of which the Johns Hopkins Americans say benefits of childhood vaccines outweigh Homewood institutional review board deems exempt 19. Ward JK, Peretti-Watel P, Verger P. Vaccine criticism risks. 2017. Available at: http://www.pewinternet.org/ from institutional review board approval (approval no. on the Internet: propositions for future research. Hum 2017/02/02/vast-majority-of-americans-say-benefits- 2011123). Vaccin Immunother. 2016;12(7):1924–1929. of-childhood-vaccines-outweigh-risks. Accessed Febru- 20. Popken B. Twitter deleted Russian troll tweets. So we ary 14, 2017. REFERENCES published more than 200,000 of them. Available at: 1. Kata A. Anti-vaccine activists, Web 2.0, and the https://www.nbcnews.com/tech/social-media/now- postmodern paradigm—an overview of tactics and tropes available-more-200-000-deleted-russian-troll-tweets- used online by the anti-vaccination movement. Vaccine. n844731. Accessed March 11, 2018. – 2012;30(25):3778 3789. 21. Lee K, Eoff BD, Caverlee J. Seven months with the 2. Breland JY, Quintiliani LM, Schneider KL, May CN, devils: a long-term study of content polluters on Twitter. Pagoto S. Social media as a tool to increase the impact of Available at: https://www.aaai.org/ocs/index.php/ public health research. Am J Public Health. 2017;107(12): ICWSM/ICWSM11/paper/view/2780. Accessed 1890–1891. March 11, 2018. 3. Luxton DD, June JD, Fairall JM. Social media and 22. Cresci S, Di Pietro R, Petrocchi M, Spognardi A, suicide: a public health perspective. Am J Public Health. Tesconi M. Fame for sale: efficient detection of fake 2012;102(suppl 2):S195–S200. Twitter followers. Decis Support Syst. 2015;80:56–71. 4. Witteman HO, Zikmund-Fisher BJ. The defining 23. Cresci S, Pietro RD, Petrocchi M, Spognardi A, characteristics of Web 2.0 and their potential influence in Tesconi M. Social fingerprinting: detection of spambot the online vaccination debate. Vaccine. 2012;30(25): groups through DNA-inspired behavioral modeling. 3734–3740. IEEE Trans Dependable and Secure Comput. 2018;15(4): – 5. Quinn SC. Probing beyond individual factors to un- 561 576. derstand influenza and pneumococcal vaccine uptake. Am 24. Cresci S, Di Pietro R, Petrocchi M, Spognardi A. The J Public Health. 2018;108(4):427–429. paradigm-shift of social spambots: evidence, theories, and

1384 Research Peer Reviewed Broniatowski et al. AJPH October 2018, Vol 108, No. 10 Opinion

VIEWPOINT Communicating About Vaccines in a Fact-Resistant World

Saad B. Omer, MBBS, The continued success of vaccines, one of the most ef- phenomenon, not only for parents but also for physi- MPH, PhD fective public health interventions, depends on high cians. For example, physicians who graduated from Rollins School of Public rates of acceptance. Vaccine refusal in the United States medical school between 1995 and 2002 had relatively Health, Emory has increased since the late 1990s.1 This trend has co- less favorable attitudes regarding vaccines compared University, Atlanta, 4 Georgia. incided with an increase in vaccine safety concerns. Such with those who graduated between 1954 and 1964. concerns result from easy recall of adverse events, mis- Avnika B. Amin, MSPH information, and human tendency to poorly judge prob- Countering Misinformation Rollins School of Public abilities. When a significant proportion of the US popu- and the Boomerang Effect Health, Emory lation is impervious to scientific facts, such as belief in The instinctive response to vaccine-related misinforma- University, Atlanta, Georgia. human-induced climate change, it is difficult to commu- tion is to provide correct information. However, this nicate vaccine-related information to patients. information correction–based approach has limitations Rupali J. Limaye, PhD Parent-physician communication in such condi- and can backfire. For many, processing information on Bloomberg School of tions is challenging and, if done improperly,may worsen controversial topics occurs in a way that preserves pre- Public Health, Johns Hopkins University, the problem. Although the evidence base for vaccine- existing beliefs. Individuals who receive messages op- Baltimore, Maryland. related communications is still emerging, we present de- posing their pre-existing beliefs may not just resist chal- velopments in social and behavioral communication, be- lenges to their views but support their original opinion havioral economics, social psychology, and persuasion even more.5 Coined the boomerang effect by psycholo- theory to guide productive vaccine discussions in the gists in the 1950s, several others have explored this con- clinic. cept in various disciplines.6 For example, political scientists Nyhan and Reifler5 Availability Heuristic had participants read mock news articles about weap- When faced with immediate decisions, such as vaccina- ons of mass destruction in Iraq that included either a mis- tion during a routine clinic visit, humans often lack the leading claim from a politician or a misleading claim and time or resources to examine all plausible options. Heu- a correction. Conservative participants who were pre- ristics are mental shortcuts that allow humans to arrive sented with a claim and a correction were more likely to quickly at an answer. For example, after seeing several agree that Iraq had weapons of mass destruction than news reports about violence, someone might judge those who only received misinformation.5 This back- that violence is much more common than it actually is fire effect was also observed in a trial to evaluate effec- in one’s own neighborhood and may subsequently be- tiveness of correcting misperceptions regarding the have congruently with this judgment. Although help- measles, mumps, and rubella vaccine.7 When pre- ful, heuristics can result in errors in judgment and deci- sented with information refuting claims of a link be- sion making. One heuristic particularly relevant to tween the measles, mumps, and rubella vaccine with au- vaccine-related perceptions is the availability heuristic, tism, parents who distrusted vaccines demonstrated first described by Nobel Laureate Daniel Kahneman and reduced intention to vaccinate their children despite his colleague, Amos Tversky, in their landmark 1974 greater knowledge of the lack of association between article.2 The availability heuristic describes our propen- . sity to estimate the probability of an event based on how easily an instance of that event comes to mind. For ex- Focus on the Disease ample, plane crashes attract substantial public atten- Given the possibility of backfire, one promising ap- tion and media coverage, although the odds of dying in proach is to avoid correcting misperceptions regarding a plane crash are 862 times lower than the odds of dy- vaccine adverse events and to instead pivot the conver- ing in a car crash.3 Nonetheless, more people are afraid sation to the disease itself. The extended parallel pro- of flying than driving. cessing model is a behavioral framework for situations After successful introduction of a vaccine for a spe- whenanindividualperceivesathreatofadisease.Insuch cific disease, rates of that disease start decreasing. Ow- situations, individuals will either address the issue ing to the increase in the number of vaccine doses de- head-on or become cognitively frozen and incapable of Corresponding livered, the absolute number of real (and perceived) action. To effectively stimulate action to address pos- Author: Saad B. Omer, MBBS, MPH, PhD, adverse events after vaccination starts increasing. Suc- sible disease, an individual must perceive they are at risk Rollins School of Public cessive cohorts of parents primarily hear about these ad- for disease (risk perception) and believe there is an ef- Health, Emory verse events, and the collective memory of the disease fective action (response efficacy) and that they are ca- University, 1518 Clifton declines. This change in the ease of recalling benefits vs pable of taking that action (self-efficacy). Rd NE, Atlanta, GA 30322 (somer@emory risks can result in a change of one’s appreciation of and Primary literature focused on influencing per- .edu). confidence in vaccines. Empirical evidence supports this ceived severity and susceptibility is limited, to our knowl-

jamapediatrics.com (Reprinted) JAMA Pediatrics October 2017 Volume 171, Number 10 929

© 2017 American Medical Association. All rights reserved.

Downloaded From: by a Yale University User on 04/20/2018 Opinion Viewpoint

edge. However, there are some promising leads. Sadique et al8 pre- false to signal the mind to be alert. Third and most importantly, an sented mothers with general information about the risks and severity alternative explanation should be provided. Attempts to correct the of a hypothetical disease and of adverse effects from a vaccine. Sus- myth without filling in mental gaps are likely to be unsuccessful. An ceptibility was manipulated by presenting different probabilities of explanation about why the myth is wrong and/or why some people the infection or the adverse event. Severity was influenced by vary- promote the myth can be used to fill this gap. ing the type of symptoms (irritability, fever, and bowel obstruc- tion) and the duration of the symptoms such that the descriptions Leverage the Power of Defaults clearly differed in seriousness. They then asked the mothers to de- A binary taxonomy of vaccine acceptors or refusers is somewhat sim- cide whether to vaccinate their child against this hypothetical dis- plistic. Many parents are “fencesitters” and are uncertain about the ease. Mothers with high knowledge of real disease prior to the study benefit of vaccination outweighing potential adverse events. For were more willing to vaccinate their child compared with those these parents, nudges and interventions that leverage defaults with low knowledge, even with the risk of an adverse event from (ie, whatever outcome happens if no action is taken) can be helpful vaccination.8 What ultimately predicted the likelihood of vaccinat- in bridging the intention-to-action gap. ing was not the probability of getting sick or experiencing an ad- Such interventions may include presumptive communication, verse event, but the perceived seriousness of the disease or the which shapes discussion with the presumption that the parent adverse event.8 These perceptions of severity were influenced by will vaccinate their child. Participatory communication is shaped by the researchers by presenting different symptoms associated with asking whether the parent would like their child to be vaccinated the disease or the adverse event. As described previously, the ease (eg, presumptive: “It’s time for little Johnny to get vaccinated!” vs of recall of relevant information influences perceptions around participatory: “Should little Johnny get vaccinated at this visit?”). susceptibility and severity; therefore, clinicians can use recent news More parents voice resistance to vaccines when a physician uses a stories of outbreaks to increase disease salience. participatory approach to vaccinations, while fewer parents will Although it is preferable to focus on the disease rather than the resist when a presumptive approach is used.10 vaccine, directly addressing vaccine-related myths relies on 3 principles.9 First, confrontation of the myth should be focused on Conclusions key facts, instead of every supportive fact. Too much information The aforementioned list of strategies includes only a few promising may inadvertently reinforce the myth, whereas straightforward evidence-based approaches rather than an exhaustive list of all pos- factswilldecreasemisperceptions.9 Keymessagescancenteraround sible interventions. However, in an environment where fact-based facts such as “no recommended childhood vaccines contain thi- interventions have limitations, there is a need for moving from wis- merosal” to convey a simple, clear message.9 Second, before the dom-based to evidence-based tailored approaches to increase myth is mentioned, clearly indicate that subsequent information is vaccine acceptance.

ARTICLE INFORMATION 2. Tversky A, Kahneman D. Judgment under 7. Nyhan B, Reifler J, Richey S, Freed GL. Effective Published Online: August 14, 2017. uncertainty: heuristics and biases. Science. 1974;185 messages in vaccine promotion: a randomized trial. doi:10.1001/jamapediatrics.2017.2219 (4157):1124-1131. Pediatrics. 2014;133(4):e835-e842. Correction: This article was corrected on 3. Injury Facts 2016. Itasca, IL: National Safety 8. Sadique MZ, Devlin N, Edmunds WJ, Parkin D. October 2, 2017, to fix an incorrect degree Council; 2016. The effect of perceived risks on the demand for in the author byline. 4. Mergler MJ, Omer SB, Pan WKY, et al. Are recent vaccination: results from a discrete choice experiment. PLoS One. 2013;8(2):e54149. Conflict of Interest Disclosures: None reported. medical graduates more skeptical of vaccines? Vaccines (Basel). 2013;1(2):154-166. 9. Cook J, Lewandowsky S. The Debunking REFERENCES 5. Nyhan B, Reifler J. When corrections fail: the Handbook. St Lucia, Queensland, Australia: University of Queensland; 2011. 1. Omer SB, Salmon DA, Orenstein WA, deHart MP, persistence of political misperceptions. Polit Behav. Halsey N. Vaccine refusal, mandatory 2010;32(2):303-330. 10. Opel DJ, Heritage J, Taylor JA, et al. immunization, and the risks of vaccine-preventable 6. Hovland CI, Janis IL, Kelley HH. Communication The architecture of provider-parent vaccine diseases. N Engl J Med. 2009;360(19):1981-1988. and Persuasion. New Haven, CT: Yale University Press; discussions at health supervision visits. Pediatrics. 1953. 2013;132(6):1037-1046.

930 JAMA Pediatrics October 2017 Volume 171, Number 10 (Reprinted) jamapediatrics.com

© 2017 American Medical Association. All rights reserved.

Downloaded From: by a Yale University User on 04/20/2018