An Assessment of Real-World Data, Artificial Intelligence, and Their Use in Drug Development Kenneth Hu, Pharm.D., Patrick LaFontaine, Pharm.D., Angelina Mandic, Pharm.D., Evelyn Hermes-DeSantis, Pharm.D., BCPS, Michael Toscani, Pharm.D., Joseph A. Barone, Pharm.D., FCCP Rutgers Institute for Pharmaceutical Industry Fellowships, Ernest Mario School of , Rutgers University, Piscataway, NJ

Background Results  Real-world data (RWD) and real-world Timeline of Select Communications and Pharma Investments evidence (RWE) o Historically used to monitor safety o Proposed to evaluate drug efficacy for 2016 2017 2018 approvals and label expansions. o No universally accepted definition: 8/15/16 – US FDA PDUFA VI includes 2/1/17 – London School of Economics and 2/21/18 – PMDA publishes guidance on  RWD is “data relating to patient plans to enhance the use of RWE for use Political Science publication reliability of datasets status and/or the delivery of health care in regulatory decision-making 2/22/17 – European Agency 5/4/18 – Heads of Medicines Agencies routinely collected from a variety of 12/13/16 – US Congress 21st Century (EMA) publishes report on big data in (HMA)/EMA Joint Big Data Task Force sources” per US Food and Drug Cures Act directs FDA to evaluate the drug development holds international workshop on big data Administration (FDA)1,2 potential uses of RWE 6/9/17 – Pharmaceuticals and Medical and related challenges  RWE is the “clinical evidence” “derived Devices Agency (PMDA) guidance on 8/18/18 – US FDA publishes guidance on using 11/8/16 – BenevolentAI and Janssen from analysis of RWD” per US FDA1,2 using health information databases data Pharmaceutica NV collaborate to use  Artificial intelligence (AI) 9/13/17 – The Duke Margolis Center for 12/6/18 – US FDA outlines plans on using deep learning to study clinical-stage o Use of computers to imitate human Health Policy publishes a framework for RWE to support approvals and surveillance drug candidates thinking3 using RWE in regulatory decisions 12/19/18 – EMA publishes draft strategy to o Generally used to identify patterns in promote use of RWD in regulatory decision- Select Regulator and Academic 7/2/17 – GlaxoSmithKline and Exscientia datasets to find new targets and making Communications enter collaboration on small molecule compounds to develop as therapies3 12/20/18 – US FDA publishes draft guidance Select Pharma Investments research using AI on using patient experience data Objective Red text for United States Initiatives 8/13/17 – GNS Healthcare receives funds Orange text for European Initiatives from Amgen Ventures to develop causal 4/6/18 – Roche acquires Flatiron Health to  To assess the opportunities, challenges, and Green text for Asian Initiatives machine learning and simulation develop RWE for cancer research using their current applications of RWD and AI in the Bold text for Industry Initiatives electronic health record software approval and post-marketing process 10/25/18 – Chinese AI XtalPi receives funds for Methods Opportunities Challenges ID4 (Intelligent Digital Drug Discovery and  A literature review was conducted from • Better screening for successful • No standardization of current data Development) platform September through December 2018. drug candidates3 systems8  Media releases from Pink Sheet, Prevision • Provide product effectiveness • Data collection and analytical methods not Conclusions Policy, FirstWord Pharma, Endpoints News, RWD data for label changes1,2 validated to prevent bias and variability8  There is an increasing investment from FiercePharma, and STAT News were tracked • Faster and cheaper market access • Limited guidance on data security and international regulators to provide guidance for communications and investments. • Better-informed regulatory privacy5 on the use of RWD to assess drug product decision making1,2,4 • Proprietary nature of large datasets9 performance, with PMDA focusing on safety • Predict molecules’ behavior and • No standardized systems or definitions3 surveillance and US FDA shifting towards screen for drug targets5,6 • Patient reluctance to try new technology3 product effectiveness with ongoing • Discover unseen patterns in high- • Data is currently dispersed and demonstration projects. AI volume datasets7 unorganized5  AI technologies are being heavily invested in • Capability to shorten drug design • High standards in clinical care3 yet no clear framework or guidance has been timeline provided to regulate these technologies to ensure validity and security. Author Contact Information Please see Supplementary Results and Kenneth Hu, Pharm.D. Patrick LaFontaine, Pharm.D. Angelina Mandic, Pharm.D. Selected Bibliography below. [email protected] [email protected] [email protected] An Assessment of Real-World Data, Artificial Intelligence, and Their Use in Drug Development Kenneth Hu, Pharm.D., Patrick LaFontaine, Pharm.D., Angelina Mandic, Pharm.D., Evelyn Hermes-DeSantis, Pharm.D., BCPS, Michael Toscani, Pharm.D., Joseph A. Barone, Pharm.D., FCCP Rutgers Institute for Pharmaceutical Industry Fellowships, Ernest Mario School of Pharmacy, Rutgers University, Piscataway, NJ

Supplementary Results Timeline of Select Communications and Pharma Investments 2016 2017 2018

2/3/16 – US Food and Drug Administration (FDA) 2/1/17 – London School of Economics and Political Science 2/21/18 – PMDA publishes “Points to Consider for Ensuring Data announces plan to adapt Sentinel infrastructure to publishes “RWE in Europe – An Analysis” in the Reliability on Post-Marketing Database Study for Drugs” guidance develop the “Guardian” system to gather information Journal of Health Policy & Outcomes Research 4/1/18 – PMDA launches MID-NET® (Medical Information Database about the performance of regulated medical products 2/22/17 – European Medicines Agency (EMA) publishes Network) combining data from 23 hospitals to benefit regulatory from pre-existing electronic healthcare data report “Identifying opportunities for ‘big data’ in decision-making 8/15/16 – US FDA publishes PDUFA VI for fiscal years medicines development and regulatory science” 5/4/18 – Heads of Medicines Agencies (HMA)/EMA Joint Big Data Task 2018-2022 with plans to enhance the use of RWE for 6/9/17 – Japanese Pharmaceuticals and Medical Devices Force holds international workshop to inform thinking on big data use in regulatory decision-making. Agency (PMDA) publishes “Basic Principles on the and related challenges with plans to make recommendations in the 12/8/16 – US FDA representatives publish “Real-World Utilization of Health Information Databases for Post- field Evidence — What Is It and What Can It Tell Us” in Marketing Surveillance of Medical Products” guidance 5/24/18 – Japanese university and federal researchers publish The New England Journal of 8/22/17 – The Korea Pharmaceutical and Bio-Pharma “Regulatory Science on AI-based Medical Devices and Systems” with 12/13/16 – US Congress signs into law the 21st Century Manufacturers Association initiates “AI-backed drug the PMDA Cures Act directing FDA to establish a program to development” taskforce 8/18/18 – US FDA publishes “Use of Electronic Health Record Data in evaluate the potential use of RWE 9/13/17 – The Duke Margolis Center for Health Policy Clinical Investigations Guidance for Industry” publishes “A Framework for Regulatory Use of Real- 9/4/18 – US FDA representatives publish “Real-World Evidence and 11/8/16 – BenevolentAI enters exclusive collaboration World Evidence” in the US Real-World Data for Evaluating Drug Safety and Effectiveness” in with Janssen Pharmaceutica NV to develop clinical- 9/14/17 – ISPOR and ISPE publish “Good Practices for the Journal of the American Medical Association stage drug candidates using deep learning to mine and Real‐World Data Studies of Treatment and/or 11/6/18 – US FDA launches MyStudies app to facilitate the collection of analyze biomedical information Comparative Effectiveness” RWE via patients’ mobile devices 10/20/17 – PMDA implements Conditional Early Approval 11/30/18 – EMA publishes “Guideline on clinical investigation of System for drugs with requirement for reconfirmation of recombinant and human plasma-derived factor IX products” Select Regulator and Academic Communications post-marketing efficacy and safety guidance allowing certain data to be collected from patient registries 12/6/18 – US FDA publishes “Framework for FDA’s Real-World Select Pharma Investments 7/2/17 – GlaxoSmithKline and Exscientia enter Evidence Program” outlining how the Agency plans to evaluate the collaboration for potentially £33 million to identify Red text for United States Initiatives use of RWE small molecules for up to 10 -related targets Orange text for European Initiatives 12/19/18 – EMA publishes draft “Regulatory Science to 2025” strategy using AI Green text for Asian Initiatives to support use of high-quality RWD in decision making 8/13/17 – GNS Healthcare announces that it has raised Bold text for Industry Initiatives 12/20/18 – US FDA publishes “Developing and Submitting Proposed US$6 million in equity from Amgen Ventures to develop Draft Guidance Relating to Patient Experience Data Guidance for their REFS™ causal machine learning and simulation Industry and Other Stakeholders” platform to examine datasets and identify causal mechanisms between variables 4/6/18 – Roche acquires Flatiron Health for US$1.9 billion to develop RWE for cancer research using their oncology electronic health record software 10/25/18 – Chinese AI biotech start-up XtalPi raises US$46 million for its ID4 (Intelligent Digital Drug Discovery and Development) platform An Assessment of Real-World Data, Artificial Intelligence, and Their Use in Drug Development Kenneth Hu, Pharm.D., Patrick LaFontaine, Pharm.D., Angelina Mandic, Pharm.D., Evelyn Hermes-DeSantis, Pharm.D., BCPS, Michael Toscani, Pharm.D., Joseph A. Barone, Pharm.D., FCCP Rutgers Institute for Pharmaceutical Industry Fellowships, Ernest Mario School of Pharmacy, Rutgers University, Piscataway, NJ

Selected Bibliography 1. U.S. Food & Drug Administration. “Framework for FDA’s Real-World Evidence Program”. December 6, 2018. 2. Duke Margolis Center for Health Policy. “A Framework for Regulatory Use of Real-World Evidence”. September 13, 2017. 3. Maddox TM, Rumsfeld JS, Payne PRO. The Journal of the American Medical Association. “Questions for Artificial Intelligence in Health Care”. December 10, 2018. 4. European Medicines Agency. “EMA Regulatory Science to 2025 Strategic reflection”. December 19, 2018. 5. Basak S, Khurana S. “Artificial Intelligence for modern drug development”. May 13, 2018. 6. European Medicines Agency. “Identifying opportunities for ‘big data’ in medicines development and regulatory science”. February 2, 2017. 7. Fleming N. Nature. “How artificial intelligence is changing drug discovery”. May 30, 2018. 8. Uyama Y. DIA Global Forum. “Utilizing Real World Data: A PMDA Perspective”. August 1, 2018. 9. Platt R, Brown JS, Robb M, McClellan M, Ball R, Nguyen MD, Sherman RE. N Engl J Med. “The FDA Sentinel Initiative - An Evolving National Resource”. November 29, 2018.